Fonts packaging policy rewrite proposal
by Nicolas Mailhot
Hi,
A fonts packaging policy rewrite proposal has been pushed to FPC today:
https://pagure.io/packaging-committee/pull-request/934
It should be clearer, more opinionated, and take into account:
– updates of The OpenType standard
– variable fonts
– web fonts
– upstream depreciation of non OpenType formats: final stages of the
Harfbuzz consolidation decided at the 2006 Text Layout summit
https://www.freedesktop.org/wiki/TextLayout/
– appstream & fonts
– weak dependencies
– and probably more I forget here
It is based on the new fonts-rpm-macros project for automation:
This project builds on tooling enhancements in redhat-rpm-config and rpm
itself, done during the past two years for the Forge and Go sets of
packaging macros. It started 2 years ago as a fork of fontpackages,
which is the core of our current fonts packaging guidelines.
It will require putting the fonts-srpm-macros package in the default
build root, like is done for other domain-specific packaging macro
sets.
Major additions:
– better documentation (clearer and more complete)
– better automation (less packager hassle for better and more complete
results)
Major removals:
– tools and scripts
– fixing metadata with ttname
Mostly because no one seems willing to maintain those scripts, or port
ttname to python 3.
https://copr.fedorainfracloud.org/coprs/nim/fonts-rpm-macros/builds/
showcases the new policy on 62 real-world source packages, generating
139 installation packages. Some of those are badly delayed updates to
Fedora packages, others are brand-new packages ready for Fedora
inclusion. They include major font packages such as Stix, DejaVu, Droid,
IBM Plex.
Existing Fedora packages will continue to build, the old fontpackages
macros are grandfathered in fonts-rpm-macros for now. They will be
removed in a few years to give packagers time to apply the new
guidelines.
Regards,
--
Nicolas Mailhot
3 years, 8 months
Re: [Fontconfig] Fontconfig language update proposal
by Nicolas Mailhot
Le 2019-11-28 12:19, Akira TAGOH a écrit :
> On Thu, Nov 28, 2019 at 6:36 PM Nicolas Mailhot
> <nicolas.mailhot(a)laposte.net> wrote:
>>
>> Le 2019-11-28 09:48, Akira TAGOH a écrit :
>> > On Thu, Nov 28, 2019 at 4:07 PM Nicolas Mailhot
>> > <nicolas.mailhot(a)laposte.net> wrote:
>> >> You're already doing it for
>> >>
>> >> <fontvariations>AXIS1=VALUE;AXIS2=VALUE</fontvariations>
>> >>
>> >> Except it's neither an XML nor a YAML list, it’s a one-of-a-kind dict
>> >> format that can not be parsed natively by any off-the-shelf XML or
>> >> YAML
>> >> parser.
>> >
>> > That is just a string *in* fontconfig.
>>
>> It’s not just a string *in* fontconfig. It affects styles. style
>> manipulation is part of fontconfig rules. Except that here it relies
>> on
>> an un-parsable dict (without specific code).
>
> Well, it *is*, at least at this moment. and you are misunderstanding
> on it. we don't have any agreements on how fontvariations property can
> be parameterized in configuration yet.
Ok, I misunderstood, sorry. The examples in the proposal show the same
need as the requester at the config level (human setting not library
setting).
>> I would leave the exact rewriting strategy to the engine, so a single
>> *uniform* rewriting strategy was applied to all fonts on the system,
>> and
>> every font packager need not specify all of the nitty gritty details
>> by
>> hand every time there is a new font file to add, and need no check
>> that
>> other font files were processed with the same nitty gritty details.
>
> I see, but I'm wondering if it is really possible,
It is really possible. Font creators make the same mistakes in all font
projects and they can all be fixed the same way. In that sense modern
complex fonts projects are really not interesting: they contain the same
mistakes that 20 years old projects. Except that now there is a lot more
font files to make mistakes in, we have more font files to fix.
> particularly doubt
> as long as we have something in configuration. we could reduce the
> amount of *characters* in a file to modify perhaps but always need to
> check if it is valid or not when font vendors/authors has a new
> release.
Human time is limited. Sure, checking the result does not go away. Not
having to comb thousands of conf lines for synbax or declaration
problems frees up time to actually check the result.
>
>> It’s not remotely the same thing. The alternatives you talk about
>> require changing and maintaining thousands of low-level instructions
>> in
>> config files. Those low-level instructions interact with thousands of
>> other low-level instructions with complex prioritization rules (not
>> prioritization rules handled engine side, prioritization rules that
>> happen as a side-effect of the ordering of the low-level
>> instructions).
>
> I'm sorry, I missed your point again. If I'm not missing the
> discussion here, we were talking about locale-specific recipe in
> configuration, particularly my proposal to drop unexpected language
> coverage from caches instead of checking the desired language at
> runtime, right?
> There are no prioritization rules there.
Lang block selection interacts with prioritization both at the font
family and at the generic level. It's inherently a prioritization
problem.
> Well, I meant to be "generic syntax vs dedicated syntax". as a
> contrast, "simple and dedicated syntax" is what you want, no? syntax
> sugars are dedicated though.
I want simple syntax to handle generic problems. It's not “dedicated”
syntax. The same problems occur again and again in most font projects
regardless of their foundry.
Regards,
--
Nicolas Mailhot
4 years
Re: [Fontconfig] Fontconfig language update proposal
by Nicolas Mailhot
Le 2019-11-28 09:48, Akira TAGOH a écrit :
> On Thu, Nov 28, 2019 at 4:07 PM Nicolas Mailhot
> <nicolas.mailhot(a)laposte.net> wrote:
>> You're already doing it for
>>
>> <fontvariations>AXIS1=VALUE;AXIS2=VALUE</fontvariations>
>>
>> Except it's neither an XML nor a YAML list, it’s a one-of-a-kind dict
>> format that can not be parsed natively by any off-the-shelf XML or
>> YAML
>> parser.
>
> That is just a string *in* fontconfig.
It’s not just a string *in* fontconfig. It affects styles. style
manipulation is part of fontconfig rules. Except that here it relies on
an un-parsable dict (without specific code).
>> That's 100% due to a bad abstraction level in fontconfig, where
>> instead
>> of telling the engine what locale a font file is good for (letting the
>> engine compute the appropriate priorization strategy), users have to
>> hardcode a specific handling strategy in their config files.
>
> Hmm, I'm still not clear what you are trying... in that sense, what
> you are proposing would also introduce rewriting.
I would leave the exact rewriting strategy to the engine, so a single
*uniform* rewriting strategy was applied to all fonts on the system, and
every font packager need not specify all of the nitty gritty details by
hand every time there is a new font file to add, and need no check that
other font files were processed with the same nitty gritty details.
Rewriting is forced on us by the way foundries release font files. We’ve
waited two decades for foundries to apply the OpenType naming
recommendations that would make a lot of rewriting unnecessary. Well,
foundries didn't and won't apply those consistently. So rewriting needs
to happen fontconfig side.
Without rewriting font substitution does not work, even within a single
upstream font project, because of small discrepancies in upstream font
files.
> it may be "only
> once", but the above changes you referenced may be "only once" as
> well.
It’s not remotely the same thing. The alternatives you talk about
require changing and maintaining thousands of low-level instructions in
config files. Those low-level instructions interact with thousands of
other low-level instructions with complex prioritization rules (not
prioritization rules handled engine side, prioritization rules that
happen as a side-effect of the ordering of the low-level instructions).
The result is an house of cards. It is not viable long term. Next time
the OpenType spec adds some twists, or experience shows the low-level
stuff needs tweaking, those thousands of instructions will need changing
accordingly.
My proposal shrinks the configuration to the bare minimum human info
fontconfig does not autodetect today. It can only shrink further, as
fontconfig gets smarter. It does not suffer from all the low-level
prioritization interactions crap, because the config files posit a
common handling strategy engine-side, instead of considering that every
single font file on system needs a different management strategy (which
is completely insane, current fontconfig syntax makes exceptions the
general case, instead of keeping them as exceptions).
You need to evaluate fontconfig syntax a the system level, not the
individual instruction level. The low-level instructions work in
isolation. They do not work as a whole. You're asking fontconfig users
to write things in fontconfig bytecode language, instead of writing
things at the level humans expect.
And I write this as someone, who invested years understanding the
fontconfig bytecode. I am *not* a normal user. Normal users are even
*less* likely to learn and use this low-level stuff than me.
That wouldn't matter, if foundries released quality font files, that
didn’t need fixing. They do not. It's time to accept reality and move
on.
> Again, there shouldn't be that difference between them. that would be
> "complicated vs simple" or "generic vs dedicated". both have pros and
> cons.
Stop here. generic is the only way forward.
The average free desktop systems uses hundreds if not thousands of font
files. It's been a *very* long time since a free desktop was limited to
the same handful of xorg font files.
Do you want me to count the number of font files in Plex alone? In
Droid? In Fira? In Noto?
You need to go generic or give up on fontconfig as the system font
manager.
There are no magical fairies out there that will decline a generic
strategy for all the font files installed on free desktop systems, using
the hundreds of configuration lines the current syntax requires.
Regards,
--
Nicolas Mailhot
4 years
Re: [Fontconfig] Fontconfig language update proposal
by Nicolas Mailhot
Le jeudi 28 novembre 2019 à 14:59 +0900, Akira TAGOH a écrit :
>
> > I suppose it would be possible to define some XML elements, that
> > contain
> > a yaml list or dict in the text node. I don't like this kind of
> > mixed
> > syntax much, but that’s the only way I see to keep something XML-
> > ish
> > that scales humanly to the kind of list/dict we will need.
>
> That's too bad. I don't want to mix them.
You're already doing it for
<fontvariations>AXIS1=VALUE;AXIS2=VALUE</fontvariations>
Except it's neither an XML nor a YAML list, it’s a one-of-a-kind dict
format that can not be parsed natively by any off-the-shelf XML or YAML
parser.
> >
> > 2. whenever hindsight shows the low-level sequence needed to
> > achieve a
> > generic goal needs tweaking, the whole configuration needs
> > rewriting,
> > instead of just adjusting things at the engine level.
>
> I'm sorry, I don't get it. can you elaborate more details and
> specifics?
The whole locale-specific debacle for example, where you've wanted for
years to try alternative approaches, and you can't because neitheir you
nor other font packagers have any wish to rewrite all existing files.
That's 100% due to a bad abstraction level in fontconfig, where instead
of telling the engine what locale a font file is good for (letting the
engine compute the appropriate priorization strategy), users have to
hardcode a specific handling strategy in their config files.
Regards,
--
Nicolas Mailhot
4 years
Fontconfig language update proposal
by Nicolas Mailhot
Hi all,
CONTEXT
I've now spent several years redoing font packaging automation in
Fedora, culminating with this guideline change proposal:
https://pagure.io/packaging-committee/pull-request/934
It should soon be dead easy to package industrial quantities of nice
OFL fonts Fedora-side, without giving up on quality, right?
Unfortunately, no. Font files have gotten a lot more complex in the
past decade. But font authors are no more careful than they were in the
past. Thus, the amount of fixing required fontconfig-side has grown.
However, the fontconfig language is little more expressive today, than
it was a decade ago. Despite Fedora shipping fontconfig templates
maintained by Akira himself, our fontconfig state is, at best, poor.
PROPOSAL
I would like the fontconfig project to consider a configuration
language update based on the following proposal:
https://gitlab.freedesktop.org/fontconfig/fontconfig/issues/200
It is based on the experience, gained, analysing hundreds of Fedora
fontconfig files, while streamlining the corresponding packaging.
This update is a mix of:
– syntax sugar enhancements
– tweaked fontconfig engine behaviour
– new features
It is presented both in traditional XML, and in YAML.
The proposal is only intended as documentation of the kind of thing
that would be useful to fontconfig users. I don’t care if this exact
syntax is adopted, or if it’s only partially implemented. Please
provide something that does the kind of thing I documented.
The current fontconfig syntax is awkward for simple traditional font
projects, and does not scale at all to the level required for modern
complex font project.
CONCISENESS
The proposal results in drastically more succinct config files, which
are easier to write, review and maintain. Conciseness is its own
feature.
For example, the description of IBM Plex Sans takes 106 lines in the
XML variant, and 27 lines in the YAML Variant. That may seem awful,
but my current incomplete attempt to describe Plex Sans with today’s
fontconfig syntax is more than 1700 lines long.
Given what I know of today’s fontconfig syntax, this attempt will never
achieve what’s in the proposal. The necessary logic is missing in the
fontconfig engine. It's not just a templating problem.
And, the missing logic is not complex. fontconfig is very close to
where it should be engine-side. Little things just snowball out of
control in presence of complex font projects.
Plex is not a pathological font project. On the contrary, it’s well
funded, and its authors clearly try to do the right thing. But, Plex
is modern, complex font project. That leaves a lot of room for small
mistakes.
ORDERING INDEPENDANCE
The proposal results in largely order-independant config files,
removing the need to agonize over proper ordering and ordering side-
effects, and removing the need to split configuration for a font family
over multiple files (that required specifying engine behaviour tweaks).
CORRECT ABSTRACTION LEVEL
The proposal avoids over-specifying low-level fontconfig behaviour,
leaving room to tweak the logic when new lessons are learnt, without
requiring the rewrite of thousands of configuration lines spread over
hundreds of files.
DOING THE CORRECT THING BY DEFAULT
The proposal makes some fixes automatic (generally, just applying
OpenType recommendations), so they're not forgotten half the time, and
we can all focus on more interesting things.
COMPATIBILITY
The proposal is backwards-compatible.
I hope you’ll find it interesting.
Best regards,
--
Nicolas Mailhot
4 years
Re: [Fontconfig] Fontconfig language update proposal
by Nicolas Mailhot
Le 2019-11-26 10:47, Akira TAGOH a écrit :
Hi Akira,
> Ideally fonts would be available in fontconfig with minimal effort,
> and writing a configuration file to make up for missing pieces. we may
> need to figure out if we really can't implement it as a core feature
> of fontconfig and have to leave it to the users, to let them write a
> configuration.
On the long term, fontconfig should automate all the guesswork
heuristics that humans use to make sense of the pile of approximative
metadata foundries release. There’s no reason the heuristics will work
worse when automated than when done as a manual process.
This proposal is a lot more modest. It's just streamlining the current
dumb fontconfig mode, where users provide the information needed by the
engine, without any engine smarts.
Once this streamlining is finished it will be much clearer, where
fontconfig needs human help, and where more smarts is needed
engine-side.
> On Sun, Nov 24, 2019 at 4:30 AM Nicolas Mailhot
> <nicolas.mailhot(a)laposte.net> wrote:
>> It is presented both in traditional XML, and in YAML.
>
> I'm not keen on writing a configuration in YAML. particularly I don't
> think current features in XML can be represented in YAML to write some
> procedures and expressions.
Raw XML is very poor to express lists and dicts of things natively. And
the new large-encoding font projects need those lists and dict in the
config (for example, good lang lists, or variable axis step
declarations).
I suppose it would be possible to define some XML elements, that contain
a yaml list or dict in the text node. I don't like this kind of mixed
syntax much, but that’s the only way I see to keep something XML-ish
that scales humanly to the kind of list/dict we will need.
>> CORRECT ABSTRACTION LEVEL
>>
>> The proposal avoids over-specifying low-level fontconfig behaviour,
>> leaving room to tweak the logic when new lessons are learnt, without
>> requiring the rewrite of thousands of configuration lines spread over
>> hundreds of files.
>
> To be sure, about the line of "without requiring the rewrite of...",
> which one particularly were you talking about? and which one in your
> proposal will improve that?
Pretty much all its parts.
The proposal specifies font elements to group, with their associated
characteristics (good/bad lang, broken styles, etc). What fontconfig
does with this info can change over time.
That‘s a departure from the current level of abstraction, where humans
directly set all the engine low-level actions. The current state has the
following consequences:
1. it’s not possible to reconstruct reliably, the intent behind a
configuration. That means, a CI/CD system can not deduce, the objective,
behind a config file (and therefore, check it is attained)
2. whenever hindsight shows the low-level sequence needed to achieve a
generic goal needs tweaking, the whole configuration needs rewriting,
instead of just adjusting things at the engine level.
Regards,
--
Nicolas Mailhot
4 years
Proposal to revise the locale specific overrides rule in font
packages tips
by Akira TAGOH
Hi,
* Summary
Let's drop unnecessary laguages from fontconfig cache instead of
checking it at runtime.
* Background
We have a template[1] to apply for certain language only. this is
necessary for languages particularly which has similar coverage like
CJK and Arabic etc.
However this evaluation always happens when applications are going to
query a font with API. the one might be trivial but it might be not if
a lot of queries happens.
* Replacement
I'd propose to drop those unnecessary and problematic languages for a
font from fontconfig cache instead of testing a language at runtime.
it can be done like:
<match target="scan">
<test name="family">
<string>font name</string>
</test>
<edit name="lang" mode="assign">
<minus>
<name>lang</name>
<langset>
<string>xx</string>
...
</langset>
</minus>
</edit>
</match>
This is evaluated only when creating/updating fontconfig caches by
fc-cache. then the targeted cache doesn't have it and matcher won't
pick it up for a certain language anymore.
Well, the recipe is a bit complicated. we may need a syntax sugar for
that though.
Any thought?
[1].. https://fedoraproject.org/wiki/Fontconfig_packaging_tips#Locale-specific_...
--
Akira TAGOH
4 years
Fonts packaging policy rewrite proposal
by Nicolas Mailhot
Hi,
A fonts packaging policy rewrite proposal has been pushed to FPC today:
https://pagure.io/packaging-committee/pull-request/934
It should be clearer, more opinionated, and take into account:
– updates of The OpenType standard
– variable fonts
– web fonts
– upstream depreciation of non OpenType formats: final stages of the
Harfbuzz consolidation decided at the 2006 Text Layout summit
https://www.freedesktop.org/wiki/TextLayout/
– appstream & fonts
– weak dependencies
– and probably more I forget here
It is based on the new fonts-rpm-macros project for automation:
This project builds on tooling enhancements in redhat-rpm-config and
rpm itself, done during the past two years for the Forge and Go sets of
packaging macros. It started 2 years ago as a fork of fontpackages,
which is the core of our current fonts packaging guidelines.
It will require putting the fonts-srpm-macros package in the default
build root, like is done for other domain-specific packaging macro
sets.
Major additions:
– better documentation (clearer and more complete)
– better automation (less packager hassle for better and more complete
results)
Major removals:
– tools and scripts
– fixing metadata with ttname
Mostly because no one seems willing to maintain those scripts, or port
ttname to python 3.
https://copr.fedorainfracloud.org/coprs/nim/fonts-rpm-macros/builds/
showcases the new policy on 62 real-world source packages. Some of
those are badly delayed updates to Fedora packages, others are brand-new packages ready for Fedora inclusion. They include major font packages such as Stix, DejaVu, Droid, IBM Plex.
Existing Fedora packages will continue to build, the old fontpackages
macros are grandfathered in fonts-rpm-macros for now. They will be
removed in a few years to give packagers time to apply the new
guidelines.
Regards,
--
Nicolas Mailhot
4 years