Hi,
The priority number of fontconfig config files in packages is hardcoded at this moment. this requires huge works to update, particularly when changing default fonts say (even though it won't so often happens but still when fixing config files and updating for huge packages etc). so I'm pondering to manage the priority in centralized package and determine it at runtime, allowing users to have own ordering.
I don't have any implementations for this but just to share an idea with you. Please let me know if you have any concerns on it.
The priority number of fontconfig config files in packages is hardcoded at this moment. this requires huge works to update, particularly when changing default fonts say (even though it won't so often happens but still when fixing config files and updating for huge packages etc). so I'm pondering to manage the priority in centralized package and determine it at runtime, allowing users to have own ordering.
Just some inputs. As an end user finding himself customizing fontconf files system wide on many machines, it would be very interesting to see how this will make setting font preferences simple.
Right now I mostly do the following: 1. rename 57 99 /etc/fonts/conf.d/57-dejavu* 2. vim /etc/fonts/conf.d/60-latin.conf #And set preferred serif/sans-serif/monospace fonts 3. # Observe that some other fonts are still preferred by doing `fc-match sans-serif` 4. # Either rename such fonts fontconf files as in step 1, or modify their ‘<prefer>’ tag to ‘<default>’ 5. # Rinse and repeat
I don't have any implementations for this but just to share an idea with you. Please let me know if you have any concerns on it.
Hi Akira,
The priority number of fontconfig config files in packages is hardcoded at this moment. this requires huge works to update, particularly when changing default fonts say (even though it won't so often happens but still when fixing config files and updating for huge packages etc). so I'm pondering to manage the priority in centralized package and determine it at runtime, allowing users to have own ordering.
I don't have any implementations for this but just to share an idea with you.
It’s great you’re looking at improving things! fontconfig has served us well all those years. But there are really a lot more free and open fonts nowadays than when fontconfig config syntax and file layout was defined (which is wonderful!). So things are cracking a bit at the seams, and not only on your side.
If I may give my opinion (CC-ing the fontconfig ML since I don't think any of this is Fedora-specific). Probably more than what you asked, but such is life ;)
1. fontconfig config syntax is too low level for the everyday needs of 2018. It was great when fonts on freedesktop systems were a small known pile of semi-broken files squirrelled away by xorg and texlive, it’s not so great to manage today’s huge numbers of standardized opentype files. The current syntax allows writing excruciately precise and accurate rules, except no human has the time to define those rules for every possible font file we deploy, so we get to write precisely things we are not so sure about.
2. so, this precise syntax should be kept to manage special cases, and something simpler and higher level should be introduced, with more things delegated to the fontconfig engine (like priorizing, as you rightly noted). Something like, process special precise rule in low- level fontconfig syntax first, then mass-process simplified high-level syntax.
3. the first higher level object should be a set-of-rules object, so ordering can be managed independently of file layout (basically you would order sets, regardless if the sets are each in their own file or share a single file, and regardless of the naming of those files. So stop shuffling and renaming files around to order things.
4. set ordering would be undefined and left to fontconfig by default, with one explicit human-maintained per-generic sets-that-must-go- first ordered list, and another per-generic sets-that-must-go-last list. And optional per-script overrides for those lists. Everything in between we don’t have the time and energy to order manually (and there’s no need to).
5. Of course, one copy of the lists in /usr/share/fontconfig for the system integrator, with an override in /etc for the sysadmin, and another override in .config for each user
6. IMHO to keep things manageable you should have one set = one font family (in the WWS sense, ie do not create as many sets as there are font family name variations, do not create separate sets for acmefont armenian and acmefont greek). We can extend font family definition to more things than just WWS when apps learn to manage more facets than just WWS for one font family, but at this point of time just doing WWS is taxing for the average app. Things like caption fonts, math fonts, etc should probably be merged with the WWS stuff eventually once we have more experience on handling those transparently in apps.
7. that does force fontconfig to manage high-level simplified and unified font family naming, instead of hoping font files will come with clean consistent naming (they don't and they won't, stop hoping for it, stop relying on fontconfig config writers to fix it, stop inflicting on users long lists of non-merged font names and expect them to jungle with the result, managing technical splits in font families is fontconfig's job not the user job, the user should only have to specify the design and the style he wants).
8. if there are several set definitions for the same font family at the same level of config (system integrator, sysadmin or user) just merge them transparently with undefined ordering rules when merging
9. if one set exists in several level, allow the override levels (sysadmin and user) to define in their set if they want it merged with lower levels or if they want it to replace the lower level definitions
10. The typical set should probably just define: * the generic the font family belongs to (what generic should be used to complete the font family, and what generic should ne constructed from the font family) * the list of known font families with similar design or metrics (so substitute this font family to those other font families if they are absent or lacking coverage, and conversely use those other font families to complete the font family if it lacks coverage). I really don’t think there is much benefit in ordering those manually. If there is some benefit, ordering should be an explicit attribute of the list, not the default.
11. Eventually, the set could also contain * an explicit list of internal hidden font families to map to this set (ie hide those font families from font lists, present only the unified clean name, manage internally in fontconfig which font file to use depending on what needs to be rendered) * the same thing, with also a face mapping, if face needs correction too * an explicit "use this internal font family for this script" to manage situations when there are regional variations on how to render an unicode codepoint
12. though I do hope opentype tables in shipped fonts get complete enough over time, and fontconfig gets smart enough over time, that all those mappings and overrides won’t need manual declaration long term. * IMHO fontconfig should map automatically by default everything defined in the WWS whitepaper, * and probably all the per-script/region/language common name variations
13. and since people will rightfully object that naming heuristics are not reliable and fail in some cases * allow defining a set with a property that basically says "do not merge me, I’m different" * make merging the default, and not-merging an explicit exception
With just that you should be able to handle 90% of what people need to declare to fontconfig today (handwaves). And have something easy to order at system, sysadmin or user level. The 10% of very specific and complex configuration that does not fit in this simplified model, can be handled with the current syntax. It’s small enough in volume that I doubt its causes you lots of management problems.
Best regards,
Hi,
On Wed, Oct 17, 2018 at 6:16 AM Nicolas Mailhot nicolas.mailhot@laposte.net wrote:
If I may give my opinion (CC-ing the fontconfig ML since I don't think any of this is Fedora-specific). Probably more than what you asked, but such is life ;)
Sure. well, this should eventually be merged and maintained in fontconfig upstream though, I have to research what the sort of config other distros uses and can be templated in general. so just thought starting on Fedora as pilot program would be good start, but anyway. more feedbacks from the wider audience would be great.
- fontconfig config syntax is too low level for the everyday needs of
- It was great when fonts on freedesktop systems were a small known
pile of semi-broken files squirrelled away by xorg and texlive, it’s not so great to manage today’s huge numbers of standardized opentype files. The current syntax allows writing excruciately precise and accurate rules, except no human has the time to define those rules for every possible font file we deploy, so we get to write precisely things we are not so sure about.
- so, this precise syntax should be kept to manage special cases, and
something simpler and higher level should be introduced, with more things delegated to the fontconfig engine (like priorizing, as you rightly noted). Something like, process special precise rule in low- level fontconfig syntax first, then mass-process simplified high-level syntax.
- the first higher level object should be a set-of-rules object, so
ordering can be managed independently of file layout (basically you would order sets, regardless if the sets are each in their own file or share a single file, and regardless of the naming of those files. So stop shuffling and renaming files around to order things.
- set ordering would be undefined and left to fontconfig by default,
with one explicit human-maintained per-generic sets-that-must-go- first ordered list, and another per-generic sets-that-must-go-last list. And optional per-script overrides for those lists. Everything in between we don’t have the time and energy to order manually (and there’s no need to).
- Of course, one copy of the lists in /usr/share/fontconfig for the
system integrator, with an override in /etc for the sysadmin, and another override in .config for each user
- IMHO to keep things manageable you should have one set = one font
family (in the WWS sense, ie do not create as many sets as there are font family name variations, do not create separate sets for acmefont armenian and acmefont greek). We can extend font family definition to more things than just WWS when apps learn to manage more facets than just WWS for one font family, but at this point of time just doing WWS is taxing for the average app. Things like caption fonts, math fonts, etc should probably be merged with the WWS stuff eventually once we have more experience on handling those transparently in apps.
- that does force fontconfig to manage high-level simplified and
unified font family naming, instead of hoping font files will come with clean consistent naming (they don't and they won't, stop hoping for it, stop relying on fontconfig config writers to fix it, stop inflicting on users long lists of non-merged font names and expect them to jungle with the result, managing technical splits in font families is fontconfig's job not the user job, the user should only have to specify the design and the style he wants).
- if there are several set definitions for the same font family at the
same level of config (system integrator, sysadmin or user) just merge them transparently with undefined ordering rules when merging
- if one set exists in several level, allow the override levels
(sysadmin and user) to define in their set if they want it merged with lower levels or if they want it to replace the lower level definitions
- The typical set should probably just define:
- the generic the font family belongs to (what generic should be used
to complete the font family, and what generic should ne constructed from the font family)
- the list of known font families with similar design or metrics (so
substitute this font family to those other font families if they are absent or lacking coverage, and conversely use those other font families to complete the font family if it lacks coverage). I really don’t think there is much benefit in ordering those manually. If there is some benefit, ordering should be an explicit attribute of the list, not the default.
- Eventually, the set could also contain
- an explicit list of internal hidden font families to map to this set
(ie hide those font families from font lists, present only the unified clean name, manage internally in fontconfig which font file to use depending on what needs to be rendered)
- the same thing, with also a face mapping, if face needs correction
too
- an explicit "use this internal font family for this script" to manage
situations when there are regional variations on how to render an unicode codepoint
- though I do hope opentype tables in shipped fonts get complete
enough over time, and fontconfig gets smart enough over time, that all those mappings and overrides won’t need manual declaration long term.
- IMHO fontconfig should map automatically by default everything
defined in the WWS whitepaper,
- and probably all the per-script/region/language common name
variations
- and since people will rightfully object that naming heuristics are
not reliable and fail in some cases
- allow defining a set with a property that basically says "do not
merge me, I’m different"
- make merging the default, and not-merging an explicit exception
With just that you should be able to handle 90% of what people need to declare to fontconfig today (handwaves). And have something easy to order at system, sysadmin or user level. The 10% of very specific and complex configuration that does not fit in this simplified model, can be handled with the current syntax. It’s small enough in volume that I doubt its causes you lots of management problems.
For individuals, majority requirements should be satisfied by a tool such as fonts-tweak-tool. for system-wide and complicated scenarios, we need a handy way to get things done. for Fedora specific thing, I eventually want to integrate this into rpm macro and prepare an environment to have config files without any knowledge or minimal. that would helps a lot for packagers.
Best regards,
-- Nicolas Mailhot
-- Akira TAGOH
Le 2018-10-17 13:43, Akira TAGOH a écrit :
Hi,
On Wed, Oct 17, 2018 at 6:16 AM Nicolas Mailhot nicolas.mailhot@laposte.net wrote:
If I may give my opinion (CC-ing the fontconfig ML since I don't think any of this is Fedora-specific). Probably more than what you asked, but such is life ;)
Sure. well, this should eventually be merged and maintained in fontconfig upstream though, I have to research what the sort of config other distros uses and can be templated in general. so just thought starting on Fedora as pilot program would be good start, but anyway. more feedbacks from the wider audience would be great.
You seem to be thinking about generating current fontconfig syntax from templates in rpm macros. I definitively don't want to go there in Fedora font packages. Such generators are quite brittle and hell to debug/maintain. They're sort of ok for third party tools that try to coax fontconfig on doing things, they're not ok as distro core infrastructure. If you want to generate from rpm macros, you need to define something that can be generated simply (and current fontconfig syntax is not that).
If we agree on some form of simplified syntax, sufficient for the needs of the average font packager, I want the result to be read by fontconfig natively without intermediary generation. Even if it's in an experimental config namespace only activated on Fedora, that we agree can be dropped later if the experiment fails.
For individuals, majority requirements should be satisfied by a tool such as fonts-tweak-tool. for system-wide and complicated scenarios, we need a handy way to get things done. for Fedora specific thing, I eventually want to integrate this into rpm macro and prepare an environment to have config files without any knowledge or minimal. that would helps a lot for packagers.
Well I wrote down what I though minimal knowledge could be: identifying the generic (that's definitely something a human can do better than fontconfig), listing down font families that the font family can serve as substitute for/can be substituted with (fontconfig can not have any opinion on fonts families not present on system) and listing down all the font family name variations that should be merged, if fontconfig is not smart enough to identify them itself.
The merging part is hugely important from a usability POW, because the OFL has created a situation where forks lead to renames, and because font designers suck at font naming.
They will rename every possible variation of their fonts so they can wash their hands on metric compatibility, or just for marketing purposes (acmefont new better), or to enable subsetting, or to sell a pro version, but the average user is not going the have every possible font variation on his system.
So it's much better to present all the variations on acmefont as a single acmefont font family, and have fontconfig manage internally those variants, than have fontconfig fallback to an unrelated font, because the original document didn't use the same variant as the current system. These days displaying all acmefont family names in selectors is about as useful as displaying the font version embedded in the font files. Sure it allows disambiguation. But who except the original font designer cares about this? If you have the exact match on-system for one of the font other names, by all means have fontconfig return data from this file, but don't force users to do the untangling manually. They don't care enough to do it. They will just (rightly) say fontconfig font selection sucks.
On Wed, Oct 17, 2018 at 10:03 PM Nicolas Mailhot nicolas.mailhot@laposte.net wrote:
You seem to be thinking about generating current fontconfig syntax from templates in rpm macros. I definitively don't want to go there in Fedora font packages. Such generators are quite brittle and hell to debug/maintain. They're sort of ok for third party tools that try to coax fontconfig on doing things, they're not ok as distro core infrastructure. If you want to generate from rpm macros, you need to define something that can be generated simply (and current fontconfig syntax is not that).
How is it that hell to debug/maintain? if you look at the outcome, that should be obvious and what I'm planning to do isn't more than what we currently do by the hand. just thought integrating it into fontpackages' rpm macro because most of fonts packages uses it but sometimes missing config files. indeed fontconfig eventually should detect generic families without any explicit rules and that is a long standing issue for me though.
If we agree on some form of simplified syntax, sufficient for the needs of the average font packager, I want the result to be read by fontconfig natively without intermediary generation. Even if it's in an experimental config namespace only activated on Fedora, that we agree can be dropped later if the experiment fails.
I'm not sure how current recipes can be simplified though. if there are any useful sugar forms, I'm willing to add it into fontconfig if that helps something.
For individuals, majority requirements should be satisfied by a tool such as fonts-tweak-tool. for system-wide and complicated scenarios, we need a handy way to get things done. for Fedora specific thing, I eventually want to integrate this into rpm macro and prepare an environment to have config files without any knowledge or minimal. that would helps a lot for packagers.
Well I wrote down what I though minimal knowledge could be: identifying the generic (that's definitely something a human can do better than fontconfig), listing down font families that the font family can serve as substitute for/can be substituted with (fontconfig can not have any opinion on fonts families not present on system) and listing down all the font family name variations that should be merged, if fontconfig is not smart enough to identify them itself.
Sure. well, I meant about syntax-wise for fontconfig config.
The merging part is hugely important from a usability POW, because the OFL has created a situation where forks lead to renames, and because font designers suck at font naming.
Right.
They will rename every possible variation of their fonts so they can wash their hands on metric compatibility, or just for marketing purposes (acmefont new better), or to enable subsetting, or to sell a pro version, but the average user is not going the have every possible font variation on his system.
So it's much better to present all the variations on acmefont as a single acmefont font family, and have fontconfig manage internally those variants, than have fontconfig fallback to an unrelated font, because the original document didn't use the same variant as the current system. These days displaying all acmefont family names in selectors is about as useful as displaying the font version embedded in the font files. Sure it allows disambiguation. But who except the original font designer cares about this? If you have the exact match on-system for one of the font other names, by all means have fontconfig return data from this file, but don't force users to do the untangling manually. They don't care enough to do it. They will just (rightly) say fontconfig font selection sucks.
Sure. that should be improved but sounds like another topic or in the later milestones for this idea so far. but yes, definitely.
Anyway, thanks for feedbacks!
-- Nicolas Mailhot
Le 2018-10-18 08:04, Akira TAGOH a écrit :
On Wed, Oct 17, 2018 at 10:03 PM Nicolas Mailhot nicolas.mailhot@laposte.net wrote:
You seem to be thinking about generating current fontconfig syntax from templates in rpm macros. I definitively don't want to go there in Fedora font packages. Such generators are quite brittle and hell to debug/maintain. They're sort of ok for third party tools that try to coax fontconfig on doing things, they're not ok as distro core infrastructure. If you want to generate from rpm macros, you need to define something that can be generated simply (and current fontconfig syntax is not that).
How is it that hell to debug/maintain?
Well, basically, some of the things you need to pass to fontconfig are structured data (for example, lists of font families a font family can substitute/be substituted). And rpm has no notion of structured data, just individual variables. So sure, you can hack something at the rpm macro level, but it will be ugly and awkward to use, unless you mandate the use of a separate config file that uses a syntax adapted to structured data (xml, json, yaml, whatever).
And when you get to this point, giving the config file to fontconfig directly, makes loads more sense. That means you can change what you do with the config files in a central point, post deployment even, instead of needing to rebuild and reinstall every single font package every time you have a better idea on how things should be done.
Regards,
-
Akira TAGOH -
Nicolas Mailhot -
Rajeesh K V