Langpacks and the packages needed to display/input a language

[Jason L Tibbitts III]

I noticed that my F30 installs are coming out far larger than my F29
installs (by 3GB or so) and did some digging into why.

With F30 we switched away from having groups named like "korean-support"
that you could install to get input methods and fonts needed to display
a language and instead we have metapackages named like "langpacks-ko".
These metapackages have (generally) weak dependencies on the fonts and
input methods as before. But other packages have reverse weak
dependencies on the langpacks, which causes far more to get pulled in
than was previously installed.  For example, each libreoffice langpack
has a "supplements" weak reverse dependency on the base "langpacks"
metapackage.

All of this seems fine, but my original goal was to be able to properly
display, and perhaps input, various languages.  But now I get
translations and help files and such as well.  Not just for libreoffice,
but for eclipse, glibc, all of KDE as well.  And I also get
autocorrection rules, spelling dictionaries, hyphenation rules, and
terreract OCR recognition data as well.  Some of those aren't small, and
the end result is that I need to bump up the size of / quite a bit.

Note that turning off install_weak_deps is not an option because for
most of the langpacks, _all_ of the langpack are weak.  (Some do have
hard font dependencies, and I'm not sure if this inconsistency is
intentional.)

So it seems we lost the simple "here are our suggested Korean fonts and
an input method" and instead the only thing you can say is "I want
everything possible to be available in Korean".  Is there any way to
improve the granularity here?  Perhaps by having "light" and "heavy"
langpacks, or splitting them by usage (translations versus simple
display of text)?

For now I guess I will simply extract the list of fonts and input
methods I want from the langpack specfile and stop installing the actual
langpack packages.

 - J<

[Sundeep Anand]

On Sat, Jun 15, 2019 at 3:47 AM Jason L Tibbitts III <tibbs(a)math.uh.edu&gt;
wrote:

>  I noticed that my F30 installs are coming out far larger than my F29
>  installs (by 3GB or so) and did some digging into why.
>
>  With F30 we switched away from having groups named like "korean-support"
>  that you could install to get input methods and fonts needed to display
>  a language and instead we have metapackages named like "langpacks-ko".
>  These metapackages have (generally) weak dependencies on the fonts and
>  input methods as before. But other packages have reverse weak
>  dependencies on the langpacks, which causes far more to get pulled in
>  than was previously installed.  For example, each libreoffice langpack
>  has a "supplements" weak reverse dependency on the base "langpacks"
>  metapackage.
>
>  All of this seems fine, but my original goal was to be able to properly
>  display, and perhaps input, various languages.  But now I get
>  translations and help files and such as well.  Not just for libreoffice,
>  but for eclipse, glibc, all of KDE as well.  And I also get
>  autocorrection rules, spelling dictionaries, hyphenation rules, and
>  terreract OCR recognition data as well.  Some of those aren't small, and
>  the end result is that I need to bump up the size of / quite a bit.
>
>  Note that turning off install_weak_deps is not an option because for
>  most of the langpacks, _all_ of the langpack are weak.  (Some do have
>  hard font dependencies, and I'm not sure if this inconsistency is
>  intentional.)
>
>  So it seems we lost the simple "here are our suggested Korean fonts and
>  an input method" and instead the only thing you can say is "I want
>  everything possible to be available in Korean".  Is there any way to
>  improve the granularity here?  Perhaps by having "light" and "heavy"
>  langpacks, or splitting them by usage (translations versus simple
>  display of text)?
>
>  
Agree! We probably need more discussions (I'm quite interested to work on
this). This is one of the points for https://pagure.io/flock/issue/164


>  For now I guess I will simply extract the list of fonts and input
>  methods I want from the langpack specfile and stop installing the actual
>  langpack packages.
>
>   - J<
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>

[Julen Landa Alustiza]

I don't have a clear opinion on this, yet.

In my use case I totally agree with you, I don't really need all this support, just a
few thing are enough for me.

But I understand that for the general non english speaker, non techie use case, they
expect  that selecting to install their desired language once they system installs you all
the possible support for that language.

Could we go with the full metapackage on all general UI (anaconda, DE system settings and
similars) and provide a minimal metapackage at the same time?

On June 15, 2019 4:12:51 AM GMT+02:00, Sundeep Anand <suanand(a)redhat.com&gt; wrote:
On Sat, Jun 15, 2019 at 3:47 AM Jason L Tibbitts III
<tibbs(a)math.uh.edu&gt;
wrote:

 I noticed that my F30 installs are coming out far larger than my F29
 installs (by 3GB or so) and did some digging into why.

 With F30 we switched away from having groups named like
"korean-support"
that you could install to get input methods and fonts needed to
display
a language and instead we have metapackages named like
"langpacks-ko".
These metapackages have (generally) weak dependencies on the fonts
and
input methods as before. But other packages have reverse weak
dependencies on the langpacks, which causes far more to get pulled in
than was previously installed.  For example, each libreoffice
langpack
 has a "supplements" weak reverse dependency on the base "langpacks"
 metapackage.

 All of this seems fine, but my original goal was to be able to
properly
display, and perhaps input, various languages.  But now I get
translations and help files and such as well.  Not just for
libreoffice,
but for eclipse, glibc, all of KDE as well.  And I also get
autocorrection rules, spelling dictionaries, hyphenation rules, and
terreract OCR recognition data as well.  Some of those aren't small,
and
 the end result is that I need to bump up the size of / quite a bit.

 Note that turning off install_weak_deps is not an option because for
 most of the langpacks, _all_ of the langpack are weak.  (Some do have
 hard font dependencies, and I'm not sure if this inconsistency is
 intentional.)

 So it seems we lost the simple "here are our suggested Korean fonts
and
 an input method" and instead the only thing you can say is "I want
 everything possible to be available in Korean".  Is there any way to
 improve the granularity here?  Perhaps by having "light" and "heavy"
 langpacks, or splitting them by usage (translations versus simple
 display of text)?


Agree! We probably need more discussions (I'm quite interested to work
on
this). This is one of the points for https://pagure.io/flock/issue/164


For now I guess I will simply extract the list of fonts and input
methods I want from the langpack specfile and stop installing the
actual
 langpack packages.

  - J<
 devel mailing list -- devel(a)lists.fedoraproject.org
 To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
 Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
 List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
 List Archives:

https://lists.fedoraproject.org/archives/list/[e-mail]


-- 
Julen Landa Alustiza <julen(a)landa.eus&gt;
Julen Landa Alustiza <jlanda(a)fedoraproject.org&gt;

[Björn Persson]

Julen Landa Alustiza wrote:
>  But I understand that for the general non english speaker, non techie
> use case, they expect  that selecting to install their desired language once they system
> installs you all the possible support for that language. 
Of course. But complete support for one's desired language is one
thing, and the ability to display a foreign language is another.

If a few words in a foreign script appear on a web page or in an email,
then I want to see them. Even if I can't read a word in that language,
I still prefer to see the characters rather than a string of boxes.

I think a typical desktop system should have at least one font installed
for every language on Earth (and an ability to remove some of them if
the user needs the space for other things). Complete language support
should be installed only for the user's one or two preferred languages.
The latter would include GUI translations, documentation, dictionaries,
a wider selection of fonts, et cetera.

Björn Persson

[Parag Nemade]

Hi Jason,

On Sat, Jun 15, 2019 at 4:35 AM Jason L Tibbitts III <tibbs(a)math.uh.edu&gt;
wrote:

>  I noticed that my F30 installs are coming out far larger than my F29
>  installs (by 3GB or so) and did some digging into why.
>
>  With F30 we switched away from having groups named like "korean-support"
>  that you could install to get input methods and fonts needed to display
>  a language and instead we have metapackages named like "langpacks-ko".
>  These metapackages have (generally) weak dependencies on the fonts and
>  input methods as before. But other packages have reverse weak
>  dependencies on the langpacks, which causes far more to get pulled in
>  than was previously installed.  For example, each libreoffice langpack
>  has a "supplements" weak reverse dependency on the base "langpacks"
>  metapackage.
>
>  All of this seems fine, but my original goal was to be able to properly
>  display, and perhaps input, various languages.  But now I get
>  translations and help files and such as well.  Not just for libreoffice,
>  but for eclipse, glibc, all of KDE as well.  And I also get
>  autocorrection rules, spelling dictionaries, hyphenation rules, and
>  terreract OCR recognition data as well.  Some of those aren't small, and
>  the end result is that I need to bump up the size of / quite a bit.
>
>  Note that turning off install_weak_deps is not an option because for
>  most of the langpacks, _all_ of the langpack are weak.  (Some do have
>  hard font dependencies, and I'm not sure if this inconsistency is
>  intentional.)
>
>  So it seems we lost the simple "here are our suggested Korean fonts and
>  an input method" and instead the only thing you can say is "I want
>  everything possible to be available in Korean".  Is there any way to
>  improve the granularity here?  Perhaps by having "light" and "heavy"
>  langpacks, or splitting them by usage (translations versus simple
>  display of text)?
>
>  For now I guess I will simply extract the list of fonts and input
>  methods I want from the langpack specfile and stop installing the actual
>  langpack packages.
>  
Unfortunately this is the only option available for you. Remember langpacks
meta-packages are not pulling unnecessary language packages. You have
required base packages already installed on your system that is why when
you try to install langpacks-ko, you are getting all the respective Korean
language related packages. I never thought someone who want to use dekstop
in some language also will prefer not to have all language related packages
installed on his system.

If users think splitting (e.g.) langpacks-ko package into langpacks-i18n-ko
and langpacks-l10n-ko packages will make sense for them then someone please
report a bug against langpacks package and i18n group will discuss on it.

Regards,
Parag

[Nicolas Mailhot]

Le samedi 15 juin 2019 à 20:13 +0200, Julen Landa Alustiza a écrit :
>  I don't have a clear opinion on this, yet.
>  
>  In my use case I totally agree with you, I don't really need all this
>  support, just a few thing are enough for me.
>  
>  But I understand that for the general non english speaker, non techie
>  use case, they expect  that selecting to install their desired
>  language once they system installs you all the possible support for
>  that language.
>  
>  Could we go with the full metapackage on all general UI (anaconda, DE
>  system settings and similars) and provide a minimal metapackage at
>  the same time? 
The problem is not minimalism. The problem is conflating two concepts:
being able to process a language (fonts, input methods, spellcheckers)
and being able to switch UI to this language.

The two things should be managed by separate metapackages set, with (at
most) a weak dep from UI to process, and never from process to UI.

Much like our font package guidelines were carefully crafted a decade
ago to identify a functional font family unit (the thorough font
landscape analysis Microsoft wrote for WPF helped defined that unit),
forbidding the kind of app or font referse dep that results in
everything depending on everything else.

And people have started ignoring those guidelines, with the predicted
resulting mess, that is entirely self-inflicted.

-- 
Nicolas Mailhot

[Jens-Ulrik Petersen]

Jason, can you explain in more details (bug report is also fine) how
exactly you are installing?

(Because for both Workstation/Silverblue, and Server I believe, we install
fonts and input methods for Korean (and other langs like Japanese) by
default anyway - so actually no need to install langpacks-ko currently if
you don't need additional Korean support.)
Perhaps you are installing the KDE Spin? which comes without much i18n
support preinstalled I believe.

One can also just try `dnf -n install langpacks-ko` to check what fonts
(and input method) would be pulled in, and you are free to remove
langpacks-ko anyway.

I am wondering how you reached 3GB - the Noto CJK fonts we now ship are not
small on disk - that might be a part of the problem perhaps.

We do appreciate the feedback - and will be working to improve the
langpacks experience further.

Jens

On Sat, Jun 15, 2019 at 6:17 AM Jason L Tibbitts III <tibbs(a)math.uh.edu&gt;
wrote:

>  I noticed that my F30 installs are coming out far larger than my F29
>  installs (by 3GB or so) and did some digging into why.
>
>  With F30 we switched away from having groups named like "korean-support"
>  that you could install to get input methods and fonts needed to display
>  a language and instead we have metapackages named like "langpacks-ko".
>  These metapackages have (generally) weak dependencies on the fonts and
>  input methods as before. But other packages have reverse weak
>  dependencies on the langpacks, which causes far more to get pulled in
>  than was previously installed.  For example, each libreoffice langpack
>  has a "supplements" weak reverse dependency on the base "langpacks"
>  metapackage.
>
>  All of this seems fine, but my original goal was to be able to properly
>  display, and perhaps input, various languages.  But now I get
>  translations and help files and such as well.  Not just for libreoffice,
>  but for eclipse, glibc, all of KDE as well.  And I also get
>  autocorrection rules, spelling dictionaries, hyphenation rules, and
>  terreract OCR recognition data as well.  Some of those aren't small, and
>  the end result is that I need to bump up the size of / quite a bit.
>
>  Note that turning off install_weak_deps is not an option because for
>  most of the langpacks, _all_ of the langpack are weak.  (Some do have
>  hard font dependencies, and I'm not sure if this inconsistency is
>  intentional.)
>
>  So it seems we lost the simple "here are our suggested Korean fonts and
>  an input method" and instead the only thing you can say is "I want
>  everything possible to be available in Korean".  Is there any way to
>  improve the granularity here?  Perhaps by having "light" and "heavy"
>  langpacks, or splitting them by usage (translations versus simple
>  display of text)?
>
>  For now I guess I will simply extract the list of fonts and input
>  methods I want from the langpack specfile and stop installing the actual
>  langpack packages.
>
>   - J<
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]
>  

-- 
Jens Petersen
Associate Manager
RHEL Client i18n
Singapore

[Jason L Tibbitts III]

> >>>> "JP" == Jens-Ulrik Petersen
> <petersen(a)redhat.com&gt; writes: 
JP> Jason, can you explain in more details (bug report is also fine) how
JP> exactly you are installing?

I install a generic minimal system via kickstart (booted using the
Server PXE images and using the Everything repositories) and then after
the reboot, ansible runs (via ansible-pull) and installs whatever is
needed for the type of system it's configured to be.

JP> (Because for both Workstation/Silverblue, and Server I believe, we
JP> install fonts and input methods for Korean (and other langs like
JP> Japanese) by default anyway

It would really surprise me if a kickstart file with nothing listed
under %packages installed a Korean input method.

JP> Perhaps you are installing the KDE Spin? which comes without much
JP> i18n support preinstalled I believe.

No spins are involved; just a plain kickstart file pointed at the full
repository.

JP> One can also just try `dnf -n install langpacks-ko` to check what
JP> fonts (and input method) would be pulled in, and you are free to
JP> remove langpacks-ko anyway.

Yes, though it's simpler to me to just pull the dependencies out of the
langpacks specfile as I'm doing now.  Unfortunately that means I have to
periodically recheck if, for example, the recommended font set for
Gujarati happens to change in the future.  It's not an undue burden but
it was nice to be able to rely on installing @gujarati-support as was
the case pre-F30.

JP> I am wondering how you reached 3GB - the Noto CJK fonts we now ship
JP> are not small on disk - that might be a part of the problem perhaps.

Well, as I wrote previously, it's things like the Libreoffice help files
and the Tesseract recognition data which get pulled in via Supplements:.
These can be quite large.  And some of the size difference is certainly
unrelated to this issue.  (I was installing a total of twelve
langpacks but I should probably be installing even more.)

 - J<

[Jens-Ulrik Petersen]

On Tue, Jun 18, 2019 at 3:07 AM Jason L Tibbitts III <tibbs(a)math.uh.edu&gt;
wrote:

>  >>>>> "JP" == Jens-Ulrik Petersen
> <petersen(a)redhat.com&gt; writes:
>  I install a generic minimal system via kickstart (booted using the
>  Server PXE images and using the Everything repositories) and then after
>  the reboot, ansible runs (via ansible-pull) and installs whatever is
>  needed for the type of system it's configured to be.
>  
Okay I see, thanks: so presumably previously you were installing language
support groups.

Some time back lang supports groups also pulled in langpacks of
translations though.


>  JP> One can also just try `dnf -n install langpacks-ko` to check
> what
>  JP> fonts (and input method) would be pulled in, and you are free to
>  JP> remove langpacks-ko anyway.
>
>  Yes, though it's simpler to me to just pull the dependencies out of the
>  langpacks specfile as I'm doing now.  Unfortunately that means I have to
>  periodically recheck if, for example, the recommended font set for
>  Gujarati happens to change in the future.  It's not an undue burden but
>  it was nice to be able to rely on installing @gujarati-support as was
>  the case pre-F30.
>  
It sounds like what you probably really want is:

`dnf install @fonts @input-methods`

That will get you the default-installed system fonts and input methods (as
you would get in a standard Workstation install).

This kind of problem also becomes more relevant for Toolbox containers,
etc, which are also typically minimal installs.

JP> I am wondering how you reached 3GB - the Noto CJK fonts we now ship
>  JP> are not small on disk - that might be a part of the problem
> perhaps.
>
>  Well, as I wrote previously, it's things like the Libreoffice help files
>  and the Tesseract recognition data which get pulled in via Supplements:.
>  These can be quite large.  And some of the size difference is certainly
>  unrelated to this issue.  (I was installing a total of twelve
>  langpacks but I should probably be installing even more.)
>  
Currently it is only really intended for a few (one or two) langpacks-*
packages to be installed.
Normally just for one's native language say. This may change as the
experience evolves.
I agree that installing all langpacks-* does not make sense - it will also
pull in many locales packages,
possibly duplicating locales in the glibc-all-langpacks archive.

And yes as Nicolas also said maybe we need: langpacks-ko-fonts and
langpacks-ko-input-methods, etc.
This needs a little more thought and discussion I feel, but we could
consider it for F31 or F32 surely.

Jens

[Nicolas Mailhot]

Le 2019-06-19 12:48, Jens-Ulrik Petersen a écrit :
Hi

>  And yes as Nicolas also said maybe we need: langpacks-ko-fonts and
>  langpacks-ko-input-methods, etc. 
I's much more than input methods, it's anything you need to read/write a 
language (input, spell/grammar, fonts, etc)

Though the whole concept is difficult to convey in metapackage naming to 
non-technical users. After thinking about it some more, what would be 
understandable for everyone involved is

langpack-ui-foo Recommends langpack-foo

and

app-langpack-ui-foo  Recommends  app-langpack-foo
app-langpack-ui-foo  Supplements (langpack-ui-foo if app)
app-langpack-foo     Supplements (langpack-foo if app)

since UI is better known than obscure acronyms like i18n or l10n



Now UNIX UIs have historically been horrible about all this, with *nix 
devs deriving UI language from input layout and completely massing up 
things. Microsoft is much better at it because Office is a major earner 
for them, and it is used to create documents in many human languages, so 
they had to learn how to do it properly a long time ago (last 
millennium, circa ’95)

The DEs and Wayland really need to break the UI language/input tie-up 
once and for all.


A user can consider that the localization of its favorite apps sucks 
loads, and configure his system UI to use another language. That does 
not mean he can switch the language processing capabilities of his 
system to this other language, unless he can convince all the people he 
interacts with in his country to switch too. So that's one major case 
where UI settings differ from language processing settings.

Conversely, a developer will produce material using computer languages, 
which are English-oriented, so he will often keep the UI in his native 
human language, but switch the language processing to English. So, the 
reverse difference exists.

Furthermore, anyone who speaks more than a single human language, will 
want the language processing parts for all those languages. But the UI 
can be all languages at once, so he'll only need a single UI langpack.

Lastly, DE settings (not necessarily the metapackages themselves) need 
to separate cleanly input method from language processing. No one who 
writes several languages that share the same script (for example 
English/French/Italian…) will accept to switch from AZERTY to QWERTY to 
QWERTZ whenever he switches languages. He will need to switch 
spellchecking though.


So all the language UI = input method = language processing does not 
exist in the real world and building technical solutions around this 
false equivalence does not work

Regards,

-- 
Nicolas Mailhot

[Nicolas Mailhot]

Le 2019-06-19 14:32, Nicolas Mailhot a écrit :
>  Le 2019-06-19 12:48, Jens-Ulrik Petersen a écrit :
>  Hi
>  
> > And yes as Nicolas also said maybe we need: langpacks-ko-fonts and
> > langpacks-ko-input-methods, etc.
>  
>  I's much more than input methods, it's anything you need to read/write
>  a language (input, spell/grammar, fonts, etc)
>  
>  Though the whole concept is difficult to convey in metapackage naming
>  to non-technical users. After thinking about it some more, what would
>  be understandable for everyone involved is
>  
>  langpack-ui-foo Recommends langpack-foo
>  
>  and
>  
>  app-langpack-ui-foo  Recommends  app-langpack-foo
>  app-langpack-ui-foo  Supplements (langpack-ui-foo if app)
>  app-langpack-foo     Supplements (langpack-foo if app)
>  
>  since UI is better known than obscure acronyms like i18n or l10n 
with also
app-langpack-ui-foo Requires langpack-ui-foo
app-langpack-foo    Requires langpack-foo

if they can't function without the generic (not app specific) support 
installed

-- 
Nicolas Mailhot