Unacceptable size increase to ALL live images in F28 (caused by Annobin?)

[Kevin Kofler]

Hi,

on #fedora-kde, we discussed a huge increase to the size of the KDE live 
image:
F27 KDE GA: 1656752111 bytes
F27 KDE Respin: 1744830464 bytes
F28 KDE Beta: 2069889024 bytes

It turns out the LXQt live image is also hit:
F27 LXQt GA: < 1 GiB
F28 LXQt Beta: 1.4 GiB
and I am pretty sure that this is a global issue affecting ALL the live 
images.

While there is certainly more than one cause (e.g., the difference between 
F27 KDE GA and the F27 KDE Respin must be caused by package updates and/or 
added dependencies), it is striking that between a recent F27 KDE Respin 
(which has several of the KDE package updates that are also in the F28 Beta) 
and the F28 Beta, there is a size increase of more than 300 MB!

Therefore, my question:
Is this size increase caused by Annobin?

If not, then WHAT causes this size increase? And can the offending change(s) 
be reverted in time for F28 Final?

If yes, then IMHO it is time to enact the contingency plan, i.e.:
1. drop the annobin requirement from redhat-rpm-config, AND
2. perform an emergency mass rebuild to actually get rid of the bloat.

A global live image size increase of 19%-40% is just not acceptable.

        Kevin Kofler

[Stephen Gallagher]

On Thu, Mar 29, 2018 at 8:43 AM Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:

>  Hi,
>
>  on #fedora-kde, we discussed a huge increase to the size of the KDE live
>  image:
>  F27 KDE GA: 1656752111 bytes
>  F27 KDE Respin: 1744830464 bytes
>  F28 KDE Beta: 2069889024 <(206)%20988-9024> bytes
>
>  It turns out the LXQt live image is also hit:
>  F27 LXQt GA: < 1 GiB
>  F28 LXQt Beta: 1.4 GiB
>  and I am pretty sure that this is a global issue affecting ALL the live
>  images.
>
>  While there is certainly more than one cause (e.g., the difference between
>  F27 KDE GA and the F27 KDE Respin must be caused by package updates and/or
>  added dependencies), it is striking that between a recent F27 KDE Respin
>  (which has several of the KDE package updates that are also in the F28
>  Beta)
>  and the F28 Beta, there is a size increase of more than 300 MB!
>
>  Therefore, my question:
>  Is this size increase caused by Annobin?
>
>  If not, then WHAT causes this size increase? And can the offending
>  change(s)
>  be reverted in time for F28 Final?
>
>  If yes, then IMHO it is time to enact the contingency plan, i.e.:
>  1. drop the annobin requirement from redhat-rpm-config, AND
>  2. perform an emergency mass rebuild to actually get rid of the bloat.
>
>  A global live image size increase of 19%-40% is just not acceptable.
>
>          Kevin Kofler
>  

Have you done a comparison between individual package sizes and count? I
wonder if the problem is that some subset of critical packages gained a
whole lot of dependencies (intentionally or not) or if we're actually
seeing that built binaries are suddenly carrying a huge additional amount
of content. Or even if for some reason the binaries aren't being stripped
properly.

[Stephen Gallagher]

On Thu, Mar 29, 2018 at 9:16 AM Stephen Gallagher <sgallagh(a)redhat.com&gt;
wrote:

>  On Thu, Mar 29, 2018 at 8:43 AM Kevin Kofler
> <kevin.kofler(a)chello.at&gt;
>  wrote:
>
> > Hi,
> >
> > on #fedora-kde, we discussed a huge increase to the size of the KDE live
> > image:
> > F27 KDE GA: 1656752111 bytes
> > F27 KDE Respin: 1744830464 bytes
> > F28 KDE Beta: 2069889024 <(206)%20988-9024> bytes
> >
> > It turns out the LXQt live image is also hit:
> > F27 LXQt GA: < 1 GiB
> > F28 LXQt Beta: 1.4 GiB
> > and I am pretty sure that this is a global issue affecting ALL the live
> > images.
> >
> > While there is certainly more than one cause (e.g., the difference between
> > F27 KDE GA and the F27 KDE Respin must be caused by package updates and/or
> > added dependencies), it is striking that between a recent F27 KDE Respin
> > (which has several of the KDE package updates that are also in the F28
> > Beta)
> > and the F28 Beta, there is a size increase of more than 300 MB!
> >
> > Therefore, my question:
> > Is this size increase caused by Annobin?
> >
> > If not, then WHAT causes this size increase? And can the offending
> > change(s)
> > be reverted in time for F28 Final?
> >
> > If yes, then IMHO it is time to enact the contingency plan, i.e.:
> > 1. drop the annobin requirement from redhat-rpm-config, AND
> > 2. perform an emergency mass rebuild to actually get rid of the bloat.
> >
> > A global live image size increase of 19%-40% is just not acceptable.
> >
> >         Kevin Kofler
> >
>
>
>  Have you done a comparison between individual package sizes and count? I
>  wonder if the problem is that some subset of critical packages gained a
>  whole lot of dependencies (intentionally or not) or if we're actually
>  seeing that built binaries are suddenly carrying a huge additional amount
>  of content. Or even if for some reason the binaries aren't being stripped
>  properly.
>  
I just checked several packages that compiled the same sources on F26 and
F28:
* sscg: 4.31% increase in size
* nodejs 8:  22% *decrease* in size
* libmodulemd: 12% *decrease* in size
* sssd-common: 2.58% increase in size
* boost-system: 1.00% increase in size

Granted, this is just a spot-check, but it seems to me like (on the whole)
improvements in the compiler are at least balancing out the increase caused
by annobin. It definitely doesn't indicate a 19-40% increase across the
board. So I'd guess it's more likely to be dependency bloat or one or two
core packages growing significantly.

[Neal Gompa]

On Thu, Mar 29, 2018 at 9:32 AM, Stephen Gallagher <sgallagh(a)redhat.com&gt; wrote:
>  I just checked several packages that compiled the same sources on F26 and
>  F28:
>  * sscg: 4.31% increase in size
>  * nodejs 8:  22% *decrease* in size
>  * libmodulemd: 12% *decrease* in size
>  * sssd-common: 2.58% increase in size
>  * boost-system: 1.00% increase in size
>
>  Granted, this is just a spot-check, but it seems to me like (on the whole)
>  improvements in the compiler are at least balancing out the increase caused
>  by annobin. It definitely doesn't indicate a 19-40% increase across the
>  board. So I'd guess it's more likely to be dependency bloat or one or two
>  core packages growing significantly.
>  
I don't yet have proof of this, but I suspect that we might have more
packages that either:

1. Don't respect our compilation flags and thus simply are larger
because of that

or

2. Are built in static or static-like manner which promotes binary
bloat (Go, Rust, etc.)


Last I checked, we don't have anything significant on the ISOs that
fall into the latter category, so I would not be shocked if the former
is the reason why...

-- 
真実はいつも一つ！/ Always, there's only one truth!

[Martin Bříza]

On Thu, 29 Mar 2018 15:32:06 +0200, Stephen Gallagher  
<sgallagh(a)redhat.com&gt; wrote:

>  On Thu, Mar 29, 2018 at 9:16 AM Stephen Gallagher <sgallagh(a)redhat.com&gt;  
>  wrote:
> > On Thu, Mar 29, 2018 at 8:43 AM Kevin Kofler <kevin.kofler(a)chello.at&gt;  
> > wrote:
> >> Hi,
> >>
> >> on #fedora-kde, we discussed a huge increase to the size of the KDE  
> >> live
> >> image:
> >> F27 KDE GA: 1656752111 bytes
> >> F27 KDE Respin: 1744830464 bytes
> >> F28 KDE Beta: 2069889024 bytes
> >>
> >> It turns out the LXQt live image is also hit:
> >> F27 LXQt GA: < 1 GiB
> >> F28 LXQt Beta: 1.4 GiB
> >> and I am pretty sure that this is a global issue affecting ALL the live
> >> images.
> >>
> >> While there is certainly more than one cause (e.g., the difference  
> >> between
> >> F27 KDE GA and the F27 KDE Respin must be caused by package updates  
> >> and/or
> >> added dependencies), it is striking that between a recent F27 KDE  
> >> Respin
> >> (which has several of the KDE package updates that are also in the F28  
> >> Beta)
> >> and the F28 Beta, there is a size increase of more than 300 MB!
> >>
> >> Therefore, my question:
> >> Is this size increase caused by Annobin?
> >>
> >> If not, then WHAT causes this size increase? And can the offending  
> >> change(s)
> >> be reverted in time for F28 Final?
> >>
> >> If yes, then IMHO it is time to enact the contingency plan, i.e.:
> >> 1. drop the annobin requirement from redhat-rpm-config, AND
> >> 2. perform an emergency mass rebuild to actually get rid of the bloat.
> >>
> >> A global live image size increase of 19%-40% is just not acceptable.
> >>
> >>        Kevin Kofler
> >
> >
> > Have you done a comparison between individual package sizes and count?  
> > I wonder if the problem is that some subset of >>critical packages  
> > gained a whole lot of dependencies (intentionally or not) or if we're  
> > actually seeing that built >>binaries are suddenly carrying a huge  
> > additional amount of content. Or even if for some reason the binaries  
> > aren't being >>stripped properly.
>
>  I just checked several packages that compiled the same sources on F26  
>  and F28:
>  * sscg: 4.31% increase in size
>  * nodejs 8:  22% *decrease* in size
>  * libmodulemd: 12% *decrease* in size
>  * sssd-common: 2.58% increase in size
>  * boost-system: 1.00% increase in size
>
>  Granted, this is just a spot-check, but it seems to me like (on the  
>  whole) improvements in the compiler are at least >balancing out the  
>  increase caused by annobin. It definitely doesn't indicate a 19-40%  
>  increase across the board. So I'd >guess it's more likely to be  
>  dependency bloat or one or two core packages growing significantly. 
Just for fun, I wrote a patch comparing only _unrebased_ (probably just  
rebuilt) packages between F27 and F28.
For compiled packages, the size difference is just a bunch of percent.  
Some packages grew considerably, but these are mostly fonts  
(google-noto-sans-fonts by 1012%).

The diff is attached. Order of data is: percentage, package name, f27 RPM  
size, f28 RPM size.

Martin

[Florian Weimer]

On 03/29/2018 02:42 PM, Kevin Kofler wrote:
>  Is this size increase caused by Annobin? 
Very unlikely.  We only have 356 MB of annobin data in the entire x86-64 
buildroot.  That's 1.7% of the total ELF binary size.

Thanks,
Florian

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Kevin Kofler]

Hi,

we did more debugging on #fedora-kde (thanks in particular to lupinix) and
we found what seems to be the primary source of the bloat: CJK fonts!

CJK fonts are by far the largest of all fonts due to the huge number of
characters used in those languages.

Up to Fedora 26, Fedora shipped 4 CJK fonts:
      <packagereq
type="default">adobe-source-han-sans-cn-fonts</packagereq>
      <packagereq
type="default">adobe-source-han-sans-tw-fonts</packagereq>
      <packagereq
type="default">naver-nanum-gothic-fonts</packagereq>
      <packagereq type="default">vlgothic-fonts</packagereq>
The KDE and LXQt Spins actually opted to blacklist these fonts in their
kickstart, and ship one compact CJK font instead: wqy-microhei-fonts.

In Fedora 27, this Change:
https://fedoraproject.org/wiki/Changes/ChineseSerifFonts
added 2 additional fonts (for a total of 6):
      <packagereq
type="default">adobe-source-han-serif-cn-fonts</packagereq>
      <packagereq
type="default">adobe-source-han-serif-tw-fonts</packagereq>
which were unfortunately missing from the blacklist:
https://bugzilla.redhat.com/show_bug.cgi?id=1530006
already increasing the size of the image.

But now in Fedora 28, after:
https://fedoraproject.org/wiki/Changes/JPDefaultFontsToNoto
https://fedoraproject.org/wiki/Changes/KRDefaultFontsToNoto
https://fedoraproject.org/wiki/Changes/ChineseDefaultFontsToNoto
we actually ship a whopping 12 CJK fonts:
      <packagereq
type="default">google-noto-sans-jp-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-kr-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-mono-cjk-jp-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-mono-cjk-kr-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-mono-cjk-sc-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-mono-cjk-tc-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-sc-fonts</packagereq>
      <packagereq
type="default">google-noto-sans-tc-fonts</packagereq>
      <packagereq
type="default">google-noto-serif-jp-fonts</packagereq>
      <packagereq
type="default">google-noto-serif-kr-fonts</packagereq>
      <packagereq
type="default">google-noto-serif-sc-fonts</packagereq>
      <packagereq
type="default">google-noto-serif-tc-fonts</packagereq>
none of which are blacklisted in the Spins! According to lupinix, these
amount to a download size (and thus an xz-compressed size, which is also the
compression algorithm used for the live images) of 364 MiB!

The fix is to update the blacklists in the KDE and LXQt spin kickstarts, as
per the discussion under:
https://bugzilla.redhat.com/show_bug.cgi?id=1530006
IMHO, this needs to be implemented, urgently.

But I think we also need to generally consider whether it makes sense to
force 3 font variants for each CJK language on all users worldwide, and
whether there are smaller fonts that could be used. (E.g.,
wqy-microhei-fonts is very effective, but unfortunately it only covers
Simplified Chinese and the syllabic parts of Japanese and Korean, not the
Traditional Chinese, Japanese or Korean renderings of the CJK Unified
Ideographs.)

Any additional bloat is likely caused by:
* the updated glibc Unicode collation tables,
* Annobin,
but those seem to have much smaller impact than the above.

        Kevin Kofler

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Stephen Gallagher]

On Thu, Mar 29, 2018 at 10:12 AM Kevin Kofler <kevin.kofler(a)chello.at&gt;
wrote:

>  Hi,
>
>  we did more debugging on #fedora-kde (thanks in particular to lupinix) and
>  we found what seems to be the primary source of the bloat: CJK fonts!
>
>  CJK fonts are by far the largest of all fonts due to the huge number of
>  characters used in those languages.
>
>  Up to Fedora 26, Fedora shipped 4 CJK fonts:
>        <packagereq
>  type="default">adobe-source-han-sans-cn-fonts</packagereq>
>        <packagereq
>  type="default">adobe-source-han-sans-tw-fonts</packagereq>
>        <packagereq
> type="default">naver-nanum-gothic-fonts</packagereq>
>        <packagereq type="default">vlgothic-fonts</packagereq>
>  The KDE and LXQt Spins actually opted to blacklist these fonts in their
>  kickstart, and ship one compact CJK font instead: wqy-microhei-fonts.
>
>  In Fedora 27, this Change:
>  https://fedoraproject.org/wiki/Changes/ChineseSerifFonts
>  added 2 additional fonts (for a total of 6):
>        <packagereq
>  type="default">adobe-source-han-serif-cn-fonts</packagereq>
>        <packagereq
>  type="default">adobe-source-han-serif-tw-fonts</packagereq>
>  which were unfortunately missing from the blacklist:
>  https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  already increasing the size of the image.
>
>  But now in Fedora 28, after:
>  https://fedoraproject.org/wiki/Changes/JPDefaultFontsToNoto
>  https://fedoraproject.org/wiki/Changes/KRDefaultFontsToNoto
>  https://fedoraproject.org/wiki/Changes/ChineseDefaultFontsToNoto
>  we actually ship a whopping 12 CJK fonts:
>        <packagereq
> type="default">google-noto-sans-jp-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-kr-fonts</packagereq>
>        <packagereq
>  type="default">google-noto-sans-mono-cjk-jp-fonts</packagereq>
>        <packagereq
>  type="default">google-noto-sans-mono-cjk-kr-fonts</packagereq>
>        <packagereq
>  type="default">google-noto-sans-mono-cjk-sc-fonts</packagereq>
>        <packagereq
>  type="default">google-noto-sans-mono-cjk-tc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-sc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-tc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-jp-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-kr-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-sc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-tc-fonts</packagereq>
>  none of which are blacklisted in the Spins! According to lupinix, these
>  amount to a download size (and thus an xz-compressed size, which is also
>  the
>  compression algorithm used for the live images) of 364 MiB!
>
>  The fix is to update the blacklists in the KDE and LXQt spin kickstarts, as
>  per the discussion under:
>  https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  IMHO, this needs to be implemented, urgently.
>
>  But I think we also need to generally consider whether it makes sense to
>  force 3 font variants for each CJK language on all users worldwide, and
>  whether there are smaller fonts that could be used. (E.g.,
>  wqy-microhei-fonts is very effective, but unfortunately it only covers
>  Simplified Chinese and the syllabic parts of Japanese and Korean, not the
>  Traditional Chinese, Japanese or Korean renderings of the CJK Unified
>  Ideographs.)
>
>  
Kevin, thanks for the investigation and the detailed analysis. Great work
tracking this down!

I think our font strategy is a bit complicated, because we do strive to be
an international distribution, but I feel like there must be better ways to
accomplish this goal than to simply install all possible fonts by default.
I know that for GNOME at least, there's some PackageKit integration that
allows users to be prompted to install fonts as-needed (though it also
seems to trigger whenever I accidentally `cat` a binary file in the
terminal :-) ). Perhaps we can look into finding a desktop-agnostic way of
doing this and then limit our installed fonts to only a few minimal ones
based on the language selected in Anaconda?

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Adam Williamson]

On Thu, 2018-03-29 at 15:57 +0000, Stephen Gallagher wrote:
>  On Thu, Mar 29, 2018 at 10:12 AM Kevin Kofler
> <kevin.kofler(a)chello.at&gt;
>  wrote:
>  
>  > Hi,
>  > 
>  > we did more debugging on #fedora-kde (thanks in particular to lupinix) and
>  > we found what seems to be the primary source of the bloat: CJK fonts!
>  > 
>  > CJK fonts are by far the largest of all fonts due to the huge number of
>  > characters used in those languages.
>  > 
>  > Up to Fedora 26, Fedora shipped 4 CJK fonts:
>  >       <packagereq
>  > type="default">adobe-source-han-sans-cn-fonts</packagereq>
>  >       <packagereq
>  > type="default">adobe-source-han-sans-tw-fonts</packagereq>
>  >       <packagereq
> type="default">naver-nanum-gothic-fonts</packagereq>
>  >       <packagereq type="default">vlgothic-fonts</packagereq>
>  > The KDE and LXQt Spins actually opted to blacklist these fonts in their
>  > kickstart, and ship one compact CJK font instead: wqy-microhei-fonts.
>  > 
>  > In Fedora 27, this Change:
>  > https://fedoraproject.org/wiki/Changes/ChineseSerifFonts
>  > added 2 additional fonts (for a total of 6):
>  >       <packagereq
>  > type="default">adobe-source-han-serif-cn-fonts</packagereq>
>  >       <packagereq
>  > type="default">adobe-source-han-serif-tw-fonts</packagereq>
>  > which were unfortunately missing from the blacklist:
>  > https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  > already increasing the size of the image.
>  > 
>  > But now in Fedora 28, after:
>  > https://fedoraproject.org/wiki/Changes/JPDefaultFontsToNoto
>  > https://fedoraproject.org/wiki/Changes/KRDefaultFontsToNoto
>  > https://fedoraproject.org/wiki/Changes/ChineseDefaultFontsToNoto
>  > we actually ship a whopping 12 CJK fonts:
>  >       <packagereq
> type="default">google-noto-sans-jp-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-sans-kr-fonts</packagereq>
>  >       <packagereq
>  > type="default">google-noto-sans-mono-cjk-jp-fonts</packagereq>
>  >       <packagereq
>  > type="default">google-noto-sans-mono-cjk-kr-fonts</packagereq>
>  >       <packagereq
>  > type="default">google-noto-sans-mono-cjk-sc-fonts</packagereq>
>  >       <packagereq
>  > type="default">google-noto-sans-mono-cjk-tc-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-sans-sc-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-sans-tc-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-serif-jp-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-serif-kr-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-serif-sc-fonts</packagereq>
>  >       <packagereq
> type="default">google-noto-serif-tc-fonts</packagereq>
>  > none of which are blacklisted in the Spins! According to lupinix, these
>  > amount to a download size (and thus an xz-compressed size, which is also
>  > the
>  > compression algorithm used for the live images) of 364 MiB!
>  > 
>  > The fix is to update the blacklists in the KDE and LXQt spin kickstarts, as
>  > per the discussion under:
>  > https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  > IMHO, this needs to be implemented, urgently.
>  > 
>  > But I think we also need to generally consider whether it makes sense to
>  > force 3 font variants for each CJK language on all users worldwide, and
>  > whether there are smaller fonts that could be used. (E.g.,
>  > wqy-microhei-fonts is very effective, but unfortunately it only covers
>  > Simplified Chinese and the syllabic parts of Japanese and Korean, not the
>  > Traditional Chinese, Japanese or Korean renderings of the CJK Unified
>  > Ideographs.)
>  > 
>  > 
>  
>  Kevin, thanks for the investigation and the detailed analysis. Great work
>  tracking this down!
>  
>  I think our font strategy is a bit complicated, because we do strive to be
>  an international distribution, but I feel like there must be better ways to
>  accomplish this goal than to simply install all possible fonts by default. 
Well, I don't think this really is "all possible fonts". Just looking
at the list, one obvious improvement is that there now appear to be
sans, serif, and monospace fonts, whereas before there was only sans.
Would we ship images without serif and monospace fonts for Latin?

For those wondering what the other factors here are: 'jp' is Japanese,
'kr' is Korean, 'sc' is Simplified Chinese and 'tc' is
Traditional
Chinese. We could go more into the details on the relationships between
them and why we might need different fonts for them all, but I think
what we really ought to do is go to the actual experts here - the g11n
teams, and specifically CJK specialists on those teams - explain the
problem, and see what they recommend as the best way forward to provide
an acceptable experience for CJK users of our images while keeping
sizes reasonable.

>  I know that for GNOME at least, there's some PackageKit
> integration that
>  allows users to be prompted to install fonts as-needed (though it also
>  seems to trigger whenever I accidentally `cat` a binary file in the
>  terminal :-) ). Perhaps we can look into finding a desktop-agnostic way of
>  doing this and then limit our installed fonts to only a few minimal ones
>  based on the language selected in Anaconda? 
The obvious problem here is *live* images, though. I'd say we want the
live environment to be practical for CJK users without them having to
do large downloads and installs that, in the live environment, will eat
up RAM...
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Owen Taylor]

It sounds like what we are doing is shipping the "regional subset" fonts
for all 4 CJK scripts. I wonder if we would do better to ship one unified
font - I believe that at least with the GNOME stack, the rendered results
will be equivalent, since Pango+Harfbuzz should set appropriate 'locl' tags
(https://www.google.com/get/noto/help/cjk/).

Investigation would be required:
 * Am I right about GNOME?
 * Do you also get the right behavior under Qt, LibreOffice, Web browsers -
they use harfbuzz, but do they set locl tags based on $LANG?
 * How much smaller is this
 * What if the user wants to get Korean fonts in LibreOffice with
$LANG=ja_JP, or whatever?

But there would be an issue that in a font list the font would probably
show up with a name that sounds language specific, which is bad.

Could also investigate whether installing the fonts in TTC form is broadly
enough supported in the different rendering stacks and whether it gives
space saving.

I do think we need to ship enough fonts to get the user through GDM and
gnome-initial-setup to render correctly for major languages, but if we want
to start installing additional weights/style variants after installation,
there's a related issue at:

 https://pagure.io/fedora-workstation/issue/27

Owen


On Thu, Mar 29, 2018 at 12:06 PM, Adam Williamson <
adamwill(a)fedoraproject.org&gt; wrote:

>  On Thu, 2018-03-29 at 15:57 +0000, Stephen Gallagher wrote:
>  > On Thu, Mar 29, 2018 at 10:12 AM Kevin Kofler <kevin.kofler(a)chello.at&gt;
>  > wrote:
>  >
>  > > Hi,
>  > >
>  > > we did more debugging on #fedora-kde (thanks in particular to lupinix)
>  and
>  > > we found what seems to be the primary source of the bloat: CJK fonts!
>  > >
>  > > CJK fonts are by far the largest of all fonts due to the huge number of
>  > > characters used in those languages.
>  > >
>  > > Up to Fedora 26, Fedora shipped 4 CJK fonts:
>  > >       <packagereq
>  > > type="default">adobe-source-han-sans-cn-fonts</packagereq>
>  > >       <packagereq
>  > > type="default">adobe-source-han-sans-tw-fonts</packagereq>
>  > >       <packagereq
> type="default">naver-nanum-gothic-fonts</packagereq>
>  > >       <packagereq
> type="default">vlgothic-fonts</packagereq>
>  > > The KDE and LXQt Spins actually opted to blacklist these fonts in their
>  > > kickstart, and ship one compact CJK font instead: wqy-microhei-fonts.
>  > >
>  > > In Fedora 27, this Change:
>  > > https://fedoraproject.org/wiki/Changes/ChineseSerifFonts
>  > > added 2 additional fonts (for a total of 6):
>  > >       <packagereq
>  > > type="default">adobe-source-han-serif-cn-fonts</packagereq>
>  > >       <packagereq
>  > > type="default">adobe-source-han-serif-tw-fonts</packagereq>
>  > > which were unfortunately missing from the blacklist:
>  > > https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  > > already increasing the size of the image.
>  > >
>  > > But now in Fedora 28, after:
>  > > https://fedoraproject.org/wiki/Changes/JPDefaultFontsToNoto
>  > > https://fedoraproject.org/wiki/Changes/KRDefaultFontsToNoto
>  > > https://fedoraproject.org/wiki/Changes/ChineseDefaultFontsToNoto
>  > > we actually ship a whopping 12 CJK fonts:
>  > >       <packagereq type="default">google-noto-
>  sans-jp-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  sans-kr-fonts</packagereq>
>  > >       <packagereq
>  > >
> type="default">google-noto-sans-mono-cjk-jp-fonts</packagereq>
>  > >       <packagereq
>  > >
> type="default">google-noto-sans-mono-cjk-kr-fonts</packagereq>
>  > >       <packagereq
>  > >
> type="default">google-noto-sans-mono-cjk-sc-fonts</packagereq>
>  > >       <packagereq
>  > >
> type="default">google-noto-sans-mono-cjk-tc-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  sans-sc-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  sans-tc-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  serif-jp-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  serif-kr-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  serif-sc-fonts</packagereq>
>  > >       <packagereq type="default">google-noto-
>  serif-tc-fonts</packagereq>
>  > > none of which are blacklisted in the Spins! According to lupinix, these
>  > > amount to a download size (and thus an xz-compressed size, which is
>  also
>  > > the
>  > > compression algorithm used for the live images) of 364 MiB!
>  > >
>  > > The fix is to update the blacklists in the KDE and LXQt spin
>  kickstarts, as
>  > > per the discussion under:
>  > > https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  > > IMHO, this needs to be implemented, urgently.
>  > >
>  > > But I think we also need to generally consider whether it makes sense
>  to
>  > > force 3 font variants for each CJK language on all users worldwide, and
>  > > whether there are smaller fonts that could be used. (E.g.,
>  > > wqy-microhei-fonts is very effective, but unfortunately it only covers
>  > > Simplified Chinese and the syllabic parts of Japanese and Korean, not
>  the
>  > > Traditional Chinese, Japanese or Korean renderings of the CJK Unified
>  > > Ideographs.)
>  > >
>  > >
>  >
>  > Kevin, thanks for the investigation and the detailed analysis. Great work
>  > tracking this down!
>  >
>  > I think our font strategy is a bit complicated, because we do strive to
>  be
>  > an international distribution, but I feel like there must be better ways
>  to
>  > accomplish this goal than to simply install all possible fonts by
>  default.
>
>  Well, I don't think this really is "all possible fonts". Just looking
>  at the list, one obvious improvement is that there now appear to be
>  sans, serif, and monospace fonts, whereas before there was only sans.
>  Would we ship images without serif and monospace fonts for Latin?
>
>  For those wondering what the other factors here are: 'jp' is Japanese,
>  'kr' is Korean, 'sc' is Simplified Chinese and 'tc' is
> Traditional
>  Chinese. We could go more into the details on the relationships between
>  them and why we might need different fonts for them all, but I think
>  what we really ought to do is go to the actual experts here - the g11n
>  teams, and specifically CJK specialists on those teams - explain the
>  problem, and see what they recommend as the best way forward to provide
>  an acceptable experience for CJK users of our images while keeping
>  sizes reasonable.
>
>  > I know that for GNOME at least, there's some PackageKit integration that
>  > allows users to be prompted to install fonts as-needed (though it also
>  > seems to trigger whenever I accidentally `cat` a binary file in the
>  > terminal :-) ). Perhaps we can look into finding a desktop-agnostic way
>  of
>  > doing this and then limit our installed fonts to only a few minimal ones
>  > based on the language selected in Anaconda?
>
>  The obvious problem here is *live* images, though. I'd say we want the
>  live environment to be practical for CJK users without them having to
>  do large downloads and installs that, in the live environment, will eat
>  up RAM...
>  --
>  Adam Williamson
>  Fedora QA Community Monkey
>  IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
>  http://www.happyassassin.net
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Chris Murphy]

Short version:
I suggest setting the Live ISO size limit to 4GB (3.7GiB), and
including whatever fonts the g11n folks think a global operating
system should have. Alternatively, have Latin and Global Live media
but I think the download savings having two separate media is probably
outweighed by producing dual media.


Longer version:

-rw-r-----. 1 chris chris 4.1G Jul  2  2017  Win10_1703_English_x64.iso
-rw-r-----. 1 chris chris 4.9G Nov 28 14:22 'macOS High Sierra.dmg'
-rw-rw-r--. 1 chris chris 1.8G Mar 28 15:21
Fedora-Workstation-Live-x86_64-28_Beta-1.1.iso
-rw-rw-r--. 1 chris chris 571M Mar 28 15:16
Fedora-Server-netinst-x86_64-28_Beta-1.1.iso
-rw-rw-r--. 1 chris chris 2.2G Mar 28 22:14
Fedora-AtomicWorkstation-ostree-x86_64-28_Beta-1.3.iso

We're way lower than anyone else.

The only limitation I'm aware of is a 4GiB file size (single extent)
limit on ISO 9660 [1], which would apply to LiveOS/squashfs.img. That
file is 100MB smaller than the ISO file size, ergo 99+% of the ISO
size is this one file.

And the next USB stick size above 2GB is 4GB. Realistically they're
hard to find. I can find 8GB sticks easily, but they're far less
common than 16, 32 and even 64GB sticks and they don't cost much less
than a 16GB stick. So it's reasonable to make the Live ISO size set to
4G (or 3.7GiB) for now.

[1]
ISO 9660 Level 3 supports multi-extent files, which I think Linux
mkisofs and the kernel support, in which case the actual limit is
8TiB. There's also UDF. And since Fedora has dropped the release
blocking criterion for successful optical media creation, we're not
limited to these two choices.

--
Chris Murphy

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Chris Murphy]

On Thu, Mar 29, 2018 at 5:29 PM, Chris Murphy <lists(a)colorremedies.com&gt; wrote:
>  Short version:
>  I suggest setting the Live ISO size limit to 4GB (3.7GiB), and
>  including whatever fonts the g11n folks think a global operating
>  system should have. Alternatively, have Latin and Global Live media
>  but I think the download savings having two separate media is probably
>  outweighed by producing dual media. 
                 ^effort of producing dual media


-- 
Chris Murphy

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Adam Williamson]

On Thu, 2018-03-29 at 17:29 -0600, Chris Murphy wrote:
>  Short version:
>  I suggest setting the Live ISO size limit to 4GB (3.7GiB), and
>  including whatever fonts the g11n folks think a global operating
>  system should have. Alternatively, have Latin and Global Live media
>  but I think the download savings having two separate media is probably
>  outweighed by producing dual media.
>  
>  
>  Longer version:
>  
>  -rw-r-----. 1 chris chris 4.1G Jul  2  2017  Win10_1703_English_x64.iso
>  -rw-r-----. 1 chris chris 4.9G Nov 28 14:22 'macOS High Sierra.dmg'
>  -rw-rw-r--. 1 chris chris 1.8G Mar 28 15:21
>  Fedora-Workstation-Live-x86_64-28_Beta-1.1.iso
>  -rw-rw-r--. 1 chris chris 571M Mar 28 15:16
>  Fedora-Server-netinst-x86_64-28_Beta-1.1.iso
>  -rw-rw-r--. 1 chris chris 2.2G Mar 28 22:14
>  Fedora-AtomicWorkstation-ostree-x86_64-28_Beta-1.3.iso
>  
>  We're way lower than anyone else. 
ubuntu-17.10.1-desktop-amd64.iso  1.4G
ubuntu-17.10.1-server-amd64.iso  754M
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Chris Murphy]

On Thu, Mar 29, 2018 at 6:16 PM, Adam Williamson
<adamwill(a)fedoraproject.org&gt; wrote:
>  On Thu, 2018-03-29 at 17:29 -0600, Chris Murphy wrote:
> > Short version:
> > I suggest setting the Live ISO size limit to 4GB (3.7GiB), and
> > including whatever fonts the g11n folks think a global operating
> > system should have. Alternatively, have Latin and Global Live media
> > but I think the download savings having two separate media is probably
> > outweighed by producing dual media.
> >
> >
> > Longer version:
> >
> > -rw-r-----. 1 chris chris 4.1G Jul  2  2017  Win10_1703_English_x64.iso
> > -rw-r-----. 1 chris chris 4.9G Nov 28 14:22 'macOS High Sierra.dmg'
> > -rw-rw-r--. 1 chris chris 1.8G Mar 28 15:21
> > Fedora-Workstation-Live-x86_64-28_Beta-1.1.iso
> > -rw-rw-r--. 1 chris chris 571M Mar 28 15:16
> > Fedora-Server-netinst-x86_64-28_Beta-1.1.iso
> > -rw-rw-r--. 1 chris chris 2.2G Mar 28 22:14
> > Fedora-AtomicWorkstation-ostree-x86_64-28_Beta-1.3.iso
> >
> > We're way lower than anyone else.
>
>  ubuntu-17.10.1-desktop-amd64.iso  1.4G
>  ubuntu-17.10.1-server-amd64.iso  754M 
Yes the linux distros are all over the map, and it's not always
obvious how complete these ISOs really are. openSUSE has a 1.6G image
the isn't complete and downloads a bunch of additional stuff, where
their 4.3G image is a complete ISO. And the Endless ISO is 13G and is
very complete. But the point is that a 4GB installer size is not out
of the ordinary.



-- 
Chris Murphy

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Adam Williamson]

On Thu, 2018-03-29 at 16:12 +0200, Kevin Kofler wrote:
>  wqy-microhei-fonts is very effective, but unfortunately it only
> covers
>  Simplified Chinese and the syllabic parts of Japanese and Korean, not the
>  Traditional Chinese, Japanese or Korean renderings of the CJK Unified
>  Ideographs.) 
AIUI, this means it really only works for simplified Chinese users
(which means, approximately, mainland China; other Chinese-speaking
territories tend to use traditional Chinese).

Simplified Chinese is a subset of traditional Chinese, so many
characters needed to render TC properly will be missing entirely from
it. Korean is technically related but in practice is really an entirely
different character set; AIUI being able to read Korean does not mean
you'd be able to read the 'equivalent' Chinese characters at all. You
couldn't just render a Korean text with Chinese ideographs in place of
the Korean ones and expect it to be at all OK for Korean readers.

Japanese ideographs are a subset of traditional Chinese too, but a
*different* subset. AIUI it's a smaller subset but includes some
characters that simplified Chinese doesn't include, so a simplified
Chinese font would again be missing some needed characters for full
Japanese rendering.

(Note on 'syllabic' vs. 'ideographic': Japanese and Korean have both
alphabet(s) for phonetic/syllabic rendering of words, like Latin, and
much larger alphabets of ideographic characters. We don't need to go
into the reasons for this or when one is used vs. the others; all
that's important is that any practical use of written Japanese and
Korean is going to involve the ideographic characters, so a font which
doesn't cover the ideographic characters is entirely insufficient).

So calling it a 'CJK' font is, well, stretching the point quite a lot.
;) It's really a mainland Chinese font, nothing more. Again, AIUI; it
would really be best to speak to the experts.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Kevin Kofler]

Adam Williamson wrote:
>  AIUI, this means it really only works for simplified Chinese users
>  (which means, approximately, mainland China; other Chinese-speaking
>  territories tend to use traditional Chinese). 
AIUI, WQY MicroHei actually tries to cover the traditional-only,
Japanese-only or Korean-only glyphs, too. According to 
http://wenq.org/wqy2/index.cgi?action=browse&id=Home&lang=en they cover the 
whole GBK set and 80% of CJK Extension A. According to
http://wenq.org/wqy2/index.cgi?MicroHei_BigBang_README the 2009 release 
claimed support for 99% of Japanese, 100% of Korean, and 99% of Traditional 
Chinese.

The main issue, as I was told, is those CJK Unified codepoints that are 
normally rendered/drawn differently in the different countries, but were 
unified into a single Unicode codepoint. I was told by the experts that WQY 
MicorHei (like all WQY fonts) only contains one rendering (the Simplified 
Chinese one) for those, and Wikipedia confirms it: 
https://en.wikipedia.org/wiki/WenQuanYi#Glyph

So in short, if you point it to Traditional Chinese, Japanese or Korean 
text, you should see SOME glyph for each character in the text. (I tested it 
on the KDE Live image some time ago and I did not get the feared black 
boxes.) But that glyph is not necessarily what a speaker of those languages 
expects to see and might in some cases be hard or impossible for them to 
read.

What also speaks against WQY MicroHei is that there has been no release 
since 2009.

The one thing that speaks for it is size: it is only ~4 MiB xz-compressed, 
whereas a typical font for any single CJK language (which may or may not 
have the same limited support as described above for the other 3 languages; 
often, there are no characters at all for the unsupported languages) is
20+ MiB xz-compressed. At the time where the decision to ship it on the KDE 
Spin was made, the choice was between WQY MicroHei or no CJK support at all.

        Kevin Kofler

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Adam Williamson]

On Thu, 2018-03-29 at 22:37 +0200, Kevin Kofler wrote:
>  Adam Williamson wrote:
>  > AIUI, this means it really only works for simplified Chinese users
>  > (which means, approximately, mainland China; other Chinese-speaking
>  > territories tend to use traditional Chinese).
>  
>  AIUI, WQY MicroHei actually tries to cover the traditional-only,
>  Japanese-only or Korean-only glyphs, too. According to 
>  http://wenq.org/wqy2/index.cgi?action=browse&id=Home&lang=en they cover the 
>  whole GBK set and 80% of CJK Extension A. According to
>  http://wenq.org/wqy2/index.cgi?MicroHei_BigBang_README the 2009 release 
>  claimed support for 99% of Japanese, 100% of Korean, and 99% of Traditional 
>  Chinese.
>  
>  The main issue, as I was told, is those CJK Unified codepoints that are 
>  normally rendered/drawn differently in the different countries, but were 
>  unified into a single Unicode codepoint. I was told by the experts that WQY 
>  MicorHei (like all WQY fonts) only contains one rendering (the Simplified 
>  Chinese one) for those, and Wikipedia confirms it: 
>  https://en.wikipedia.org/wiki/WenQuanYi#Glyph
>  
>  So in short, if you point it to Traditional Chinese, Japanese or Korean 
>  text, you should see SOME glyph for each character in the text. (I tested it 
>  on the KDE Live image some time ago and I did not get the feared black 
>  boxes.) But that glyph is not necessarily what a speaker of those languages 
>  expects to see and might in some cases be hard or impossible for them to 
>  read. 
Thanks for the clarification. That does make the case more debatable.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Gerald Henriksen]

On Thu, 29 Mar 2018 22:37:12 +0200, you wrote:

> The one thing that speaks for it is size: it is only ~4 MiB
> xz-compressed, 
> whereas a typical font for any single CJK language (which may or may not 
> have the same limited support as described above for the other 3 languages; 
> often, there are no characters at all for the unsupported languages) is
> 20+ MiB xz-compressed. At the time where the decision to ship it on the KDE 
> Spin was made, the choice was between WQY MicroHei or no CJK support at all. 
I think looking into size increases is useful, if for no other reason
than to ensure mistakes don't get overlooked.

But I also think worrying about an artificial size limit, a limit that
made sense in the past, is damaging.

In the past arguments have been made that we have to keep it small for
those with limited and/or expensive bandwidth.

While a noble goal, it is unfortunately something that is only paid
attention to when creating the live images and then immediately
forgotten.

I just started up one of my Fedora vm's, which was last updated 3 days
ago.

In 3 days I now have 50 packages to update/install with a total
download size of 171M.

So in 3 days I have accumulated downloads totally half the size
increase causing all this angst.  While certainly not perfect, it
roughly gives me 10G of updates to download and install over a 6 month
life span of a typical Fedora release.

Consult the relevant experts, and based on their recommendations
mandate a base set of fonts that provide a quality first experience
with Fedora for everyone regardless of where they live and what
language they read/write.  Making compromises to save 400M on a
distribution that will need 20+G over  year doesn't seem very wise.

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Chris Murphy]

On Fri, Mar 30, 2018 at 8:15 AM, Gerald Henriksen <ghenriks(a)gmail.com&gt; wrote:

>  Consult the relevant experts, and based on their recommendations
>  mandate a base set of fonts that provide a quality first experience
>  with Fedora for everyone regardless of where they live and what
>  language they read/write.  Making compromises to save 400M on a
>  distribution that will need 20+G over  year doesn't seem very wise. 

Compression of small data like this might benefit from tuning. Zstd
offers a training mode to produce a dictionary used for compression,
offering upwards of 3x more compression. Speed of compression and
decompression is also improved. Zstd even without a dictionary
outperforms xz, in particular decompression speeds. But at the sheer
volume of xz (or bzip2?) compression for RPMs and LiveOS images in
koji, I'd bet the compression time savings would be significant,
without dictionary. And if training is possible, the advantage in
compression ratio and performance would be huge.

The Zstd patches for squashfs-tools don't appear to be upstream still
for whatever reason, but patches are available. The kernel code for
squashfs and Btrfs has been there for a while. So implementing this
for LiveOS images is probably a good deal easier than whatever RPM and
koji would have to learn to make it happen for all rpms.



-- 
Chris Murphy

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Peter Robinson]

On Sat, 31 Mar 2018, 03:07 Chris Murphy, <lists(a)colorremedies.com&gt; wrote:

>  On Fri, Mar 30, 2018 at 8:15 AM, Gerald Henriksen
> <ghenriks(a)gmail.com&gt;
>  wrote:
>
>  > Consult the relevant experts, and based on their recommendations
>  > mandate a base set of fonts that provide a quality first experience
>  > with Fedora for everyone regardless of where they live and what
>  > language they read/write.  Making compromises to save 400M on a
>  > distribution that will need 20+G over  year doesn't seem very wise.
>
>
>  Compression of small data like this might benefit from tuning. Zstd
>  offers a training mode to produce a dictionary used for compression,
>  offering upwards of 3x more compression. Speed of compression and
>  decompression is also improved. Zstd even without a dictionary
>  outperforms xz, in particular decompression speeds. But at the sheer
>  volume of xz (or bzip2?) compression for RPMs and LiveOS images in
>  koji, I'd bet the compression time savings would be significant,
>  without dictionary. And if training is possible, the advantage in
>  compression ratio and performance would be huge.
>  
The compression used in rpms is mostly irrelevant in these cases because
when the rpm is installed on disk it's not compressed.

You also have the problem of availability of tools to handle the
compression, xz is a widely used format which has advantages too. In terms
of training I suspect given the wide range of data that gets jammed into
rpms you'd find the overall improvement is minimal.

The Zstd patches for squashfs-tools don't appear to be upstream still
>  for whatever reason, but patches are available. The kernel code for
>  squashfs and Btrfs has been there for a while. So implementing this
>  for LiveOS images is probably a good deal easier than whatever RPM and
>  koji would have to learn to make it happen for all rpms.
>  
Live images are but one of the artifacts that we distribute, we also have
cloud, container, vagrant and the arm images all of which could be used on
top of other OSes which would need wide support for the format. Btrfs is
also not a default in any of the delivered artefacts.

Peter


>  --
>  Chris Murphy
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Jens-Ulrik Petersen]

On Fri, Mar 30, 2018 at 1:22 AM, Adam Williamson <adamwill(a)fedoraproject.org
>  wrote: 
>  On Thu, 2018-03-29 at 16:12 +0200, Kevin Kofler wrote:
>  > wqy-microhei-fonts is very effective, but unfortunately it only covers
>  > Simplified Chinese and the syllabic parts of Japanese and Korean, not the
>  > Traditional Chinese, Japanese or Korean renderings of the CJK Unified
>  > Ideographs.)
>
>  AIUI, this means it really only works for simplified Chinese users
>  (which means, approximately, mainland China; other Chinese-speaking
>  territories tend to use traditional Chinese).
>  
Correct

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Akira TAGOH]

Hi,

Well, there should be things can be improved in the long term as some
topics on the thread already raised and need to keep discussions,
investigation and fixes as needed though, I'd suggest for quick
workaround. that is to ship limited variants only and move other extra
variants into sub-package. here is the estimation:

font sizes for CJK fonts installed by default on f27:
$ rpm -qlva vlgothic-fonts naver-nanum-gothic-fonts
adobe-source-han-*-{cn,tw}-fonts | grep -E 'ttf|otf'|awk
'BEGIN{a=0}{a+=$5}END{print a}'
235970044

font sizes for CJK on f28 so far:
$ rpm -qlva google-noto-*-{jp,kr,tc,sc}-fonts | awk
'BEGIN{a=0}{a+=$5}END{print a}'
508140498

Proposed workaround - font sizes for CJK fonts with Regular only on f28:
$ rpm -qlva google-noto-*-{jp,kr,tc,sc}-fonts | grep Regular | awk
'BEGIN{a=0}{a+=$5}END{print a}'
119507812

another - font sizes for CJK fonts with Regular and Bold only on f28:
$ rpm -qlva google-noto-*-{jp,kr,tc,sc}-fonts | grep -E
'Regular|\-Bold\.' | awk 'BEGIN{a=0}{a+=$5}END{print a}'
243328916

If +7MB is still acceptable, want to ship Regular and Bold together
because it should has better hinting than synthetic emboldening.
otherwise only Regular is fine. that's better than missing serif and
mono like older releases.

The problem on this workaround is that we need an exception for
packaging because the packaging guidelines mentions we must package
variants into the same (sub-)package for family[1]

How does that look for you? any feedbacks are welcome.

[1]...https://fedoraproject.org/wiki/Packaging:FontsPolicy#Package_layout_...

On Thu, Mar 29, 2018 at 11:12 PM, Kevin Kofler <kevin.kofler(a)chello.at&gt; wrote:
>  Hi,
>
>  we did more debugging on #fedora-kde (thanks in particular to lupinix) and
>  we found what seems to be the primary source of the bloat: CJK fonts!
>
>  CJK fonts are by far the largest of all fonts due to the huge number of
>  characters used in those languages.
>
>  Up to Fedora 26, Fedora shipped 4 CJK fonts:
>        <packagereq
> type="default">adobe-source-han-sans-cn-fonts</packagereq>
>        <packagereq
> type="default">adobe-source-han-sans-tw-fonts</packagereq>
>        <packagereq
> type="default">naver-nanum-gothic-fonts</packagereq>
>        <packagereq type="default">vlgothic-fonts</packagereq>
>  The KDE and LXQt Spins actually opted to blacklist these fonts in their
>  kickstart, and ship one compact CJK font instead: wqy-microhei-fonts.
>
>  In Fedora 27, this Change:
>  https://fedoraproject.org/wiki/Changes/ChineseSerifFonts
>  added 2 additional fonts (for a total of 6):
>        <packagereq
> type="default">adobe-source-han-serif-cn-fonts</packagereq>
>        <packagereq
> type="default">adobe-source-han-serif-tw-fonts</packagereq>
>  which were unfortunately missing from the blacklist:
>  https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  already increasing the size of the image.
>
>  But now in Fedora 28, after:
>  https://fedoraproject.org/wiki/Changes/JPDefaultFontsToNoto
>  https://fedoraproject.org/wiki/Changes/KRDefaultFontsToNoto
>  https://fedoraproject.org/wiki/Changes/ChineseDefaultFontsToNoto
>  we actually ship a whopping 12 CJK fonts:
>        <packagereq
> type="default">google-noto-sans-jp-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-kr-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-mono-cjk-jp-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-mono-cjk-kr-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-mono-cjk-sc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-mono-cjk-tc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-sc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-sans-tc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-jp-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-kr-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-sc-fonts</packagereq>
>        <packagereq
> type="default">google-noto-serif-tc-fonts</packagereq>
>  none of which are blacklisted in the Spins! According to lupinix, these
>  amount to a download size (and thus an xz-compressed size, which is also the
>  compression algorithm used for the live images) of 364 MiB!
>
>  The fix is to update the blacklists in the KDE and LXQt spin kickstarts, as
>  per the discussion under:
>  https://bugzilla.redhat.com/show_bug.cgi?id=1530006
>  IMHO, this needs to be implemented, urgently.
>
>  But I think we also need to generally consider whether it makes sense to
>  force 3 font variants for each CJK language on all users worldwide, and
>  whether there are smaller fonts that could be used. (E.g.,
>  wqy-microhei-fonts is very effective, but unfortunately it only covers
>  Simplified Chinese and the syllabic parts of Japanese and Korean, not the
>  Traditional Chinese, Japanese or Korean renderings of the CJK Unified
>  Ideographs.)
>
>  Any additional bloat is likely caused by:
>  * the updated glibc Unicode collation tables,
>  * Annobin,
>  but those seem to have much smaller impact than the above.
>
>          Kevin Kofler
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org 


-- 
Akira TAGOH

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Nicolas Mailhot]

Le vendredi 30 mars 2018 à 16:45 +0900, Akira TAGOH a écrit :
>  
>  The problem on this workaround is that we need an exception for
>  packaging because the packaging guidelines mentions we must package
>  variants into the same (sub-)package for family[1] 
And the problem is not so much the exception to guidelines but why it
was written in the first place.

Installing just part of a font family is massively unintuitive for
users, when they see a font in their font list they expect it to be
complete, when it's there but without bits they need they either
conclude it's immature and stop using it or (if they know it should
contain more parts) that the Fedora version is broken. Very few will
infer “I need to install the missing parts”.

-- 
Nicolas Mailhot

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Akira TAGOH]

Sorry for late response. I somehow missed this mail.

On Sat, Mar 31, 2018 at 7:35 PM, Nicolas Mailhot
<nicolas.mailhot(a)laposte.net&gt; wrote:
>  And the problem is not so much the exception to guidelines but why
> it
>  was written in the first place.
>
>  Installing just part of a font family is massively unintuitive for
>  users, when they see a font in their font list they expect it to be
>  complete, when it's there but without bits they need they either
>  conclude it's immature and stop using it or (if they know it should
>  contain more parts) that the Fedora version is broken. Very few will
>  infer “I need to install the missing parts”. 
Right. but everyone doesn't necessarily need everything. native
speakers and a few people may care about it. I'm wondering if weak
deps and/or langpacks may helps in that case. of course if we can go
ahead without any workaround on this, that would be better though.

>  --
>  Nicolas Mailhot
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org 


-- 
Akira TAGOH

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Kevin Kofler]

Akira TAGOH wrote:
>  of course if we can go ahead without any workaround on this, that
> would be
>  better though. 
FYI, the plan of the KDE SIG is to simply bump the target size of the image 
yet again.

I do not like that approach, but I am not the one who decides (anymore – I 
used to de-facto maintain the KDE Live image for a while, but no longer).

For a long-term solution: In principle, I think shipping a smaller font 
(optimized for size, by composing strokes instead of copy&pasting them, see 
e.g. WQY MicroHei) altogether rather than a subset of Noto would be a better 
approach (because it would avoid the user confusion and would also be even 
more compact), but unfortunately I don't know of any such font that is up to 
date and TC/J/K-friendly (WQY MicroHei is neither of those). (As was pointed 
out elsewhere in the thread, for the different SC/TC/J/K renderings, it 
should be technically possible to put them LOCL-tagged into the one font. 
WQY MicroHei is already a TTC font.) And I also don't think we really need 
all 3 of Sans, Serif and Mono. There are other scripts where we only ship 
one variant, too, and Sans is essentially monospaced for CJK anyway.

        Kevin Kofler

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Nicolas Mailhot]

Le mercredi 04 avril 2018 à 17:23 +0200, Kevin Kofler a écrit :
>  Akira TAGOH wrote:
>  > And I also don't think we really
>  need 
>  all 3 of Sans, Serif and Mono.  
Technically, those are three separate font families that happen to be
more or less coordinated, and culling along font family lines is not
problematic (as opposed to culling within the weight width or stretch
variants of a single family)

Regards,
-- 
Nicolas Mailhot

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Jens-Ulrik Petersen]

On Sat, Mar 31, 2018 at 7:35 PM, Nicolas Mailhot <
nicolas.mailhot(a)laposte.net&gt; wrote:

>  Le vendredi 30 mars 2018 à 16:45 +0900, Akira TAGOH a écrit :
>  > The problem on this workaround is that we need an exception for
>  > packaging because the packaging guidelines mentions we must package
>  > variants into the same (sub-)package for family[1]
>  

>  Installing just part of a font family is massively unintuitive for
>  users, when they see a font in their font list they expect it to be
>  complete, when it's there but without bits they need they either
>  conclude it's immature and stop using it or (if they know it should
>  contain more parts) that the Fedora version is broken. Very few will
>  infer “I need to install the missing parts”.
>  
Google Noto CJK has 7 weight variants for each Sans and Serif face... [1]
In addition to Regular and Bold, the Noto Sans and Serif CJK fonts have 5
other weight variants.
(We have already shipped the equivalent Adobe Source Han Sans and Serif
fonts for Chinese since 2014.)

So that is 7 variants for both Sans and Serif for each of Japanese, Korean,
Simplified Chinese, and Traditional Chinese. And these are not small
fonts...

7 × 4 × 2 = 56 large OTF fonts

This seems a lot for most Fedora users.
It is hard to see how global Fedora users would benefit from having 56 CJK
.otf files installed by default.
I do not know how much most native users even benefit from having so many
weight variants available by default.

Additionally the Noto Sans Mono CJK fonts, needed for terminals, are much
larger since they include full CJK coverage for each of the languages.

In contrast the rest of the Noto (non-CJK) fonts only have Regular and Bold
weights, and of course compared to CJK are rather small anyway. [1]

If we could subpackage the uncommon weights it would allow decreasing the
font footprint for East Asian fonts considerably while still improving the
UX over F27 for Japanese and Korean (and largely preserving Chinese).
We could use langpack weak deps to pull in the extra weights for the native
desktop locale.

For fun U already made a copr repo with subpackaging and weakdeps:
https://copr.fedorainfracloud.org/coprs/petersen/google-noto-cjk-fonts/

In future (hopefully for F29) switching to OpenType Collections of the Noto
CJK TTC fonts would provide a considerable space saving since they combine
the Sans and Mono fonts into a share font file, and Serif would be smaller
too.

Failing all this should we drop the Serif fonts for Japanese and Korean -
but it seems unfair to single them out?

Jens

[1] https://www.google.com/get/noto/help/cjk/
[2] https://www.google.com/get/noto/

ps For more size analysis by Akira see
https://bugzilla.redhat.com/show_bug.cgi?id=1562315

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Kevin Kofler]

Jens-Ulrik Petersen wrote:
>  Failing all this should we drop the Serif fonts for Japanese and
> Korean -
>  but it seems unfair to single them out? 
So just drop ALL the CJK Serif fonts, and also the monospace ones (CJK Sans 
is essentially monospaced anyway), bringing us back to where we were in F26 
(except that we were also using smaller fonts than Noto there, may be worth 
reverting to those, too).

And IMHO the KDE and LXQt Spins should revert to shipping only
wqy-microhei-fonts and no other CJK fonts. They never promised CJK support 
beyond "we can bring up SOME rendering for every character of the typical 
SC/TC/J/K text" (and the support is and has always been read-only anyway, no 
input methods).

        Kevin Kofler

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Jens-Ulrik Petersen]

On Tue, Apr 10, 2018 at 9:11 AM, Kevin Kofler <kevin.kofler(a)chello.at&gt;
wrote:

>  And IMHO the KDE and LXQt Spins should revert to shipping only
>  wqy-microhei-fonts and no other CJK fonts. They never promised CJK support
>  beyond "we can bring up SOME rendering for every character of the typical
>  SC/TC/J/K text" (and the support is and has always been read-only anyway,
>  no
>  input methods). 

I didn't respond earlier but using the WQY fonts for Japanese, Korean, or
Traditional Chinese is not acceptable.
They are purely for Simplified Chinese.

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Tomasz Torcz 👁️]

On Tue, Apr 10, 2018 at 02:11:40AM +0200, Kevin Kofler wrote:
>  Jens-Ulrik Petersen wrote:
>  > Failing all this should we drop the Serif fonts for Japanese and Korean -
>  > but it seems unfair to single them out?
>  
>  So just drop ALL the CJK Serif fonts, and also the monospace ones (CJK Sans 
>  is essentially monospaced anyway), bringing us back to where we were in F26 
>  (except that we were also using smaller fonts than Noto there, may be worth 
>  reverting to those, too).
>  
>  And IMHO the KDE and LXQt Spins should revert to shipping only
>  wqy-microhei-fonts and no other CJK fonts. They never promised CJK support 
>  beyond "we can bring up SOME rendering for every character of the typical 
>  SC/TC/J/K text" (and the support is and has always been read-only anyway, no 
>  input methods). 
  While we at it, let's drop all latin fonts, too. Cyrillic should be
enable to cover most of the world's usage and is quite similar to basic
latin.

-- 
Tomasz Torcz                                                       72->|   80->|
xmpp: zdzichubg(a)chrome.pl                                          72->|   80->|

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Zbigniew Jędrzejewski-Szmek]

On Tue, Apr 10, 2018 at 06:27:10AM +0200, Tomasz Torcz ?️ wrote:
>  On Tue, Apr 10, 2018 at 02:11:40AM +0200, Kevin Kofler wrote:
>  > Jens-Ulrik Petersen wrote:
>  > > Failing all this should we drop the Serif fonts for Japanese and Korean -
>  > > but it seems unfair to single them out?
>  > 
>  > So just drop ALL the CJK Serif fonts, and also the monospace ones (CJK Sans 
>  > is essentially monospaced anyway), bringing us back to where we were in F26 
>  > (except that we were also using smaller fonts than Noto there, may be worth 
>  > reverting to those, too).
>  > 
>  > And IMHO the KDE and LXQt Spins should revert to shipping only
>  > wqy-microhei-fonts and no other CJK fonts. They never promised CJK support 
>  > beyond "we can bring up SOME rendering for every character of the typical 
>  > SC/TC/J/K text" (and the support is and has always been read-only anyway, no 
>  > input methods).
>  
>    While we at it, let's drop all latin fonts, too. Cyrillic should be
>  enable to cover most of the world's usage and is quite similar to basic
>  latin. 
+один

збишек

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Kevin Kofler]

Tomasz Torcz 👁️ wrote:
>    While we at it, let's drop all latin fonts, too. Cyrillic
> should be
>  enable to cover most of the world's usage and is quite similar to basic
>  latin. 
Нет! ;-)

But more seriously, this is not a fair comparison. Latin fonts typically 
take less than 1 MiB (for a pure Latin font – fonts like DejaVu that cover 
much more than that are of course larger!) and cover up to 135 languages 
(https://en.wikipedia.org/wiki/List_of_languages_by_writing_system#Latin_s...). 
CJK fonts typically take more than 20 MiB (the smallest is WQY MicroHei that 
already weighs more than 5 MiB) and typically cover 1 language (and could at 
most cover 4 languages).

        Kevin Kofler

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Gerald Henriksen]

On Tue, 10 Apr 2018 13:36:33 +0200, you wrote:

> Tomasz Torcz ?? wrote:
> >   While we at it, let's drop all latin fonts, too. Cyrillic should be
> > enable to cover most of the world's usage and is quite similar to basic
> > latin.
>
> ???! ;-)
>
> But more seriously, this is not a fair comparison.  
Why should some people of the world be excluded from Fedora though?

Shouldn't the priority be to make Fedora welcoming to all, and not to
limiting the audience based on an artificial size limitation?

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Adam Williamson]

On Tue, 2018-04-10 at 08:27 -0400, Gerald Henriksen wrote:
>  On Tue, 10 Apr 2018 13:36:33 +0200, you wrote:
>  
>  > Tomasz Torcz ?? wrote:
>  > >   While we at it, let's drop all latin fonts, too. Cyrillic should be
>  > > enable to cover most of the world's usage and is quite similar to basic
>  > > latin.
>  > 
>  > ???! ;-)
>  > 
>  > But more seriously, this is not a fair comparison. 
>  
>  Why should some people of the world be excluded from Fedora though?
>  
>  Shouldn't the priority be to make Fedora welcoming to all, and not to
>  limiting the audience based on an artificial size limitation? 
This is actually a tricky equation, because the reason to care about
size is actually *the same*: making Fedora 'welcoming'. Not everyone
has a big fat broadband pipe and a huge transfer cap (or unlimited
transfers). So if we include lots of i18n support we're making Fedora
'more welcoming' in one way, but *less* welcoming in another way, to a
different (but probably overlapping) set of folks. It's a tricky
conundrum.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Chris Murphy]

On Thu, Apr 12, 2018 at 11:30 AM, Adam Williamson
<adamwill(a)fedoraproject.org&gt; wrote:
>  On Tue, 2018-04-10 at 08:27 -0400, Gerald Henriksen wrote: 
> > Shouldn't the priority be to make Fedora welcoming to all,
> and not to
> > limiting the audience based on an artificial size limitation?
>
>  This is actually a tricky equation, because the reason to care about
>  size is actually *the same*: making Fedora 'welcoming'. Not everyone
>  has a big fat broadband pipe and a huge transfer cap (or unlimited
>  transfers). So if we include lots of i18n support we're making Fedora
>  'more welcoming' in one way, but *less* welcoming in another way, to a
>  different (but probably overlapping) set of folks. It's a tricky
>  conundrum. 
My experience with crap bandwidth: I'm way more sensitive to the
persistent metadata downloads, than that of rare and planned ISO
download size.

1Mbps download means a 1.6G ISO is 3h50m and a 1.9G ISO is 4h30m. That
difference doesn't even compute, I'm gonna use curl and walk away no
matter what. And when I had the pure amazing joy of 350kbps recently,
this is now 11 vs 13 hours, again it's the same: craptastic.

People who are bandwidth sensitive, especially if they have multiple
computers, need to setup a local mirror. *shrug*

https://pagure.io/fedora-workstation/issue/46 says it might be too
late to use OTC fonts but doesn't definitively state that it is. With
five days before freeze: OTC vs subpackage vs grow the ISO by 260MB vs
use Fedora 27 fonts. I would push for going in that order through
whatever path is the least resistant assuming no already known
liabilities and all of those have been tested. Going with Fedora 27
fonts I'd consider the last restort, it's a conservative fallback, it
will work but with no improvements for Fedora 28.



-- 
Chris Murphy

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Jens-Ulrik Petersen]

Thanks Chris for the follow up.

I also wanted to go for OTC fonts, but both Akira Tagoh and Owen Taylor who
are fonts rendering experts advised against switching this late in the
cycle to the .ttc fonts in case there are any serious regressions for any
apps (we are not aware of any yet but there could well be for some special
apps that use fonts more directly). We should probably have been braver and
tried for OTC from the start: I guess we were a little cautious in the
beginning with this switch, and hadn't fully realised the size impact - the
changes went in 2 months ago though...

So we made the hard but safe decision to revert back to the f27 CJK font
set for F28 GA alas.

While the initial reaction was that the size increase was not such a big
deal, it is now release blocker, and we are not getting sufficient help
with the alternative workarounds so we felt better to revert early than
have to scramble right before GA RC's etc or even delay the release because
of this issue.

Thanks, Jens


On Fri, Apr 13, 2018 at 5:38 AM, Chris Murphy <lists(a)colorremedies.com&gt;
wrote:

>  On Thu, Apr 12, 2018 at 11:30 AM, Adam Williamson
>  <adamwill(a)fedoraproject.org&gt; wrote:
>  > Not everyone has a big fat broadband pipe and a huge transfer cap (or
>  unlimited
>  > transfers).
>  
It also put more pressure on our mirrors: with the many iso files
increasing in size.

Unacceptable size increase to ALL live images in F28: Noto CJK Fonts

[Nicolas Mailhot]

Le 2018-04-10 02:11, Kevin Kofler a écrit :
>  Jens-Ulrik Petersen wrote:
> > Failing all this should we drop the Serif fonts for Japanese and 
> > Korean -
> > but it seems unfair to single them out?
>  
>  So just drop ALL the CJK Serif fonts, 
Yes, there’s nothing unfair there, better a single working complete font 
family than a set of incomplete fonts missing bits right and left.

Regards,

-- 
Nicolas Mailhot