5:47 a.m.
Hello,
I've got an idea and I'd like to know an opinion of a wider audience.
Would it make sense to split translations from binary packages to
a noarch subpackage?
For example libreoffice does that already, it even splits the languages
by country, which may or may not be applicable to other (larger)
packages as well.
What this could help with is a save of disk space on the build servers
and mirrors. There might not be any significant difference for users,
due to the main package would require the "langs" subpackage [*]. One
may even avoid multilib issues on translation files (happened in the
past for Evolution due to it removing unused translation strings during
the build, to save package size).
To give an example, I tried this with Evolution package and the result
is that the langs subpackage is ~6MB large, while the main package is
~4MB large. Count that the languages are stored for each architecture,
and koji builds for 6 of them at the moment, then the change of split
means saving 30MB on disk for each single build of Evolution. When
searching koji for Evolution packages then there had been done more
than 70 builds since the first build for Fedora 24 (which is built for
3 arches only), which means more than 1GB on disk (roughly, counting 3
arches only). Multiply this by number of mirrors and so on.
I know I can do this for packages I maintain, but I though it would
make sense to think of it globally. Maybe?
Bye,
Milan
[*] The only difference for users would be to not download languages
again when installing two+ architectures in parallel, thus less data to
download, thus only a benefit for them.
7:18 a.m.
On 04/24/2017 06:47 AM, Milan Crha wrote:
Hello,
I've got an idea and I'd like to know an opinion of a wider audience.
Would it make sense to split translations from binary packages to
a noarch subpackage?
For example libreoffice does that already, it even splits the languages
by country, which may or may not be applicable to other (larger)
packages as well.
What this could help with is a save of disk space on the build servers
and mirrors. There might not be any significant difference for users,
due to the main package would require the "langs" subpackage [*]. One
may even avoid multilib issues on translation files (happened in the
past for Evolution due to it removing unused translation strings during
the build, to save package size).
To give an example, I tried this with Evolution package and the result
is that the langs subpackage is ~6MB large, while the main package is
~4MB large. Count that the languages are stored for each architecture,
and koji builds for 6 of them at the moment, then the change of split
means saving 30MB on disk for each single build of Evolution. When
searching koji for Evolution packages then there had been done more
than 70 builds since the first build for Fedora 24 (which is built for
3 arches only), which means more than 1GB on disk (roughly, counting 3
arches only). Multiply this by number of mirrors and so on.
I know I can do this for packages I maintain, but I though it would
make sense to think of it globally. Maybe?
Bye,
Milan
[*] The only difference for users would be to not download languages
again when installing two+ architectures in parallel, thus less data to
download, thus only a benefit for them.
If we went this route, I'd love to see us attempt to solve this
generically for all packages if at all possible. Something like having
the rpmbuild step extract the *.po files automatically into their own
subpackages and set the appropriate `Supplements: langpack-<lang>`[1]
values for them. That way the next time we do a mass-rebuild, every
package in the distribution would take advantage of it automatically.
[1] https://fedoraproject.org/wiki/Packaging:Langpacks
11:15 a.m.
On Mon, 2017-04-24 at 08:18 -0400, Stephen Gallagher wrote:
If we went this route, I'd love to see us attempt to solve this
generically for all packages if at all possible.
Hi,
right, having this done transparently for the packagers would be ideal.
You only need to decide from which build/architecture you'll compose
that noarch package. Evolution package used to remove unused translated
strings in the past, during the build.
My main issue with libreoffice langpacks divided by country was that
even I chose my mother language during installation of Fedora, that
particular langpack wasn't installed, thus libreoffice was not
localized to my mother language, though other parts, like GNOME Shell,
were. This is few Fedora's back, on a machine which I keep updating,
instead of installing from scratch, thus it's possible the behaviour
changed meanwhile.
I mean, maybe it doesn't make always sense to divide langpacks by
country.
Bye,
Milan
1:37 p.m.
On 04/24/2017 12:15 PM, Milan Crha wrote:
On Mon, 2017-04-24 at 08:18 -0400, Stephen Gallagher wrote:
> If we went this route, I'd love to see us attempt to solve this
> generically for all packages if at all possible.
Hi,
right, having this done transparently for the packagers would be ideal.
You only need to decide from which build/architecture you'll compose
that noarch package. Evolution package used to remove unused translated
strings in the past, during the build.
My main issue with libreoffice langpacks divided by country was that
even I chose my mother language during installation of Fedora, that
particular langpack wasn't installed, thus libreoffice was not
localized to my mother language, though other parts, like GNOME Shell,
were. This is few Fedora's back, on a machine which I keep updating,
instead of installing from scratch, thus it's possible the behaviour
changed meanwhile.
I mean, maybe it doesn't make always sense to divide langpacks by
country.
These days, I think we divide langpacks (at least at the distro/glibc
level) by language rather than country.
I have `langpacks-en` on my system, for example.
3:33 p.m.
On 24 April 2017 at 20:37, Stephen Gallagher <sgallagh(a)redhat.com> wrote:
These days, I think we divide langpacks (at least at the
distro/glibc
level) by language rather than country.
I have `langpacks-en` on my system, for example.
Yep .. "these days" glibc is one of the best anti examples about how to
write specs files.
kloczek
--
Tomasz Kłoczko | LinkedIn: *http://lnkd.in/FXPWxH <http://lnkd.in/FXPWxH>*
6:31 a.m.
Hi,
On Mon, Apr 24, 2017 at 06:15:49PM +0200, Milan Crha wrote:
On Mon, 2017-04-24 at 08:18 -0400, Stephen Gallagher wrote:
> If we went this route, I'd love to see us attempt to solve this
> generically for all packages if at all possible.
My main issue with libreoffice langpacks divided by country was that
even I chose my mother language during installation of Fedora, that
particular langpack wasn't installed, thus libreoffice was not
localized to my mother language, though other parts, like GNOME Shell,
were. This is few Fedora's back, on a machine which I keep updating,
instead of installing from scratch, thus it's possible the behaviour
changed meanwhile.
https://bugzilla.redhat.com/show_bug.cgi?id=1401096
I mean, maybe it doesn't make always sense to divide langpacks by
country.
There is a reason that libreoffice puts l10n data into separate
subpackages. libreoffice currently builds with 69 languages. The size of
UI l10n data is 2-3 MB per language. In addition, about half of the
langpacks contain help. That's another 20+ MB per language. That makes
1.4 GB of installed size (194 MB even without help). Plus dependencies:
the langpacks pull in hunspell, mythes, autocorr and font packages for
given langugage.
Btw, I'm not sure what do you mean by "divide langpacks by country". The
langpacks are divided by language; only pt is further divided by region
(pt-PT and pt-BR) and zh by variant (zh-Hans and zh-Hant).
D.
7:07 a.m.
On Thu, 2017-04-27 at 13:31 +0200, David Tardon wrote:
There is a reason that libreoffice puts l10n data into separate
subpackages.
Hi,
yes, there is a very good reason for it and I'm not questioning that.
libreoffice was just an example of a package which does that already,
but also fails to do the right (or better "expected by me") thing, for
which you gave a bug link. Thanks for it.
As had been said in this thread, one approach doesn't work for all
packages. I made it my way for the core evolution packages this morning
for rawhide. There is only one langpacks subpackage. I looked also on
syncevolution, because I've been curious, and the split translations
package was around 90KB of size, thus it didn't make any sense to split
the translations and I left syncevolution unchanged. That makes me
think that there are these kinds of packages:
a) without any translation - no changes for these
b) with only a small translations - like syncevolution, split doesn't
make sense
c) with semi-large translations - packages, where split translations by
language is less useful or impractical, thus one package with all
languages is preferred/used
d) with large translations - languages are that large that it's
better to split the langpacks into per-language subpackages
e) with large translations and additions - like libreoffice,
translations split by language as well, but libreoffice also adds
more dependencies to these translation packages (as you said).
I did with evolution core packages what I thought is the best. I do not
use weak dependencies, I define a hard dependency between the langpacks
and the binary package, both ways, thus users get either both or none
of them. That makes the change backward compatible, while still helping
with disk usage on build servers and mirrors. One part of backward
compatibility in this case is also related to d),e) above, where, for
the way I made it, users can switch languages and they will have the UI
translated without a need to install additional package(s).
When starting this thread here, I did not intent to suggest any change
on the build servers to do things automatically, I was more interested
in ideas and opinions of other packagers. It would be nice to have
these things done transparently, with minimal lines in RPM .spec file,
but as there are a-b-c-d-e cases named above (and/or there can be found
even more), then it makes it impractical. If this thread will lead to
some "nice-to-have packaging guideline", then it would be welcome and
more than I expected from it.
Btw, I'm not sure what do you mean by "divide langpacks by
country".
It's just a terminology. What I call "by country", other (more
educated) folks call "by language/region/variant". Simply
not-all-translations in one single subpackage.
Thanks and bye,
Milan
1:05 p.m.
On Thu, Apr 27, 2017 at 02:07:34PM +0200, Milan Crha wrote:
I did with evolution core packages what I thought is the best. I do
not
use weak dependencies, I define a hard dependency between the langpacks
and the binary package, both ways, thus users get either both or none
of them. That makes the change backward compatible, while still helping
with disk usage on build servers and mirrors.
There is a better way to preserve backwards compat:
define Obsoletes: %{name} < [version] in both the main package and the
langpacks packages. Then dnf will install both on upgrade, and users
can still uninstall langpacks.
See https://bugzilla.redhat.com/show_bug.cgi?id=1260394 for some discussion.
Zbyszek
2:25 p.m.
Seems like splitting langpacks subpackages in all evolution packages few
minutes ago just started.
No generall agreement for such changes, ignoring already mentioned
arguments.
Seems only just because "we can".
Guys if you are bored just please undress, sit down next to your clothes
and watch your cloths. This will be way more productive.
kloczek
PS. Did anyone notice that Fedora in popularity ranking on distrowatch.com
dropped recently?
On Thu, 27 Apr 2017 at 20:07, Zbigniew Jędrzejewski-Szmek <zbyszek(a)in.waw.pl>
wrote:
On Thu, Apr 27, 2017 at 02:07:34PM +0200, Milan Crha wrote:
> I did with evolution core packages what I thought is the best. I do not
> use weak dependencies, I define a hard dependency between the langpacks
> and the binary package, both ways, thus users get either both or none
> of them. That makes the change backward compatible, while still helping
> with disk usage on build servers and mirrors.
There is a better way to preserve backwards compat:
define Obsoletes: %{name} < [version] in both the main package and the
langpacks packages. Then dnf will install both on upgrade, and users
can still uninstall langpacks.
See https://bugzilla.redhat.com/show_bug.cgi?id=1260394 for some
discussion.
Zbyszek
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Tomasz Kłoczko | Tel: 0774 1209067 | LinkedIn: *http://lnkd.in/FXPWxH
<http://lnkd.in/FXPWxH>*
2:46 p.m.
On Thu, Apr 27, 2017 at 07:25:32PM +0000, Tomasz Kłoczko wrote:
Seems like splitting langpacks subpackages in all evolution packages
few
minutes ago just started.
No generall agreement for such changes, ignoring already mentioned
arguments.
Seems only just because "we can".
That's true. Within the existing guidelines and policies, we give our
packagers a lot of latitude for solving the problems in the packages
they maintain. If we decide we need a general policy for something, we
can do that, but we don't have to nor do we always want to.
Guys if you are bored just please undress, sit down next to your
clothes
and watch your cloths. This will be way more productive.
Seriously, please. This is not constructive.
PS. Did anyone notice that Fedora in popularity ranking on
distrowatch.com
dropped recently?
That seems... decidedly unrelated. And, Distrowatch is a silly thing to
use as a metric for _anything_. As it says itself, the rankings
"correlate neither to usage nor to quality and should not be used to
measure the market share of distributions."
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2:50 p.m.
OK so what kind of problem solves those changes? Can I have look on some
bugzilla entry?
kloczek
On Thu, 27 Apr 2017 at 21:47, Matthew Miller <mattdm(a)fedoraproject.org>
wrote:
On Thu, Apr 27, 2017 at 07:25:32PM +0000, Tomasz Kłoczko wrote:
> Seems like splitting langpacks subpackages in all evolution packages few
> minutes ago just started.
> No generall agreement for such changes, ignoring already mentioned
> arguments.
> Seems only just because "we can".
That's true. Within the existing guidelines and policies, we give our
packagers a lot of latitude for solving the problems in the packages
they maintain. If we decide we need a general policy for something, we
can do that, but we don't have to nor do we always want to.
> Guys if you are bored just please undress, sit down next to your clothes
> and watch your cloths. This will be way more productive.
Seriously, please. This is not constructive.
> PS. Did anyone notice that Fedora in popularity ranking on
distrowatch.com
> dropped recently?
That seems... decidedly unrelated. And, Distrowatch is a silly thing to
use as a metric for _anything_. As it says itself, the rankings
"correlate neither to usage nor to quality and should not be used to
measure the market share of distributions."
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
--
Tomasz Kłoczko | Tel: 0774 1209067 | LinkedIn: *http://lnkd.in/FXPWxH
<http://lnkd.in/FXPWxH>*
3:02 p.m.
On Thu, Apr 27, 2017 at 07:50:40PM +0000, Tomasz Kłoczko wrote:
OK so what kind of problem solves those changes? Can I have look on
some
bugzilla entry?
Talking to the maintainers of that package nicely is generally the best
approach. Or even better, *build* that better approach and *show*.
Telling people that we should switch to a theoretical solution that you
*say* would be quite easy is a lot different from actually _making_ it
easy for people.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2:13 a.m.
Hi,
On Thu, 2017-04-27 at 19:25 +0000, Tomasz Kłoczko wrote:
No generall agreement for such changes, ignoring already mentioned
arguments.
Seems only just because "we can".
Nope, you are wrong. I gave clear explanation why I did so. Sure, some
people, like you, might not like it. Okay, that happens. Different
people has different point of view, different opinions, different
priorities.
In any way, the way you speak in this thread makes me just ignore you
specifically, because you are not adding anything constructive. I'm
sorry. That's just my opinion.
Bye,
Milan
2:09 p.m.
On Fri, 28 Apr 2017 at 09:19, Milan Crha <mcrha(a)redhat.com> wrote:
Different
people has different point of view, different opinions, different
priorities.
Can you tell us something about those priorities or maybe point to some URL
where those priorities are listed?
I'm not like you RedHat employee so I have no idea about what you are
talking about.
On what bases your different opinion and what exactly is this opinion?
Did you ever try to use IPS? Did you saw its source code or how it works?
kloczek
3:28 p.m.
On Fri, 2017-04-28 at 21:09 +0200, Tomasz Kłoczko wrote:
Can you tell us something about those priorities
Hi,
I've been generally speaking. Your "priority" is obviously IPS. Mine
not. Read the very first mail in this thread, to see my priorities and
why I started it.
or maybe point to some URL where those priorities are listed?
I do not have any.
I'm not like you RedHat employee so I have no idea about what you
are
talking about.
I see. I'm not talking for Red Hat, I'm talking for myself.
Did you ever try to use IPS?
No.
Did you saw its source code or how it works?
Neither this.
I'm just an average packager. I thought I'll ask wider audience about
an idea for something small and simple, nothing like changing packaging
system, need a learn new tool or whatever, just something simple which
can have interesting impact, but I didn't think I'll be punished for
it, by you, this way. Again, I speak for myself.
Anyway, this thread is over for me. Thank you for a valuable input.
Have a nice weekend,
Milan
6:16 p.m.
24.04.2017 12:47 Milan Crha <mcrha(a)redhat.com> wrote:
[...]
I know I can do this for packages I maintain, but I though it would
make sense to think of it globally. Maybe?
Bye,
Milan
If I may drop my 2¢… this sounds good to me for large packages,
for example LibreOffice, glibc and KDE which do it already and
Evolution as a good candidate to implement this change.
But for small packages which have not so many translatable
messages producing dozens of small RPMs ~1 kilobyte each or even
less would doesn't sound like a good solution. So I suggest to
introduce this feature but not globally and obligatorily for
all packages.
Additionally, I think that for some purposes installing all
languages for some specific applications would be useful so
I also suggest providing something like glibc-all-langpacks.
Regards,
Rafal
6:10 a.m.
On 25 April 2017 at 01:16, Rafal Luzynski
<digitalfreak(a)lingonborough.com> wrote:
But for small packages which have not so many translatable
messages producing dozens of small RPMs ~1 kilobyte each or even
less would doesn't sound like a good solution. So I suggest to
introduce this feature but not globally and obligatorily for
all packages.
Additionally, I think that for some purposes installing all
languages for some specific applications would be useful so
I also suggest providing something like glibc-all-langpacks.
So you guys want to say that you never heard that in %files is
possible to add %lang(<language>) tokens and when you will put in
/etc/rpm/rpmrc line:
%_install_langs <lang1>[,<lang2>,[..]]
or
%_install_langs all
you evey install new or upgrade existing package will cause that only
resources marked in %files by exact %lang() tokens will be installed?
???
And you never heard about this possibility to add %packages section
name param like:
%packages --instLangs <lang1>[,<lang2>,[..]]
by which you initial installation using anaconda installed will be
done only with installed files marked only exact languages tokens?
Really?
I'm really disappointed as I'm personally contributed more than 10
years ago some parts in find-lang.sh script which is behind %find_lang
macro which simplifies adding %lang() tokens to some well known
classes of files which may be a part of some packages.
Looks as well like some people never heard that in %files section is
possible to add %doc tokens and by this possible to choose install
system without installed documentation (and this is why this overkill
with doc packages already happens). Installation with documentation
exclusion is fully supported by rpm, dff and anaconda installer as
well.
kloczek
--
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
10:41 a.m.
On Tue, Apr 25, 2017 at 7:10 AM, Tomasz Kłoczko
<kloczko.tomasz(a)gmail.com> wrote:
On 25 April 2017 at 01:16, Rafal Luzynski
<digitalfreak(a)lingonborough.com> wrote:
>
> But for small packages which have not so many translatable
> messages producing dozens of small RPMs ~1 kilobyte each or even
> less would doesn't sound like a good solution. So I suggest to
> introduce this feature but not globally and obligatorily for
> all packages.
>
> Additionally, I think that for some purposes installing all
> languages for some specific applications would be useful so
> I also suggest providing something like glibc-all-langpacks.
So you guys want to say that you never heard that in %files is
possible to add %lang(<language>) tokens and when you will put in
/etc/rpm/rpmrc line:
%_install_langs <lang1>[,<lang2>,[..]]
or
%_install_langs all
you evey install new or upgrade existing package will cause that only
resources marked in %files by exact %lang() tokens will be installed?
???
And you never heard about this possibility to add %packages section
name param like:
%packages --instLangs <lang1>[,<lang2>,[..]]
by which you initial installation using anaconda installed will be
done only with installed files marked only exact languages tokens?
Really?
I'm really disappointed as I'm personally contributed more than 10
years ago some parts in find-lang.sh script which is behind %find_lang
macro which simplifies adding %lang() tokens to some well known
classes of files which may be a part of some packages.
Looks as well like some people never heard that in %files section is
possible to add %doc tokens and by this possible to choose install
system without installed documentation (and this is why this overkill
with doc packages already happens). Installation with documentation
exclusion is fully supported by rpm, dff and anaconda installer as
well.
The problem with this approach is that it's not possible for the user
to get additional locale content without reinstalling after enabling
another locale (by installing the associated system langpack package
that controls this). Splitting them out into langpack subpackages
guarantees that it's possible to do that.
The filtering is useful for inconsequential ones, where it's just not
worth it to split, but otherwise, it's not.
--
真実はいつも一つ！/ Always, there's only one truth!
2:55 p.m.
On 26 April 2017 at 17:41, Neal Gompa <ngompa13(a)gmail.com> wrote:
The problem with this approach is that it's not possible for the
user
to get additional locale content without reinstalling after enabling
another locale (by installing the associated system langpack package
that controls this). Splitting them out into langpack subpackages
guarantees that it's possible to do that.
No it is not a problem generally at all.
This has been solved in Solaris IPS 6-7 years ago.
As long as package in IPS does not exist in form of archive and it
exist only as object in repository operation like:
# pkg change-facet lang.fr=true
checks in repository which one files are marked by "facet
lang.fr=true" and adds those files to the already installed files
across all already installed packages. Because packages all packages
exists in repo as sets of files with some attributes pkg downloads
only those missing files (not whole packages).
IPS developers learned from RPM about %doc and %lang() nd they've
chose some generic markings over "facet" and they've changed it
generic technique. Truth is that in recent years RPM stagnated and
effectively is much simpler and at the same time much more advanced
than RPM.
Instead trying to find "workaround" RPM should follow he same path
which IPS proven that it may solve a lot more serious problems on
front of which are now Modules Fedora Project developers with provide
alternatives within packages management.
RPM with current paradigm of the packages will be not able to solve
those challenges. If RPM will be not extended to start operate on
packages existing only in repositories IMO more likely some Linux
folks will fork for example Fedora to build Linux distribution using
IPS instead RPM. It will be quite easy because using pkgbuild
(https://pkgbuild.sf.net/)is possible to build IPS packages using only
slightly adapted RPM spec files.
kloczek
--
Tomasz Kłoczko | LinkedIn: http://lnkd.in/FXPWxH
2:21 a.m.
On Tue, 2017-04-25 at 13:10 +0200, Tomasz Kłoczko wrote:
So you guys want to say that you never heard that in %files is
possible to add %lang(<language>) tokens
Hi,
no, I never heard about that, though I'm not a good packager, thus it
doesn't mean much. Though it can be because %lang() is way too
complicated with compare of "Recommended method of handling i18n
files" [1] as referenced from [2], which is significantly easier for
packagers like me and which is used in evolution packages for years.
Bye,
Milan
[1] https://fedoraproject.org/wiki/Packaging:Guidelines#Handling_Locale_Files
[2] https://fedoraproject.org/wiki/How_to_create_an_RPM_package
2:08 a.m.
Hello all,
I agree with Rafal. I think that in the case of small or medium
packages this would be counterproductive.
Kind regards,Silvia
On Tue, 2017-04-25 at 01:16 +0200, Rafal Luzynski wrote:
24.04.2017 12:47 Milan Crha <mcrha(a)redhat.com> wrote:
> [...]
> I know I can do this for packages I maintain, but I though it would
> make sense to think of it globally. Maybe?
>
> Bye,
> Milan
If I may drop my 2¢… this sounds good to me for large packages,
for example LibreOffice, glibc and KDE which do it already and
Evolution as a good candidate to implement this change.
But for small packages which have not so many translatable
messages producing dozens of small RPMs ~1 kilobyte each or even
less would doesn't sound like a good solution. So I suggest to
introduce this feature but not globally and obligatorily for
all packages.
Additionally, I think that for some purposes installing all
languages for some specific applications would be useful so
I also suggest providing something like glibc-all-langpacks.
Regards,
Rafal
_______________________________________________
devel mailing list -- devel(a)lists.fedoraproject.org
To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
9:55 a.m.
On Tue, Apr 25, 2017 at 01:16:44AM +0200, Rafal Luzynski wrote:
But for small packages which have not so many translatable
messages producing dozens of small RPMs ~1 kilobyte each or even
less would doesn't sound like a good solution. So I suggest to
introduce this feature but not globally and obligatorily for
all packages.
I think we should look at this from a different angle. I think we
should split them out where the languages are large _or_ the packages
are likely to go in release artifacts where size is at a premium:
Atomic Host, Container Base Image, and the Live images. If it's only a
few K and isn't used in a size-important case, skip it.
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
2566
days inactive
2570
days old
21 comments
9 participants
participants (9)
-
David Tardon -
Matthew Miller -
Milan Crha -
Neal Gompa -
Rafal Luzynski -
Silvia Sanchez -
Stephen Gallagher -
Tomasz Kłoczko -
Zbigniew Jędrzejewski-Szmek