Hi!
I have several very similar packages being reviewed and two different reviewers reviewing different packages have reach contradicting conclusions about how the packaging guidelines should be interpreted. Since it doesn't make sense that the same issue is resolved differently depending on who happens to be the reviewer, and the reviewers have failed to reach a common viewpoint I send this mail to this list in the hope that there will be a way to resolve the issue consistently for all packages.
Here is a description of the problem at hand:
When upstream distributes sources in a gigantic installer containing the sources for 300+ packages it doesn't make sense to include this full tarfile for each SRPM, since less than 1% of it is used to compile each package. Instead the relevant subdirectory is extracted from this beast (properly documented in the specfile in accordance to the packaging guidelines).
Then the question is how should the following guideline be interpreted:
"If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %doc."
Does this text refer to the big gigantic installer or the extracted source tarfile in this case.
One reviewer strongly argues the first point and will only approve packages where the license file is included, the other strongly argues the second point and will only approve packages where the license file is not included.
Both quote the above rule as the foundation for there standpoint. Since it doesn't make sense to do things differently depending on who happens to be the reviewer, I would like to have an official view of the packaging experts on this issue.
Mattias
On Mon, 2009-04-20 at 10:01 +0200, Mattias Ellert wrote:
Then the question is how should the following guideline be interpreted:
"If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %doc."
Does this text refer to the big gigantic installer or the extracted source tarfile in this case.
My 0.02€:
If everything in the gigantic tarball is under the same license, then it should be included.
If the subpackages are from different upstreams and they are not under the same license, then if no license file is distributed with the subpackage it is not put into %doc.
Of course, the situation is trickier if the upstream tarball contains many license files, e.g. COPYING.BSD, COPYING.MIT and COPYING.GPLv2; in that case the license file should be included in the (sub)package rpm even though the license file does not exist in the subpackage directory of the upstream tarball.
mån 2009-04-20 klockan 11:15 +0300 skrev Jussi Lehtola:
On Mon, 2009-04-20 at 10:01 +0200, Mattias Ellert wrote:
Then the question is how should the following guideline be interpreted:
"If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %doc."
Does this text refer to the big gigantic installer or the extracted source tarfile in this case.
My 0.02€:
If everything in the gigantic tarball is under the same license, then it should be included.
If the subpackages are from different upstreams and they are not under the same license, then if no license file is distributed with the subpackage it is not put into %doc.
Of course, the situation is trickier if the upstream tarball contains many license files, e.g. COPYING.BSD, COPYING.MIT and COPYING.GPLv2; in that case the license file should be included in the (sub)package rpm even though the license file does not exist in the subpackage directory of the upstream tarball.
The upstream tarball contains one license file that applies to all code developed by upstream. The upstream tarball also contain copies of the source tree for some of its dependencies (openssl, libtool, libxml2, ...) which also have separate license files. The presence of these additional license files was given as an additional argument not to include the license in the packages by the reviewer exercising this position. These additional licences are rather irrelevant since those parts of the upstream tarball will never be packaged for Fedora, since the Fedora packages for these dependences are used.
Mattias
On Mon, 2009-04-20 at 10:30 +0200, Mattias Ellert wrote:
mån 2009-04-20 klockan 11:15 +0300 skrev Jussi Lehtola:
On Mon, 2009-04-20 at 10:01 +0200, Mattias Ellert wrote:
The upstream tarball contains one license file that applies to all code developed by upstream. The upstream tarball also contain copies of the source tree for some of its dependencies (openssl, libtool, libxml2, ...) which also have separate license files. The presence of these additional license files was given as an additional argument not to include the license in the packages by the reviewer exercising this position. These additional licences are rather irrelevant since those parts of the upstream tarball will never be packaged for Fedora, since the Fedora packages for these dependences are used.
In that case I think the situation is clear: the license file must be present in %doc, since the subpackage rpms are all created from the same tarball.
If everything is built in one big specfile, the license goes in the %doc of every subpackage that can be installed separately. (E.g. devel doesn't need to have the license, if devel requires the main package which already contains the license.)
What do the others think?
Mattias Ellert wrote:
Hi!
Then the question is how should the following guideline be interpreted:
"If (and only if) the source package includes the text of the license(s) in its own file, then that file, containing the text of the license(s) for the package must be included in %doc."
Does this text refer to the big gigantic installer or the extracted source tarfile in this case.
Neither.
This paragraph aims at the problem of tarballs containing "inlined licenses" vs. packages containing "detached licenses".
With * "inlined licenses": source files contain a "license section" within their code. * "detached licenses": A tarball contains one or more files, describing the source's contents.
Somewhat oversimplified, this guideline essentially means: "If the tarball has a license file, then you must include it - if it doesn't, you must not create one"
Ralf
mån 2009-04-20 klockan 11:22 +0200 skrev Ralf Corsepius:
This paragraph aims at the problem of tarballs containing "inlined licenses" vs. packages containing "detached licenses".
With
- "inlined licenses": source files contain a "license section" within
their code.
- "detached licenses": A tarball contains one or more files, describing
the source's contents.
Somewhat oversimplified, this guideline essentially means: "If the tarball has a license file, then you must include it - if it doesn't, you must not create one"
Ralf
The question at hand is not whether the tarball contains inlined or detached licenses. The question is which tarball the guideline refers to. If it is the large upstream installer it does include a detached license file. If it is the extracted tarball it does not.
Mattias
Mattias Ellert wrote:
mån 2009-04-20 klockan 11:22 +0200 skrev Ralf Corsepius:
This paragraph aims at the problem of tarballs containing "inlined licenses" vs. packages containing "detached licenses".
With
- "inlined licenses": source files contain a "license section" within
their code.
- "detached licenses": A tarball contains one or more files, describing
the source's contents.
Somewhat oversimplified, this guideline essentially means: "If the tarball has a license file, then you must include it - if it doesn't, you must not create one"
Ralf
The question at hand is not whether the tarball contains inlined or detached licenses. The question is which tarball the guideline refers to. If it is the large upstream installer it does include a detached license file. If it is the extracted tarball it does not.
The upstream distribution is the big installer containing the license file. The creation of the extracted tarballs is part of the packaging process.
Suppose the packager of a "standard" package with a detached license file in the upstream tarball made an extracted tarball containing everything but the upstream license file and then used that as the basis for a package. Would that make sense? No. Neither does the omission of the upstream license in the case of the humongous installer. The extracted tarball should include the upstream license and the subdirectory of interest from the installer, and then the resulting package should include the license file.
Paul.
On 04/20/2009 06:28 AM, Mattias Ellert wrote:
The question at hand is not whether the tarball contains inlined or detached licenses. The question is which tarball the guideline refers to. If it is the large upstream installer it does include a detached license file. If it is the extracted tarball it does not.
*puts on his Fedora Legal hat*
Does the tarball that you're using as Source0 (or whatever Source lines are in the spec) contain a separate (and relevant) license text file? If so, you MUST have it as %doc. If not, you (the packager) can choose to add it to the package as %doc if you want, but you are NOT required to do so.
Hope that helps,
~spot
Tom "spot" Callaway wrote:
On 04/20/2009 06:28 AM, Mattias Ellert wrote:
The question at hand is not whether the tarball contains inlined or detached licenses. The question is which tarball the guideline refers to. If it is the large upstream installer it does include a detached license file. If it is the extracted tarball it does not.
*puts on his Fedora Legal hat*
Does the tarball that you're using as Source0 (or whatever Source lines are in the spec) contain a separate (and relevant) license text file? If so, you MUST have it as %doc. If not, you (the packager) can choose to add it to the package as %doc if you want, but you are NOT required to do so.
But the packager is generating the tarball listed in Source0....so wouldn't this then be that the packager must include the license from the original upstream tarball used to generate the Source0 tarball. And preferably, they should include that license in the Source0 tarball?
-Toshio
Toshio Kuratomi wrote:
Tom "spot" Callaway wrote:
On 04/20/2009 06:28 AM, Mattias Ellert wrote:
The question at hand is not whether the tarball contains inlined or detached licenses. The question is which tarball the guideline refers to. If it is the large upstream installer it does include a detached license file. If it is the extracted tarball it does not.
*puts on his Fedora Legal hat*
Does the tarball that you're using as Source0 (or whatever Source lines are in the spec) contain a separate (and relevant) license text file? If so, you MUST have it as %doc. If not, you (the packager) can choose to add it to the package as %doc if you want, but you are NOT required to do so.
But the packager is generating the tarball listed in Source0....so wouldn't this then be that the packager must include the license from the original upstream tarball used to generate the Source0 tarball.
If the original tarball contains one, yes.
And preferably, they should include that license in the Source0 tarball?
IMO, not only "preferable", but they likely must include it, because their works is a derivative work of other parties.
On 04/20/2009 11:50 AM, Toshio Kuratomi wrote:
But the packager is generating the tarball listed in Source0....so wouldn't this then be that the packager must include the license from the original upstream tarball used to generate the Source0 tarball. And preferably, they should include that license in the Source0 tarball?
Hmm. Okay, yeah, if the packager is making a custom tarball from the larger one, they should include the license text in the custom tarball.
~spot
On Mon, 2009-04-20 at 11:22 +0200, Ralf Corsepius wrote:
[clip]
Somewhat oversimplified, this guideline essentially means: "If the tarball has a license file, then you must include it - if it doesn't, you must not create one"
I've never encountered hitherto an interpretation in which a license file has to be created if upstream hasn't supplied any. AFAIK that is the policy in Debian, not in Fedora.
Maybe it should be made clearer in the guidelines what must be done also in this case.
Jussi Lehtola wrote:
On Mon, 2009-04-20 at 11:22 +0200, Ralf Corsepius wrote:
[clip]
Somewhat oversimplified, this guideline essentially means: "If the tarball has a license file, then you must include it - if it doesn't, you must not create one"
I've never encountered hitherto an interpretation in which a license file has to be created if upstream hasn't supplied any.
Well, packagers wanting to add license files had been a common case in the early Fedora days.
It had been (and still occasionally is) a typical packaging newcomer mistake, originating from people who interpret "include license files" as "must add one, if not present".
You can still find packages which do so in Fedora.
AFAIK that is the policy in Debian, not in Fedora.
No idea what Debian does.
Maybe it should be made clearer in the guidelines what must be done also in this case.
This section is almost as old as Fedora. It has hardly been a problem before.
Ralf
Mattias Ellert wrote:
Here is a description of the problem at hand:
When upstream distributes sources in a gigantic installer containing the sources for 300+ packages it doesn't make sense to include this full tarfile for each SRPM, since less than 1% of it is used to compile each package. Instead the relevant subdirectory is extracted from this beast (properly documented in the specfile in accordance to the packaging guidelines).
What's the bugzilla URL? I think people have answered the licence question pretty well but I'm curious to see how the split up of the 300+ packages is being accomplished. That seems like it would be a more contentious area.
-Toshio
20 apr 2009 kl. 14.58 skrev Toshio Kuratomi:
Mattias Ellert wrote:
Here is a description of the problem at hand:
When upstream distributes sources in a gigantic installer containing the sources for 300+ packages it doesn't make sense to include this full tarfile for each SRPM, since less than 1% of it is used to compile each package. Instead the relevant subdirectory is extracted from this beast (properly documented in the specfile in accordance to the packaging guidelines).
What's the bugzilla URL? I think people have answered the licence question pretty well but I'm curious to see how the split up of the 300+ packages is being accomplished. That seems like it would be a more contentious area.
-Toshio
Here is the reviewer saying "Will not approve package unless license file is removed": https://bugzilla.redhat.com/show_bug.cgi?id=467235
Here is the reviewer saying "Will not approve package unless license file is added": https://bugzilla.redhat.com/show_bug.cgi?id=478917
The specfiles for the two packages are almost identical.
The split of the huge upstream installer was not an issue with either reviewer, except one of them requested it should be better documented - after implementing that he was happy.
Mattias
Mattias Ellert wrote:
20 apr 2009 kl. 14.58 skrev Toshio Kuratomi:
Mattias Ellert wrote:
Here is a description of the problem at hand:
When upstream distributes sources in a gigantic installer containing the sources for 300+ packages it doesn't make sense to include this full tarfile for each SRPM, since less than 1% of it is used to compile each package. Instead the relevant subdirectory is extracted from this beast (properly documented in the specfile in accordance to the packaging guidelines).
What's the bugzilla URL? I think people have answered the licence question pretty well but I'm curious to see how the split up of the 300+ packages is being accomplished. That seems like it would be a more contentious area.
-Toshio
Here is the reviewer saying "Will not approve package unless license file is removed": https://bugzilla.redhat.com/show_bug.cgi?id=467235
Here is the reviewer saying "Will not approve package unless license file is added": https://bugzilla.redhat.com/show_bug.cgi?id=478917
The specfiles for the two packages are almost identical.
The split of the huge upstream installer was not an issue with either reviewer, except one of them requested it should be better documented - after implementing that he was happy.
Ugh, upstream does put you in a bit of a bind, don't they? :-(
I think that you're pretty clearly in violation of this guideline: https://fedoraproject.org/wiki/Packaging/SourceURL#Referencing_Source
Have you asked upstream whether they'd consider releasing individual tarballs for all components? Since they release individual update tarballs, this might be an oversight rather than something that they don't want to do. This would be the ideal outcome for us.
If they won't do this, there's a couple things you could do:
1) Package the monolithic tarball and remove the extraneous library sources at buildtime. You'd put each component into a subpackage of the main package (so far the same as any other package). When upstream releases one of their update tarballs with just an individual component, you'd include that as a separate Source.
2) Make separate packages each using the same tarball (unless an update tarball was released with just that component). This doesn't take up much more room on the server (since the huge tarball will have the same name and md5sum) but it does make building locally a bit more painful.
3) Ask the packaging Committee for an exception to the Source Rule so you can modify the source tarball as you're doing now.
As for the specific licensing case. When the packager is doing the splitting of the tarball I think that Ralf's note that you may be legally obligated to copy the license file into the new tarball carries a lot of weight. If you have to keep splitting the tarball at the packager level I'd amend the instructions for generating the Source0 tarball to copy the license file from the upstream source in addition to the module directory.
For the case of packages and subpackages that Orcan speaks of, when a Requires between a subpackage and another package built from the same RPM exists, we don't need to include the license file in both binary packages. When there is not a Requires, we still have a relationship among packages via the SRPM and we could use a -common subpackage or a main package (which is the most common usage) that has the licenses. The FPC touched briefly on this when discussing Duplicate Files:: https://fedoraproject.org/wiki/Packaging:Minutes/20090217#t12:18
As yet, no one has brought to our attention a specific, real life package to work on as a corner case yet.
-Toshio
mån 2009-04-20 klockan 14:57 -0700 skrev Toshio Kuratomi:
Mattias Ellert wrote:
20 apr 2009 kl. 14.58 skrev Toshio Kuratomi:
What's the bugzilla URL? I think people have answered the licence question pretty well but I'm curious to see how the split up of the 300+ packages is being accomplished. That seems like it would be a more contentious area.
-Toshio
Here is the reviewer saying "Will not approve package unless license file is removed": https://bugzilla.redhat.com/show_bug.cgi?id=467235
Here is the reviewer saying "Will not approve package unless license file is added": https://bugzilla.redhat.com/show_bug.cgi?id=478917
The specfiles for the two packages are almost identical.
The split of the huge upstream installer was not an issue with either reviewer, except one of them requested it should be better documented - after implementing that he was happy.
Ugh, upstream does put you in a bit of a bind, don't they? :-(
Thank you for pointing out the obvious.
I think that you're pretty clearly in violation of this guideline: https://fedoraproject.org/wiki/Packaging/SourceURL#Referencing_Source
I don't see the violation here. The page clearly states what you have to do if upstream does not provide the source tarball for your package. The recipe is to state the commands needed to reproduce the tarball from what is provided by upstream, which is exactly what is done in this case. Quoting the guideline verbatim:
"There are several cases where upstream is not providing the source to you in an upstream tarball. In these cases you must document how to generate the tarball used in the rpm either through a spec file comment or a script included as a separate SourceX."
After which follows a few examples. There is no substantial difference between the example stated on the page:
# The source for this package was pulled from upstream's vcs. Use the # following commands to generate the tarball: # svn export -r 250 http://www.example.com/svn/foo/trunk foo-20070221 # tar -czvf foo-20070221.tar.gz foo-20070221 Source0: foo-20070221.tar.gz
and the description in for extracting the source in this case:
# Source is extracted from the globus toolkit installer: # wget -N http://www-unix.globus.org/ftppub/gt4/4.2.1/installers/src/gt4.2.1-all-sourc... # tar -jxf gt4.2.1-all-source-installer.tar.bz2 # mv gt4.2.1-all-source-installer/source-trees/core/source globus_core-5.15 # tar -zcf globus_core-5.15.tar.gz globus_core-5.15 Source: %{_name}-%{version}.tar.gz
So the way the source is extracted and how it is documented is within the existing guidelines. Three different reviewers have already approved packages where the source has been created and documented in this way.
Have you asked upstream whether they'd consider releasing individual tarballs for all components? Since they release individual update tarballs, this might be an oversight rather than something that they don't want to do. This would be the ideal outcome for us.
If they won't do this, there's a couple things you could do:
- Package the monolithic tarball and remove the extraneous library
sources at buildtime. You'd put each component into a subpackage of the main package (so far the same as any other package). When upstream releases one of their update tarballs with just an individual component, you'd include that as a separate Source.
This won't work. Each package expects that the previous packages it depends on are already installed in the final location when it is built. The upstream installer script does: configure A, make A, make install A, configure B, make B, make install B, and so on. Besides, each package has a separate version number, and I haven't seen an SRPM building packages with different version numbers yet. And do you really want an SRPM building something like 600-800 binary packages? Apart from these technical objections, such a package would be almost unmaintainable. You are not the first one to suggest this, but it is really the wrong thing to do.
- Make separate packages each using the same tarball (unless an update
tarball was released with just that component). This doesn't take up much more room on the server (since the huge tarball will have the same name and md5sum) but it does make building locally a bit more painful.
The lookaside cache http://cvs.fedoraproject.org/repo/pkgs/ is unfortunately split per package, so this will not save any space on the Fedora server. The really huge waste would of course be that this huge unused piece of code would be inside each SRPM and replicated to all Fedora mirrors wasting both bandwidth and disk space. So this is also not the right thing to do.
- Ask the packaging Committee for an exception to the Source Rule so
you can modify the source tarball as you're doing now.
I fail to see where the exception is (see above).
As for the specific licensing case. When the packager is doing the splitting of the tarball I think that Ralf's note that you may be legally obligated to copy the license file into the new tarball carries a lot of weight. If you have to keep splitting the tarball at the packager level I'd amend the instructions for generating the Source0 tarball to copy the license file from the upstream source in addition to the module directory.
To summarize this thread, I conclude that the majority thinks that the license file must be part of the package. I also conclude that rather than being made a separate source file (which is how it was originally implemented) the license file should be copied into and made part of the extracted source tarball.
Is this a correct assessment of the view of the members of this list?
-Toshio
Mattias
Mattias Ellert wrote:
mån 2009-04-20 klockan 14:57 -0700 skrev Toshio Kuratomi:
Mattias Ellert wrote:
20 apr 2009 kl. 14.58 skrev Toshio Kuratomi:
What's the bugzilla URL? I think people have answered the licence question pretty well but I'm curious to see how the split up of the 300+ packages is being accomplished. That seems like it would be a more contentious area.
-Toshio
Here is the reviewer saying "Will not approve package unless license file is removed": https://bugzilla.redhat.com/show_bug.cgi?id=467235
Here is the reviewer saying "Will not approve package unless license file is added": https://bugzilla.redhat.com/show_bug.cgi?id=478917
The specfiles for the two packages are almost identical.
The split of the huge upstream installer was not an issue with either reviewer, except one of them requested it should be better documented - after implementing that he was happy.
Ugh, upstream does put you in a bit of a bind, don't they? :-(
Thank you for pointing out the obvious.
I think that you're pretty clearly in violation of this guideline: https://fedoraproject.org/wiki/Packaging/SourceURL#Referencing_Source
I don't see the violation here. The page clearly states what you have to do if upstream does not provide the source tarball for your package. The recipe is to state the commands needed to reproduce the tarball from what is provided by upstream, which is exactly what is done in this case. Quoting the guideline verbatim:
"There are several cases where upstream is not providing the source to you in an upstream tarball. In these cases you must document how to generate the tarball used in the rpm either through a spec file comment or a script included as a separate SourceX."
The difference is that in this case, upstream is providing you with a tarball.
[...]
Have you asked upstream whether they'd consider releasing individual tarballs for all components? Since they release individual update tarballs, this might be an oversight rather than something that they don't want to do. This would be the ideal outcome for us.
I love how everyone I talk to about guidelines violations ignores my upstream comment :-/ Upstream is the first thing to try in any situation. Has this been tried here?
[...]
- Ask the packaging Committee for an exception to the Source Rule so
you can modify the source tarball as you're doing now.
I fail to see where the exception is (see above).
Failing upstream cooperation, this seems like the best option.
To summarize this thread, I conclude that the majority thinks that the license file must be part of the package. I also conclude that rather than being made a separate source file (which is how it was originally implemented) the license file should be copied into and made part of the extracted source tarball.
Is this a correct assessment of the view of the members of this list?
Yes.
-Toshio
packaging@lists.fedoraproject.org
-
Jussi Lehtola -
Mattias Ellert -
Paul Howarth -
Ralf Corsepius -
Tom Callaway -
Toshio Kuratomi