Hello all, In today's FESCo meeting we discussed the fact that there are many RPMs currently in Fedora (a reported 244 in Rawhide currently) that are defining a `Provides: bundled(<lib>) = <version>` but excluding the version completely[0][1]. This removes that ability to properly perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we handle this? Is this something that should just simply be fixed by the packages currently violating the Guidelines, should the Guidelines be altered in a way that makes this easier to deal with for Packagers but also provides what is needed for auditing and vulnerability tracking, or is there simply clarification needed by what is required in the <version> field?
I look forward to the discussion.
Thank you, -AdamM
[0] - https://pagure.io/fesco/issue/1734 [1] - https://pagure.io/packaging-committee/issue/696
On Fri, Jul 7, 2017 at 10:55 AM, Adam Miller maxamillion@fedoraproject.org wrote:
Hello all, In today's FESCo meeting we discussed the fact that there are many RPMs currently in Fedora (a reported 244 in Rawhide currently) that are defining a `Provides: bundled(<lib>) = <version>` but excluding the version completely[0][1]. This removes that ability to properly perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we handle this? Is this something that should just simply be fixed by the packages currently violating the Guidelines, should the Guidelines be altered in a way that makes this easier to deal with for Packagers but also provides what is needed for auditing and vulnerability tracking, or is there simply clarification needed by what is required in the <version> field?
How many of those are bundled(jquery)? A couple of years ago, those of us with packages that generate documentation via doxygen were asked to add that to our spec files. (I no longer remember who asked us, sorry.) The reasoning was that someday, somebody would want to do something about fixing doxygen to point to a system version of jquery, and adding those Provides would make the process of finding the affected packages easier. I haven't heard anything more since then, so I don't know if anybody is even working on the issue.
On Fri, Jul 7, 2017 at 10:55 AM, Adam Miller maxamillion@fedoraproject.org wrote:
Hello all,
After which my response was met with:
------------------------------------------------------------------------------ Your message to the packaging mailing-list was rejected for the following reasons:
The message is not from a list member
The original message as received by Mailman is attached. ------------------------------------------------------------------------------
You cross-posted to closed lists. Please don't do that unless you know for certain that the lists have identical membership, which pretty much obviates the need for posting to multiple lists. If you want to send the same message to multiple closed mailing lists, then send copies of it, one at a time, to each list. Thank you,
On Fri, Jul 7, 2017 at 12:10 PM, Jerry James loganjerry@gmail.com wrote:
On Fri, Jul 7, 2017 at 10:55 AM, Adam Miller maxamillion@fedoraproject.org wrote:
Hello all,
After which my response was met with:
Your message to the packaging mailing-list was rejected for the following reasons:
The message is not from a list member
The original message as received by Mailman is attached.
You cross-posted to closed lists. Please don't do that unless you know for certain that the lists have identical membership, which pretty much obviates the need for posting to multiple lists. If you want to send the same message to multiple closed mailing lists, then send copies of it, one at a time, to each list. Thank you,
Apologies, it was requested in the FESCo meeting that I CC the FPC so I did. I wasn't aware of the restrictions on the mailing list. I'll be sure to verify in the future.
-AdamM
-- Jerry James http://www.jamezone.org/ _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On Fri, 2017-07-07 at 11:55 -0500, Adam Miller wrote:
My question to the Fedora Contributor Community is, how should we handle this?
I am guilty of doing this a few times in a couple of my packages. My reasons when I did it were that I didn't know the version, and didn't know an easy way to determine the version. I figured it was better to at least declare that I'm bundling it with uncertainty on the version than to not declare at all.
On Fri, Jul 7, 2017 at 12:35 PM, Randy Barlow bowlofeggs@fedoraproject.org wrote:
I am guilty of doing this a few times in a couple of my packages. My reasons when I did it were that I didn't know the version, and didn't know an easy way to determine the version. I figured it was better to at least declare that I'm bundling it with uncertainty on the version than to not declare at all.
We do this for WebKit, which bundles ANGLE and various other crap Google libraries, because crap Google libraries do not have releases or versions. They do generally have tags corresponding to Chrome releases, but I think we generally use git snapshots. So just thinking up a version number to use for the provides is not easy.
Another problem is selecting the name for the provides. E.g. pick between bundled(ANGLE) or bundled(libangle). With the next evince update, evince will bundle a LZMA decoder. Should it be bundled(LZMA) or bundled(lzma-sdk) or bundled(7zip)? Some of these options seem better than others, but all are reasonable and maintainers are likely to choose different ones....
Michael
Michael Catanzaro wrote:
Another problem is selecting the name for the provides. E.g. pick between bundled(ANGLE) or bundled(libangle). With the next evince update, evince will bundle a LZMA decoder. Should it be bundled(LZMA) or bundled(lzma-sdk) or bundled(7zip)? Some of these options seem better than others, but all are reasonable and maintainers are likely to choose different ones....
Indeed, that's the big flaw with this way of tracking bundled stuff, which I pointed out when it was first proposed. It seems that nobody has a good answer.
Björn Persson
"AM" == Adam Miller maxamillion@fedoraproject.org writes:
[...] AM> RPMs currently in Fedora (a reported 244 in Rawhide currently) that AM> are defining a `Provides: bundled(<lib>) = <version>` but excluding AM> the version completely[0][1]. This removes that ability to properly AM> perform source code auditing and security vulnerability tracking.
I would argue that it doesn't remove the ability, but that it does make it more difficult to do in an automated fashion. Basically you can see that something has a bundled library but then you need to do manual inspection to go further.
AM> My question to the Fedora Contributor Community is, how should we AM> handle this?
Identify and mail lists of the problematic packages to devel (using find-package-maintainers from https://pagure.io/fedora-misc-package-utilities if possible). Figure out if there are any cases which aren't easy to fix for some reason.
If there are any, then see if a change is needed to accommodate.
If I had to hazard a guess, I would say that there are at least some cases where it's not really obvious what version to use. This would make sense in the case of a fork that's undergone significant rewriting. Though I wonder if any bundled(X) tag is even warranted in that case.
Alternatively, say that you don't have to specify a version, but if you don't then you will get every related security bug filed against your package instead of having those filtered by version.
- J<
Jason L Tibbitts III wrote:
Alternatively, say that you don't have to specify a version, but if you don't then you will get every related security bug filed against your package instead of having those filtered by version.
Perhaps with a notice included in each such bug report, along the lines of "Because the version of the bundled library is unspecified, we must assume that it is a vulnerable version.", to make people aware that they can avoid irrelevant bug reports by adding a version number if one exists.
Björn Persson
7.7.2017 20.45 "Jason L Tibbitts III" tibbs@math.uh.edu kirjoitti:
I would argue that it doesn't remove the ability, but that it does make it more difficult to do in an automated fashion. Basically you can see that something has a bundled library but then you need to do manual inspection to go further.
I think the versioning isn't worth much at all.
If the bundled version corresponds to an upstream release to an extent that it can be called that version, and checks like the discussed one could be skipped just by looking at the version label, then it must be practically the same. So why is it bundled in the first place?
On the other hand if there is a "good" reason it is bundled, that reason quite probably is that it is a modified version. So it's different than the upstream one, and thus knowledge whether an upstream release is vulnerable or not cannot be just assumed based on the version label a packager has attached to it. It needs to be checked anyway.
On Saturday, 08 July 2017 at 17:24, Ville Skyttä wrote: [...]
If the bundled version corresponds to an upstream release to an extent that it can be called that version, and checks like the discussed one could be skipped just by looking at the version label, then it must be practically the same. So why is it bundled in the first place?
There can be many reasons: copylibs, no upstream support for building as shared library, no stable API/ABI, etc. Many of them are listed on the wiki page: https://fedoraproject.org/wiki/Bundled_Libraries_Virtual_Provides
Regards, Dominik
----- Original Message -----
On Saturday, 08 July 2017 at 17:24, Ville Skyttä wrote: [...]
If the bundled version corresponds to an upstream release to an extent that it can be called that version, and checks like the discussed one could be skipped just by looking at the version label, then it must be practically the same. So why is it bundled in the first place?
There can be many reasons: copylibs, no upstream support for building as shared library, no stable API/ABI, etc. Many of them are listed on the wiki page: https://fedoraproject.org/wiki/Bundled_Libraries_Virtual_Provides
There's no mention of zlib or LZMA SDK libs in there. Should those be added?
On Monday, 10 July 2017 at 13:56, Bastien Nocera wrote:
----- Original Message -----
[...]
There can be many reasons: copylibs, no upstream support for building as shared library, no stable API/ABI, etc. Many of them are listed on the wiki page: https://fedoraproject.org/wiki/Bundled_Libraries_Virtual_Provides
There's no mention of zlib or LZMA SDK libs in there. Should those be added?
Yes, please do.
Regards, Dominik
On Sat, Jul 8, 2017 at 10:24 AM, Ville Skyttä ville.skytta@iki.fi wrote:
7.7.2017 20.45 "Jason L Tibbitts III" tibbs@math.uh.edu kirjoitti:
I would argue that it doesn't remove the ability, but that it does make it more difficult to do in an automated fashion. Basically you can see that something has a bundled library but then you need to do manual inspection to go further.
I think the versioning isn't worth much at all.
If the bundled version corresponds to an upstream release to an extent that it can be called that version, and checks like the discussed one could be skipped just by looking at the version label, then it must be practically the same. So why is it bundled in the first place?
On the other hand if there is a "good" reason it is bundled, that reason quite probably is that it is a modified version. So it's different than the upstream one, and thus knowledge whether an upstream release is vulnerable or not cannot be just assumed based on the version label a packager has attached to it. It needs to be checked anyway.
The main motivation for bundling as of late is golang[0], it's extremely common out in the community for software to pull in "Vendored" libraries even if they are exact copies of remote upstreams (this is common with tools like godep[1]) because golang is statically compiled by default (you can dynamically link with gcc-go and I *think* recent releases of cgo but I've yet to find a golang project that officially uses or endorses it's use). It's also extremely painful to unbundle these as some more popular software written in golang such as docker[2], kubernetes[3], and openshift[4] have upwards of 100 bundled libs (over 1000 for OpenShift).
-AdamM
[0] - https://golang.org/ [1] - https://github.com/tools/godep [2] - http://pkgs.fedoraproject.org/cgit/rpms/docker.git/tree/docker.spec [3] - http://pkgs.fedoraproject.org/cgit/rpms/kubernetes.git/tree/kubernetes.spec [4] - http://pkgs.fedoraproject.org/cgit/rpms/origin.git/tree/origin.spec
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On Mon, 2017-07-10 at 11:40 -0500, Adam Miller wrote:
The main motivation for bundling as of late is golang[0], it's extremely common out in the community for software to pull in "Vendored" libraries even if they are exact copies of remote upstreams (this is common with tools like godep[1]) because golang is statically compiled by default (you can dynamically link with gcc-go and I *think* recent releases of cgo but I've yet to find a golang project that officially uses or endorses it's use). It's also extremely painful to unbundle these as some more popular software written in golang such as docker[2], kubernetes[3], and openshift[4] have upwards of 100 bundled libs (over 1000 for OpenShift).
On top of these problems I have also observed a trend away from having releases and versions at all. Lots of golang programs just depend on git commit hashes of libraries that don't make releases. I've also observed this problem in some JavaScript libraries.
"RB" == Randy Barlow bowlofeggs@fedoraproject.org writes:
RB> On top of these problems I have also observed a trend away from RB> having releases and versions at all. Lots of golang programs just RB> depend on git commit hashes of libraries that don't make RB> releases. I've also observed this problem in some JavaScript RB> libraries.
That's true, but we have a scheme for versioning packages containing such things, and I see no reason that scheme isn't applicable anywhere we might need to make reference to a version of a particular piece of software.
- J<
RB> _______________________________________________ devel mailing list RB> -- devel@lists.fedoraproject.org To unsubscribe send an email to RB> devel-leave@lists.fedoraproject.org
Adam Miller wrote:
In today's FESCo meeting we discussed the fact that there are manyRPMs currently in Fedora (a reported 244 in Rawhide currently) that are defining a `Provides: bundled(<lib>) = <version>` but excluding the version completely[0][1]. This removes that ability to properly perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we handle this? Is this something that should just simply be fixed by the packages currently violating the Guidelines, should the Guidelines be altered in a way that makes this easier to deal with for Packagers but also provides what is needed for auditing and vulnerability tracking, or is there simply clarification needed by what is required in the <version> field?
A version number may not even exist at all. Not all code that people copy is a library with a version number. Copylibs often don't bother doing releases because everyone just embeds it as a git submodule or checks out some random revision to copy into their own SCM. Hence, it is not realistic to require a version number.
Kevin Kofler
On Sun, Jul 9, 2017 at 5:36 PM, Kevin Kofler kevin.kofler@chello.at wrote:
Adam Miller wrote:
In today's FESCo meeting we discussed the fact that there are manyRPMs currently in Fedora (a reported 244 in Rawhide currently) that are defining a `Provides: bundled(<lib>) = <version>` but excluding the version completely[0][1]. This removes that ability to properly perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we handle this? Is this something that should just simply be fixed by the packages currently violating the Guidelines, should the Guidelines be altered in a way that makes this easier to deal with for Packagers but also provides what is needed for auditing and vulnerability tracking, or is there simply clarification needed by what is required in the <version> field?
A version number may not even exist at all. Not all code that people copy is a library with a version number. Copylibs often don't bother doing releases because everyone just embeds it as a git submodule or checks out some random revision to copy into their own SCM. Hence, it is not realistic to require a version number.
So should we just stop requiring any RPMs be versioned since it's not realistic to require a version number?
-AdamM
Kevin Kofler
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org
On 2017-07-07, Adam Miller maxamillion@fedoraproject.org wrote:
`Provides: bundled(<lib>) = <version>` but excluding the version completely[0][1]. This removes that ability to properly perform source code auditing and security vulnerability tracking.
My question to the Fedora Contributor Community is, how should we handle this?
It's wrong assumption that a version of the bundled code is known. Either the bundled upstream has no versions, or the bundled code is heavily modified (hence the reason for not unbundling and the need for Provides: bundled()), or there the bundled code has no upstream (anymore). I usually try to track the version when the code was bundled from, but sometimes it's impossible to figure out.
Thus I recommend relaxing the guidelines. It's still better to have an unversioned bundled() RPM symbol than nothing.
-- Petr
-
Adam Miller -
Bastien Nocera -
Björn Persson -
Dominik 'Rathann' Mierzejewski -
Jason L Tibbitts III -
Jerry James -
Kevin Kofler -
Michael Catanzaro -
Petr Pisar -
Randy Barlow -
Ville Skyttä