Hi,
I have a package (keyutils) that produces three RPMs: keyutils (programs), keyutils-libs and keyutils-devel. The programs in the keyutils RPM depend on the libraries in the keyutils-libs RPM and pick up implicit dependencies thus:
warthog>rpm -qRp keyutils-1.5.1-1.fc14.x86_64.rpm ... config(keyutils) = 1.5.1-1.fc14 ... libkeyutils.so.1()(64bit) libkeyutils.so.1(KEYUTILS_0.3)(64bit) libkeyutils.so.1(KEYUTILS_1.0)(64bit) libkeyutils.so.1(KEYUTILS_1.3)(64bit) libkeyutils.so.1(KEYUTILS_1.4)(64bit) ...
So does the keyutils rpm need an explicit Requires on the keyutils-libs rpm in the specfile or is the implicit library dependency sufficient?
rpmlint does not complain, but, for RHEL, rpmdiff does.
David
On 08/11/2011 01:19 PM, David Howells wrote:
I have a package (keyutils) that produces three RPMs: keyutils (programs), keyutils-libs and keyutils-devel. The programs in the keyutils RPM depend on the libraries in the keyutils-libs RPM and pick up implicit dependencies thus:
warthog>rpm -qRp keyutils-1.5.1-1.fc14.x86_64.rpm ... config(keyutils) = 1.5.1-1.fc14 ... libkeyutils.so.1()(64bit) libkeyutils.so.1(KEYUTILS_0.3)(64bit) libkeyutils.so.1(KEYUTILS_1.0)(64bit) libkeyutils.so.1(KEYUTILS_1.3)(64bit) libkeyutils.so.1(KEYUTILS_1.4)(64bit) ...
So does the keyutils rpm need an explicit Requires on the keyutils-libs rpm in the specfile or is the implicit library dependency sufficient?
Library requirements should be implicit unless there's a good reason otherwise; see: http://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
rpmlint does not complain, but, for RHEL, rpmdiff does.
What is the rpmdiff "complaint"?
Paul.
Paul Howarth paul@city-fan.org wrote:
rpmlint does not complain, but, for RHEL, rpmdiff does.
What is the rpmdiff "complaint"?
Report from TEST_REQUIRES: [VERIFY] [keyutils] Subpackage keyutils on i686 ppc s390 consumes libraries libkeyutils.so.1 libkeyutils.so.1(KEYUTILS_0.3) libkeyutils.so.1(KEYUTILS_1.0) libkeyutils.so.1(KEYUTILS_1.3) from subpackage keyutils-libs but does not have explicit package version requirement. Please add Requires: keyutils-libs = %{version}-%{release} to keyutils in the specfile to avoid the need to test interoperability between the various combinations of old and new subpackages.
I guess this is more of a RHEL policy thing.
David
On Thu, 11 Aug 2011 13:29:52 +0100, PH (Paul) wrote:
Library requirements should be implicit unless there's a good reason otherwise; see: http://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
Not true, or "not the full story".
Library SONAME requirement for _external_ builds ought to stay implicit/automatic, but _libraries and subpackages_ are a different problem space. A library update may add stuff without changing its SONAME and while staying compatible with existing executables in external packages. But executables shipped with the library package (in subpackages) may strictly need the very latest library.
I've posted another reply in this thread.
Michael Schwendt mschwendt@gmail.com writes:
Library SONAME requirement for _external_ builds ought to stay implicit/automatic, but _libraries and subpackages_ are a different problem space. A library update may add stuff without changing its SONAME and while staying compatible with existing executables in external packages. But executables shipped with the library package (in subpackages) may strictly need the very latest library.
There is no difference between "external" packages and subpackages: they both will need the new symbol some day.
Andreas.
On 08/11/2011 02:03 PM, Andreas Schwab wrote:
Michael Schwendtmschwendt@gmail.com writes:
Library SONAME requirement for _external_ builds ought to stay implicit/automatic, but _libraries and subpackages_ are a different problem space. A library update may add stuff without changing its SONAME and while staying compatible with existing executables in external packages. But executables shipped with the library package (in subpackages) may strictly need the very latest library.
There is no difference between "external" packages and subpackages: they both will need the new symbol some day.
Indeed, which is why we have version-specific library dependencies on libssh2 in curl: https://bugzilla.redhat.com/show_bug.cgi?id=642796
Paul.
On Thu, 11 Aug 2011 14:10:44 +0100, PH (Paul) wrote:
Library SONAME requirement for _external_ builds ought to stay implicit/automatic, but _libraries and subpackages_ are a different problem space. A library update may add stuff without changing its SONAME and while staying compatible with existing executables in external packages. But executables shipped with the library package (in subpackages) may strictly need the very latest library.
There is no difference between "external" packages and subpackages: they both will need the new symbol some day.
Indeed, which is why we have version-specific library dependencies on libssh2 in curl: https://bugzilla.redhat.com/show_bug.cgi?id=642796
This is just http://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires where a specific library version is needed actually and differs from what has been released before. It is neither the norm nor mandatory.
On Thu, 11 Aug 2011 15:03:47 +0200, AS (Andreas) wrote:
Library SONAME requirement for _external_ builds ought to stay implicit/automatic, but _libraries and subpackages_ are a different problem space. A library update may add stuff without changing its SONAME and while staying compatible with existing executables in external packages. But executables shipped with the library package (in subpackages) may strictly need the very latest library.
There is no difference between "external" packages and subpackages: they both will need the new symbol some day.
The difference is that the subpackages may need the new symbols immediately when installing the packages, whereas future builds of external packages would link with the latest library version that has been released before and is the one available in the build system's buildroot.
Mind you, no external packages can be built against the library update before it enters the buildroot (as a stable update).
Michael Schwendt mschwendt@gmail.com writes:
The difference is that the subpackages may need the new symbols immediately when installing the packages, whereas future builds of external packages would link with the latest library version that has been released before and is the one available in the build system's buildroot.
I don't understand. How is the contents of the buildroot relevant to "yum install foo"?
Andreas.
On Thu, 11 Aug 2011 15:24:30 +0200, AS (Andreas) wrote:
The difference is that the subpackages may need the new symbols immediately when installing the packages, whereas future builds of external packages would link with the latest library version that has been released before and is the one available in the build system's buildroot.
I don't understand. How is the contents of the buildroot relevant to "yum install foo"?
I think you do understand, but probably you just want to reinvent the wheel. Let's see:
No external package can build with new features of the new "foo-libs" package prior to making that package available in the buildroot.
If "foo" is the base package of "foo-libs", and this -libs package has added symbols needed by programs in package "foo", an older release of "foo-libs" would not suffice. On the contrary, all programs linked with an older foo library will continue to work.
For the corner-case that an update of a program in an arbitrary package strictly needs a specific minimum version of foo-libs, the packager may add an explicit dependency according to: http://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
You don't want to enforce explicit dependencies on library pkg NEVR in _all_ packages that use the library, do you?
Michael Schwendt mschwendt@gmail.com writes:
No external package can build with new features of the new "foo-libs" package prior to making that package available in the buildroot.
How is the contents of the buildroot relevant to "yum install bar"?
If "foo" is the base package of "foo-libs", and this -libs package has added symbols needed by programs in package "foo", an older release of "foo-libs" would not suffice. On the contrary, all programs linked with an older foo library will continue to work.
How is that different from the above?
Andreas.
On Thu, 11 Aug 2011 16:13:48 +0200, AS (Andreas) wrote:
No external package can build with new features of the new "foo-libs" package prior to making that package available in the buildroot.
How is the contents of the buildroot relevant to "yum install bar"?
"bar" has been built with an older libfoo. The newer libfoo isn't strictly needed, because any available package providing the needed ABI will suffice.
"foo", on the other hand, would be first to actually use the very latest API extensions. Not always, but often (and more often that many ordinary app bug-fix updates that have build dependencies on much older lib versions). The explicit dep on the lib pkg is _not_ mandatory.
If "foo" is the base package of "foo-libs", and this -libs package has added symbols needed by programs in package "foo", an older release of "foo-libs" would not suffice. On the contrary, all programs linked with an older foo library will continue to work.
How is that different from the above?
Well, feel free to toggle on/off the explicit dep whenever you think that's appropriate. Have fun with the typical breakage when subpackages get out of sync with base packages (such as in the case of a patched -devel package not depending on the patched library base package -- which has resulted in a lot of head-scratching when rebuilds against a new -devel package didn't fix an issue because the fixed library wasn't installed yet). Only a full "yum update" before and after any "yum install" action will avoid many issues with dependencies.
Michael Schwendt mschwendt@gmail.com writes:
On Thu, 11 Aug 2011 16:13:48 +0200, AS (Andreas) wrote:
No external package can build with new features of the new "foo-libs" package prior to making that package available in the buildroot.
How is the contents of the buildroot relevant to "yum install bar"?
"bar" has been built with an older libfoo.
No.
Andreas.
On Thu, 11 Aug 2011 17:23:06 +0200, AS (Andreas) wrote:
On Thu, 11 Aug 2011 16:13:48 +0200, AS (Andreas) wrote:
No external package can build with new features of the new "foo-libs" package prior to making that package available in the buildroot.
How is the contents of the buildroot relevant to "yum install bar"?
"bar" has been built with an older libfoo.
No.
http://www.youtube.com/watch?v=kQFKtI6gn9Y or what?
Any previously released "bar" has been built with an older libfoo. Do you agree here?
Any update (or upgrade) of "bar" can only be built against a newer libfoo if that enters the buildroot. Do you agree here?
For any update of "bar", you need to reconsider https://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
So, where are we now?
Michael Schwendt mschwendt@gmail.com writes:
So, where are we now?
yum install bar doesn't update foo-libs automagically.
Andreas.
On Fri, 12 Aug 2011 09:32:26 +0200, AS (Andreas) wrote:
So, where are we now?
yum install bar doesn't update foo-libs automagically.
Which is why you may benefit from an explicit dependency *if* you publish such an updated "bar" that needs a specific minimum version of foo-libs: http://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
In this thread Paul has given a real-world example: https://bugzilla.redhat.com/show_bug.cgi?id=642796
$ rpm -qR libcurl|grep ssh libssh2(x86-64) >= 1.2.7 libssh2.so.1()(64bit)
Such explicit deps are neither the norm nor mandatory. In Fedora-land, a full "yum update" (or the equivalent thereof with a different tool) is needed frequently anyway because of the updates-flood and because of "yum install" not considering newest packages only. It isn't any secret that Fedora users encounter lots of dep problems when installing without updating frequently.
Back to subpackages. I could repeat my 1st reply in this thread, but let's see. We try to keep builds from the same src.rpm (and not limited to that) in sync for various reasons. Not just because a program/extension included in a subpkg may strictly require the very latest base pkg. Or vice versa (the guidelines even mention the pitfall of creating a circular dep): https://fedoraproject.org/wiki/Packaging/Guidelines#RequiringBasePackage
A foo.src.rpm that builds libs *and* programs doesn't need to be split into several packages. It could also be split into foo + foo-utils or libfoo + foo, and the naming scheme doesn't change which is the base pkg and which is the subpkg. So, if the build creates multiple subpackages, you *could* treat the libs pkg like a separate/external build. However, library subpkg builds are special because they build from the same src.rpm, with no risk that the library package will change its name and break the explicit subpkg dep created within the same package set, with no versioned BuildRequires on an external library package, with no configure script checking for a minimum library version -> any programs built within the same src.rpm may immediately use this very latest lib API.
So, what is safer?
* To treat a foo-libs subpackage like a base package and have foo (or e.g. foo-utils) depend on it with full NEVR?
* Or to treat it like an ordinary/individual library package with just the automatic SONAME dep and reconsider an explicit dep only when building against new library updates? http://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
Michael Schwendt mschwendt@gmail.com writes:
So, what is safer?
Neither fixes the missing symbol.
Andreas.
Michael Schwendt mschwendt@gmail.com writes:
There is no missing symbol.
Of course there is. See http://permalink.gmane.org/gmane.linux.redhat.fedora.devel/152427.
Andreas.
On Fri, 12 Aug 2011 13:00:57 +0200, AS (Andreas) wrote:
There is no missing symbol.
Of course there is. See http://permalink.gmane.org/gmane.linux.redhat.fedora.devel/152427.
No, there isn't, because of a subpackage dependency with full NEVR, even if that may not be needed as a default. Elsewhere in this thread, Rex Dieter has also recommended that one with this rationale:
| Since it's a subpkg from the same srpm, I'd opt to be on the safe side, | and add an explicit versioned Requires
So, I think I understand your mission now. You're either just practising some troll skills or pedantry. :-/
The separate "bar" package still is entirely irrelevant up to the point when it is updated/upgraded and actually needs new symbols found only in a latest library update. Then we're back at https://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires and you're free to act accordingly.
If you prefer Explicit Library NEVR Requires in _all_ packages, take that up with the packaging committee.
Michael Schwendt mschwendt@gmail.com writes:
The separate "bar" package still is entirely irrelevant
It kills your entire argument.
Andreas.
On Fri, 12 Aug 2011 14:30:46 +0200, Andreas Schwab schwab@redhat.com wrote:
The separate "bar" package still is entirely irrelevant
It kills your entire argument.
Andreas.
It still doesn't. libfoo update would need to come first. First come, first served. And what packaging techniques to apply in the libfoo update has been the topic of this thread:
| I have a package (keyutils) that produces three RPMs: keyutils | (programs), keyutils-libs and keyutils-devel. | | ... | | So does the keyutils rpm need an explicit Requires on the keyutils-libs | rpm in the specfile or is the implicit library dependency sufficient?
Whether any other update built against libfoo would also benefit from an explicit dep is covered by the guidelines: https://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
http://permalink.gmane.org/gmane.linux.redhat.fedora.devel/152426
Michael Schwendt mschwendt@gmail.com writes:
libfoo update would need to come first.
How?
Andreas.
On Fri, 12 Aug 2011 15:54:45 +0200, Andreas Schwab schwab@redhat.com wrote:
libfoo update would need to come first.
How?
Andreas.
If you're serious about discussing this further, show that. I'm not going to reply to this thread anymore before tomorrow.
With your single-word reply you [once again] demonstrate that we're likely talking past eachother. Perhaps it's due to language barriers or attitude. Dunno. I don't want to read between the lines to guess whether you understand the sentence you've quoted above. I think you've misunderstood it. You sound as if you now refer to the process of installing package updates rather than preparing and releasing packages.
Your whole argument is just void. There is not a single difference between "external" packages and (sub-)packages.
Andreas.
On Fri, Aug 12, 2011 at 4:15 PM, Andreas Schwab schwab@redhat.com wrote:
Your whole argument is just void. There is not a single difference between "external" packages and (sub-)packages.
There is one difference: with subpackages it is trivial to write a Requires: that 1) is always strict enough and 2) is always satisfied. For external packages achieving both requires non-trivial human effort. Mirek
Miloslav Trmač mitr@volny.cz writes:
There is one difference: with subpackages it is trivial to write a Requires: that 1) is always strict enough and 2) is always satisfied. For external packages achieving both requires non-trivial human effort.
Yet you insist that your "external" package is always working.
Andreas.
On Fri, Aug 12, 2011 at 04:18:34PM +0200, Miloslav Trmač wrote:
On Fri, Aug 12, 2011 at 4:15 PM, Andreas Schwab schwab@redhat.com wrote:
Your whole argument is just void. There is not a single difference between "external" packages and (sub-)packages.
There is one difference: with subpackages it is trivial to write a Requires: that 1) is always strict enough and 2) is always satisfied. For external packages achieving both requires non-trivial human effort.
Not at all. The simplest thing to do is to ensure that any binary depends on at least the version of any libraries it was linked against, which is the way Debian handle this in the absence of maintainer overrides.
On Fri, Aug 12, 2011 at 04:15:49PM +0200, Andreas Schwab wrote:
Your whole argument is just void. There is not a single difference between "external" packages and (sub-)packages.
Here's a couple differences:
(libfoo-1.0 building subpackage foo-utils) is licensed LGPLv2 (libfoo-1.0.1 building subpackage foo-utils) is licensed LGPLv3
Our guidelines let you package the license information in the base package when the subpackage depends on it. In this case, the subpackage won't pull in the updated license information based on soname alone.
libfoo-1.0 has private data:
foopublic.h struct Data { int public; void *private; }
fooprivate.h
struct PrivateData { [...] }
Rightly or wrongly, upstream libfoo-1.0 has some additional utilities that access the PrivateData. Because the utilities are built from the libfoo source, they can include the fooprivate.h header file and do this. When libfoo goes to 1.0.1, upstream changes the definition of PrivateData and updates the utilities to work with the new datastructure. Since the public ABI stayed the same, the SONAME doesn't change and external programs compiled against libfoo-1.0 continue to work but the utilities built as a subpackage would be broken without stricter versioning.
-Toshio
On Fri, Aug 12, 2011 at 08:27:13AM -0700, Toshio Kuratomi wrote:
Rightly or wrongly, upstream libfoo-1.0 has some additional utilities that access the PrivateData. Because the utilities are built from the libfoo source, they can include the fooprivate.h header file and do this. When libfoo goes to 1.0.1, upstream changes the definition of PrivateData and updates the utilities to work with the new datastructure. Since the public ABI stayed the same, the SONAME doesn't change and external programs compiled against libfoo-1.0 continue to work but the utilities built as a subpackage would be broken without stricter versioning.
Upstream can change the ABI as much as they want without bumping the SONAME providing that the old interfaces are also present. It's entirely possible to end up with a situation where external binaries built against 1.0.1 won't run on 1.0.0 - the problem isn't limited to subpackages.
On Fri, 2011-08-12 at 16:40 +0100, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 08:27:13AM -0700, Toshio Kuratomi wrote:
Rightly or wrongly, upstream libfoo-1.0 has some additional utilities that access the PrivateData. Because the utilities are built from the libfoo source, they can include the fooprivate.h header file and do this. When libfoo goes to 1.0.1, upstream changes the definition of PrivateData and updates the utilities to work with the new datastructure. Since the public ABI stayed the same, the SONAME doesn't change and external programs compiled against libfoo-1.0 continue to work but the utilities built as a subpackage would be broken without stricter versioning.
Upstream can change the ABI as much as they want without bumping the SONAME providing that the old interfaces are also present. It's entirely possible to end up with a situation where external binaries built against 1.0.1 won't run on 1.0.0 - the problem isn't limited to subpackages.
If rpmbuild does not add an implicit requires with libraryX >= <version we built against> then it is certainly broken.
ABI *additions* w/o change of the SONAME happens every day in every library.
If you do not want to set a require on the fedora package version, then you could check what is the highest symbol version for all symbols in the lib and require that. Unfortunately not all libraries do symbol versioning so that would not work in many cases.
Simo.
On Fri, 12 Aug 2011 12:08:50 -0400, SS (Simo) wrote:
If rpmbuild does not add an implicit requires with libraryX >= <version we built against> then it is certainly broken.
One could also argue that an activity like "yum install ..." ought to search for and apply the latest available updates of needed packages. Such behaviour has been rejected some years ago, if memory serves correctly.
[Obviously. Because if something provides SONAME and is installed already, it can be assumed that it isn't necessary to look for something newer that also provides SONAME. A matter of definition. In the buildsys we build against latest packages, so it would be natural to let package installers drag in updates always, too -- instead of adding extra deps everywhere, which aren't future-proof.]
On Sat, 2011-08-13 at 09:19 +0200, Michael Schwendt wrote:
On Fri, 12 Aug 2011 12:08:50 -0400, SS (Simo) wrote:
If rpmbuild does not add an implicit requires with libraryX >= <version we built against> then it is certainly broken.
One could also argue that an activity like "yum install ..." ought to search for and apply the latest available updates of needed packages. Such behaviour has been rejected some years ago, if memory serves correctly.
It was rejected with good reason, I think. I do not think we should be adding functionality into yum which:
1. violates the principle of least surprise for the admin 2. covers up for not-specific-enough requirements in the packaging.
It just feels like it would be fixing a "problem" at the wrong layer.
-sv
On Mon, 15 Aug 2011 16:20:47 -0400, SV (seth) wrote:
On Sat, 2011-08-13 at 09:19 +0200, Michael Schwendt wrote:
On Fri, 12 Aug 2011 12:08:50 -0400, SS (Simo) wrote:
If rpmbuild does not add an implicit requires with libraryX >= <version we built against> then it is certainly broken.
One could also argue that an activity like "yum install ..." ought to search for and apply the latest available updates of needed packages. Such behaviour has been rejected some years ago, if memory serves correctly.
It was rejected with good reason, I think. I do not think we should be adding functionality into yum which:
- violates the principle of least surprise for the admin
- covers up for not-specific-enough requirements in the packaging.
Well, I can understand that point of view partially. The rationale isn't entirely convincing, though. The more explicitly versioned dependencies we would add to packages (either manually or automatically during rpmbuild), the more updates a "yum install" would pull in. It won't be a full update, but could break other installed packages.
Running out-of-date installations is not only a problem when a newly installed package works only with latest updates. Ordinary bugs in dependencies can also be a reason why the newly installed package will need a full "yum update" before it would work [at all or correctly]. Ever seen users with an app startup crash and "yum update APP-PKG-NAME" not fixing it becuase "yum update LIB-PKG-NAME" was necessary?
It just feels like it would be fixing a "problem" at the wrong layer.
Where is the right layer? Do we need to adjust the packaging policies wrt updates?
Fedora 15 by default displays update notifications less frequently.
The package installer could at least notify the user about available updates immediately after installing a package. Some updates could be important, especially if they are a dependency of the newly installed package.
Users with out-of-date installations often are harder to support not just because of old issues they run into.
If application misbehaviour is fixed in a system library, there is a library package update, but the application package normally isn't rebuilt just to add to it a "specific-enough requirement" on the updated library. This isn't limited to one programming language. Even a Python app could crash at run-time because of a bug in a Python module with an update that has not been installed yet.
I think we are ill-advised if we publish a steady flood of updates (even for old dist releases), but want users to install updates less frequently.
On 08/12/2011 04:40 PM, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 08:27:13AM -0700, Toshio Kuratomi wrote:
Rightly or wrongly, upstream libfoo-1.0 has some additional utilities that access the PrivateData. Because the utilities are built from the libfoo source, they can include the fooprivate.h header file and do this. When libfoo goes to 1.0.1, upstream changes the definition of PrivateData and updates the utilities to work with the new datastructure. Since the public ABI stayed the same, the SONAME doesn't change and external programs compiled against libfoo-1.0 continue to work but the utilities built as a subpackage would be broken without stricter versioning.
Upstream can change the ABI as much as they want without bumping the SONAME providing that the old interfaces are also present. It's entirely possible to end up with a situation where external binaries built against 1.0.1 won't run on 1.0.0 - the problem isn't limited to subpackages.
I think Toshio is talking about the case where the subpackage executables are using things that are explicitly outside of any defined ABI (since they are compiled from the same source tree and have access to non-public headers and types defined within them). In his example PrivateData was never part of the interface that external code using -devel sees.
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It's easy to argue that this is a misuse of the interface or that the interface is inadequate/broken but I'm sure there are packages out there with this problem.
Bryn.
On Fri, Aug 12, 2011 at 05:25:17PM +0100, Bryn M. Reeves wrote:
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It is in this situation, but there are other situations where depending on the SONAME will cause breakage. If libfoo 1.1 adds a new symbol, anything built against it may fail to run against libfoo 1.0. But how will you know that in advance if all you have in your dependencies is the SONAME?
On 8/12/11 12:28 PM, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 05:25:17PM +0100, Bryn M. Reeves wrote:
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It is in this situation, but there are other situations where depending on the SONAME will cause breakage. If libfoo 1.1 adds a new symbol, anything built against it may fail to run against libfoo 1.0. But how will you know that in advance if all you have in your dependencies is the SONAME?
In fairness, this is why rpm elaborates soname dependencies to also include symbol versions. It's a pity that symbol versions are so painful to use that really only glibc makes any effort to do it.
Hilariously gcc _does_ let you specify symbol version in a __attribute__ tag, but only on HP/UX on ia64. Thanks for that.
- ajax
On 8/12/2011 1:28 PM, Adam Jackson wrote:
On 8/12/11 12:28 PM, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 05:25:17PM +0100, Bryn M. Reeves wrote:
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It is in this situation, but there are other situations where depending on the SONAME will cause breakage. If libfoo 1.1 adds a new symbol, anything built against it may fail to run against libfoo 1.0. But how will you know that in advance if all you have in your dependencies is the SONAME?
In fairness, this is why rpm elaborates soname dependencies to also include symbol versions. It's a pity that symbol versions are so painful to use that really only glibc makes any effort to do it.
libibverbs uses symbol versions quite nicely.
Hilariously gcc _does_ let you specify symbol version in a __attribute__ tag, but only on HP/UX on ia64. Thanks for that.
- ajax
On Fri, Aug 12, 2011 at 01:28:59PM -0400, Adam Jackson wrote:
On 8/12/11 12:28 PM, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 05:25:17PM +0100, Bryn M. Reeves wrote:
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It is in this situation, but there are other situations where depending on the SONAME will cause breakage. If libfoo 1.1 adds a new symbol, anything built against it may fail to run against libfoo 1.0. But how will you know that in advance if all you have in your dependencies is the SONAME?
In fairness, this is why rpm elaborates soname dependencies to also include symbol versions. It's a pity that symbol versions are so painful to use that really only glibc makes any effort to do it.
We make use of them in libvirt, via the '-Wl,--version-script=libvirt.syms'. While it is slightly unfortunate to have to specify versions outside the header file definitions, it isn't a significant problem. It is immediately obvious if you forget to add new public APIs to sym file, since you won't be able to link/test against them. So I wouldn't really call symbol versoning painful.
What is unfortunate, is that if your library wants to also be maintained on non-Linux platforms, then you can't make full use of the things that glibc's versioning lets you do. eg introducing new APIs with the same name, but different ABI, while not breaking existing linked apps.
Daniel
On Fri, 2011-08-12 at 13:28 -0400, Adam Jackson wrote:
On 8/12/11 12:28 PM, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 05:25:17PM +0100, Bryn M. Reeves wrote:
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It is in this situation, but there are other situations where depending on the SONAME will cause breakage. If libfoo 1.1 adds a new symbol, anything built against it may fail to run against libfoo 1.0. But how will you know that in advance if all you have in your dependencies is the SONAME?
In fairness, this is why rpm elaborates soname dependencies to also include symbol versions. It's a pity that symbol versions are so painful to use that really only glibc makes any effort to do it.
FWIW: talloc, tdb, tevent, and ldb and all of samba4 also do symbol versioning since recently, thanks to waf build extensions that make it simple to do.
Hilariously gcc _does_ let you specify symbol version in a __attribute__ tag, but only on HP/UX on ia64. Thanks for that.
[fail]
Simo.
On Fri, Aug 12, 2011 at 05:28:56PM +0100, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 05:25:17PM +0100, Bryn M. Reeves wrote:
Third party code built against -devel and depending only on the SONAME is fine in this situation as it sticks to the published ABI. In-tree code that plays with non-ABI symbols will break and so may need a stricter dep.
It is in this situation, but there are other situations where depending on the SONAME will cause breakage. If libfoo 1.1 adds a new symbol, anything built against it may fail to run against libfoo 1.0. But how will you know that in advance if all you have in your dependencies is the SONAME?
Yeah, this is what mschwendt was talking about with the potential of yum install <package> leading to brokenness unless yum update is also performed.
My reply was really to refute Andreas's assertion that a subpackage doesn't have any further concerns than an external package wrt version dependencies which is wrong for both code-related reasons like this and for other factors (like the relicensing example).
This thread has shown that we should probably update the guidelines to not appear so draconian about explicit library dependencies (and highlight the ABI/API implications and possibly get changes to rpmbuild), though. Working on an FPC ticket now.
Simo or mjg, one of you want to file a bug against rpmbuild since you guys seem to know how Debian has already implemented this?
-Toshio
On Fri, Aug 12, 2011 at 04:40:20PM +0100, Matthew Garrett wrote:
On Fri, Aug 12, 2011 at 08:27:13AM -0700, Toshio Kuratomi wrote:
Rightly or wrongly, upstream libfoo-1.0 has some additional utilities that access the PrivateData. Because the utilities are built from the libfoo source, they can include the fooprivate.h header file and do this. When libfoo goes to 1.0.1, upstream changes the definition of PrivateData and updates the utilities to work with the new datastructure. Since the public ABI stayed the same, the SONAME doesn't change and external programs compiled against libfoo-1.0 continue to work but the utilities built as a subpackage would be broken without stricter versioning.
Upstream can change the ABI as much as they want without bumping the SONAME providing that the old interfaces are also present. It's entirely possible to end up with a situation where external binaries built against 1.0.1 won't run on 1.0.0 - the problem isn't limited to subpackages.
Sure. But in this case, upstream isn't changing the public ABI.
It's a different level of mistake that's being practiced here.
-Toshio
On Fri, Aug 12, 2011 at 09:26:33AM -0700, Toshio Kuratomi wrote:
On Fri, Aug 12, 2011 at 04:40:20PM +0100, Matthew Garrett wrote:
Upstream can change the ABI as much as they want without bumping the SONAME providing that the old interfaces are also present. It's entirely possible to end up with a situation where external binaries built against 1.0.1 won't run on 1.0.0 - the problem isn't limited to subpackages.
Sure. But in this case, upstream isn't changing the public ABI.
It's a different level of mistake that's being practiced here.
What difference does it make? Even if you stick to the public ABI you can't guarantee that a matching SONAME is sufficient. You need to depend on the package version you build against.
On Fri, 12 Aug 2011 17:30:10 +0100, MG (Matthew) wrote:
On Fri, Aug 12, 2011 at 09:26:33AM -0700, Toshio Kuratomi wrote:
On Fri, Aug 12, 2011 at 04:40:20PM +0100, Matthew Garrett wrote:
Upstream can change the ABI as much as they want without bumping the SONAME providing that the old interfaces are also present. It's entirely possible to end up with a situation where external binaries built against 1.0.1 won't run on 1.0.0 - the problem isn't limited to subpackages.
Sure. But in this case, upstream isn't changing the public ABI.
It's a different level of mistake that's being practiced here.
What difference does it make? Even if you stick to the public ABI you can't guarantee that a matching SONAME is sufficient. You need to depend on the package version you build against.
With the difference that a library subpackage dependency can be _exact_ always, being an "equal to" dependency on full NEVR and staying correct throughout the lifetime of this package set.
On the contrary, any separate package could at most depend on a library package with a ">=" dependency on full NEV (or NEVR), or else a minor version upgrade of the lib would break that dependency already. The dependency would still only be a snapshot made at some point in time, which would not be refreshed with updates of the library package, and which would still not protect against unexpected accidental ABI/API changes in future library updates -- and not against Epoch bumps either (e.g. with an Epoch bump, 1.0.0 suddenly would be larger than 1.0.1 as required by the external program package).
An "equal to" dep on full NEVR (not even NEV might suffice) would be an expensive work-around only, because it forces packages to be rebuilt for every minor lib update.
On Fri, Aug 12, 2011 at 02:30:46PM +0200, Andreas Schwab wrote:
Michael Schwendt mschwendt@gmail.com writes:
The separate "bar" package still is entirely irrelevant
It kills your entire argument.
AFAICS, your insistance that it is a blocker means that you aren't seeing something correctly.
-Toshio
On Thu, 11 Aug 2011 13:19:49 +0100, DH (David) wrote:
Hi,
I have a package (keyutils) that produces three RPMs: keyutils (programs), keyutils-libs and keyutils-devel. The programs in the keyutils RPM depend on the libraries in the keyutils-libs RPM and pick up implicit dependencies thus:
warthog>rpm -qRp keyutils-1.5.1-1.fc14.x86_64.rpm ... config(keyutils) = 1.5.1-1.fc14 ... libkeyutils.so.1()(64bit) libkeyutils.so.1(KEYUTILS_0.3)(64bit) libkeyutils.so.1(KEYUTILS_1.0)(64bit) libkeyutils.so.1(KEYUTILS_1.3)(64bit) libkeyutils.so.1(KEYUTILS_1.4)(64bit) ...
So does the keyutils rpm need an explicit Requires on the keyutils-libs rpm in the specfile or is the implicit library dependency sufficient?
rpmlint does not complain, but, for RHEL, rpmdiff does.
David
It's a grey area. Basically, it boils down to these two guidelines:
https://fedoraproject.org/wiki/Packaging/Guidelines#RequiringBasePackage under consideration of https://fedoraproject.org/wiki/Packaging:Guidelines#Explicit_Requires
In your case, the -libs package is the real base package in disguise. The executables in the "keyutils" pkg may need the latest libkeyutils.so.1 API, or else they would fail to link at run-time.
[...]
Occasionally, one can observe users running into this pitfall as it is not just a theoretical problem. They read about an update notification for package "foo" and try to apply that update manually with "yum update foo". However, due to weak dependencies (or not dependencies) on subpackages containing important stuff, such as the latest version-release of a "foo-libs" subpackage, the applied update is only a partial one -- and not even a "yum update foo*" might have helped, because some subpackage names are within a different prefix.
I have a package (keyutils) that produces three RPMs: keyutils (programs), keyutils-libs and keyutils-devel. The programs in the keyutils RPM depend on the libraries in the keyutils-libs RPM and pick up implicit dependencies thus:
warthog>rpm -qRp keyutils-1.5.1-1.fc14.x86_64.rpm ... config(keyutils) = 1.5.1-1.fc14 ... libkeyutils.so.1()(64bit) ... So does the keyutils rpm need an explicit Requires on the keyutils-libs rpm in the specfile or is the implicit library dependency sufficient?
Since it's a subpkg from the same srpm, I'd opt to be on the safe side, and add an explicit versioned Requires
-- rex