.so's in devel packages...

[Nelson Marques]

Hi all,

I have a doubt regarding the '.so's' in devel packages... From my
understanding they go in devel packages to allow the installation of
several packages with different versioning....
Who defined this? Is this part of some standards (ex: LSB, etc) ?

Is there some written documentation about this ?

NM

[Michael Schwendt]

On Mon, 18 Jun 2012 18:23:16 +0100, Nelson Marques wrote:
> Hi all,
>
> I have a doubt regarding the '.so's' in devel packages... From my
> understanding they go in devel packages to allow the installation of
> several packages with different versioning....
> Who defined this? Is this part of some standards (ex: LSB, etc) ?
>
> Is there some written documentation about this ?

Please be *much* more verbose.

https://fedoraproject.org/wiki/Packaging:Guidelines#Devel_Packages

Thanks in advance!

[Adam Williamson]

On Mon, 2012-06-18 at 18:23 +0100, Nelson Marques wrote:
> Hi all,
>
> I have a doubt regarding the '.so's' in devel packages... From my
> understanding they go in devel packages to allow the installation of
> several packages with different versioning....

Not really, no. They go in -devel packages because the only time it's
actually appropriate to use a library by referring to its unversioned
name is when you're compiling another application against it. It's never
safe for anything to access a library at runtime via an unversioned name
because there is no guarantee of stability; you can't be at all sure
that the version of the library you're calling is actually capable of
doing what you're asking it to do. Since the only use of the unversioned
'instance' (symlink, in Fedora...) of a library is in development,
naturally it goes in the devel package. We can take advantage of this in
generating dependencies, and we do, which is why it's important not to
put the .so file in a runtime package, or that runtime package will get
a bunch of automatically generated dependencies on -devel packages.

The whole system of naming and versioning of shared libraries, and the
appropriate way of packaging them, is somewhat complex but very neat and
makes a lot of sense. It's worth reading up and understanding exactly
how it works and what it achieves. I don't know if there's a 'canonical'
reference for this, but a bit of Googling produces:

http://static.usenix.org/publications/library/proceedings/als00/2000papers/p...

Which seems like a rather good overview.
> Who defined this? Is this part of some standards (ex: LSB, etc) ?
>
> Is there some written documentation about this ?

See above. Also
https://www.ibm.com/developerworks/linux/library/l-shlibs/index.html has
some good stuff.
http://netwinder.osuosl.org/pub/netwinder/docs/misc/GCC-HOWTO-html/GCC-HOWTO... has more of the same, and explains the wrinkle of the 'soname' that's actually embedded in the library file quite well.

This isn't Linux-specific, note. Most *nixes use the same format (ELF)
and the same conventions with regard to linking and versioning shared
libraries.

[Nelson Marques]

Adam,

Thanks, this helps a lot understanding a bit more around shared libraries;

NM

2012/6/18 Adam Williamson
> On Mon, 2012-06-18 at 18:23 +0100, Nelson Marques wrote:
> > Hi all,
> >
> > I have a doubt regarding the '.so's' in devel packages... From my
> > understanding they go in devel packages to allow the installation of
> > several packages with different versioning....
>
> Not really, no. They go in -devel packages because the only time it's
> actually appropriate to use a library by referring to its unversioned
> name is when you're compiling another application against it. It's never
> safe for anything to access a library at runtime via an unversioned name
> because there is no guarantee of stability; you can't be at all sure
> that the version of the library you're calling is actually capable of
> doing what you're asking it to do. Since the only use of the unversioned
> 'instance' (symlink, in Fedora...) of a library is in development,
> naturally it goes in the devel package. We can take advantage of this in
> generating dependencies, and we do, which is why it's important not to
> put the .so file in a runtime package, or that runtime package will get
> a bunch of automatically generated dependencies on -devel packages.
>
> The whole system of naming and versioning of shared libraries, and the
> appropriate way of packaging them, is somewhat complex but very neat and
> makes a lot of sense. It's worth reading up and understanding exactly
> how it works and what it achieves. I don't know if there's a 'canonical'
> reference for this, but a bit of Googling produces:
>
> http://static.usenix.org/publications/library/proceedings/als00/2000papers/p...
>
> Which seems like a rather good overview.
> > Who defined this? Is this part of some standards (ex: LSB, etc) ?
> >
> > Is there some written documentation about this ?
>
> See above. Also
> https://www.ibm.com/developerworks/linux/library/l-shlibs/index.html has
> some good stuff.
> http://netwinder.osuosl.org/pub/netwinder/docs/misc/GCC-HOWTO-html/GCC-HOWTO... has more of the same, and explains the wrinkle of the 'soname' that's actually embedded in the library file quite well.
>
> This isn't Linux-specific, note. Most *nixes use the same format (ELF)
> and the same conventions with regard to linking and versioning shared
> libraries.
> --
> Adam Williamson
> Fedora QA Community Monkey
> IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
> http://www.happyassassin.net
>
> --
> devel mailing list
> [e-mail]
> https://admin.fedoraproject.org/mailman/listinfo/devel

[Michael Schwendt]

On Mon, 18 Jun 2012 12:25:12 -0700, Adam Williamson wrote:
> On Mon, 2012-06-18 at 18:23 +0100, Nelson Marques wrote:
> > Hi all,
> >
> > I have a doubt regarding the '.so's' in devel packages... From my
> > understanding they go in devel packages to allow the installation of
> > several packages with different versioning....
>
> Not really, no. They go in -devel packages because the only time it's
> actually appropriate to use a library by referring to its unversioned
> name is when you're compiling another application against it. It's never
> safe for anything to access a library at runtime via an unversioned name
> because there is no guarantee of stability; you can't be at all sure
> that the version of the library you're calling is actually capable of
> doing what you're asking it to do. Since the only use of the unversioned
> 'instance' (symlink, in Fedora...) of a library is in development,
> naturally it goes in the devel package. We can take advantage of this in
> generating dependencies, and we do, which is why it's important not to
> put the .so file in a runtime package, or that runtime package will get
> a bunch of automatically generated dependencies on -devel packages.

And again, this is not the full story. There is no hard rule on where 
non-versioned .so files are to be packaged. They could still be local libs
(with no API for public consumption) strictly required by an application.
They could still be plug-ins, libraries loaded by an application at
run-time. They could even be symlinks to versioned plug-in libs, with
the application strictly requiring the non-versioned symlink when trying
to link with a plug-in at run-time.

Where .so files are to be put solely depends on their purpose. Many
non-versioned libfoo.so files are just the symlink that make -lfoo work
during compilation/linking. But that is not sufficient to require *all*
non-versioned .so files to be placed in -devel packages.

If Fedora Packaging Guidelines are still not clear about this, we may need
another update for them. But detailed feedback on them would be appreciated
first, IMHO.

[Adam Williamson]

On Tue, 2012-06-19 at 09:50 +0200, Michael Schwendt wrote:
> On Mon, 18 Jun 2012 12:25:12 -0700, Adam Williamson wrote:
> > On Mon, 2012-06-18 at 18:23 +0100, Nelson Marques wrote:
> > > Hi all,
> > >
> > > I have a doubt regarding the '.so's' in devel packages... From my
> > > understanding they go in devel packages to allow the installation of
> > > several packages with different versioning....
> >
> > Not really, no. They go in -devel packages because the only time it's
> > actually appropriate to use a library by referring to its unversioned
> > name is when you're compiling another application against it. It's never
> > safe for anything to access a library at runtime via an unversioned name
> > because there is no guarantee of stability; you can't be at all sure
> > that the version of the library you're calling is actually capable of
> > doing what you're asking it to do. Since the only use of the unversioned
> > 'instance' (symlink, in Fedora...) of a library is in development,
> > naturally it goes in the devel package. We can take advantage of this in
> > generating dependencies, and we do, which is why it's important not to
> > put the .so file in a runtime package, or that runtime package will get
> > a bunch of automatically generated dependencies on -devel packages.
>
> And again, this is not the full story.

I was trying to keep it simple.
> There is no hard rule on where 
> non-versioned .so files are to be packaged. They could still be local libs
> (with no API for public consumption) strictly required by an application.

I tend to talk about these as 'plugins', myself, to keep them distinct.
They're really _not_ shared libraries, in my worldview anyway - as you
say, they present no public API, they're not intended to be used by
anything else. They just happen to use the same file format. In the
context of the question - which was about .so's which _are_ in -devel
packages - it was clear the OP was asking about true shared libraries,
not private plugins.
> They could still be plug-ins, libraries loaded by an application at
> run-time. They could even be symlinks to versioned plug-in libs, with
> the application strictly requiring the non-versioned symlink when trying
> to link with a plug-in at run-time.
>
> Where .so files are to be put solely depends on their purpose. Many
> non-versioned libfoo.so files are just the symlink that make -lfoo work
> during compilation/linking. But that is not sufficient to require *all*
> non-versioned .so files to be placed in -devel packages.

Indeed.
> If Fedora Packaging Guidelines are still not clear about this, we may need
> another update for them. But detailed feedback on them would be appreciated
> first, IMHO.

I think the guidelines assume a baseline level of knowledge about
library versioning, which probably isn't a terrible thing, but maybe a
reference or two would help.

[Michael Schwendt]

On Tue, 19 Jun 2012 08:39:40 -0700, Adam Williamson wrote:
> > And again, this is not the full story.
>
> I was trying to keep it simple.

The simplified case/explanation is what encourages packagers to misplace
.so files. For example, -devel packages that contain .so files but no
header files should be a sign of warning to the packagers.
> > There is no hard rule on where 
> > non-versioned .so files are to be packaged. They could still be local libs
> > (with no API for public consumption) strictly required by an application.
>
> I tend to talk about these as 'plugins', myself, to keep them distinct.
> They're really _not_ shared libraries, in my worldview anyway - as you
> say, they present no public API, they're not intended to be used by
> anything else.

Cannot agree with that worldview then. ;) They are dynamic shared object
files at the technical level and may be shared by multiple executables
and/or other libraries, too. Even if they are not versioned and don't use
a SONAME. The subtle (and important) difference for the packager is that
they may be _local_ to a package.
> They just happen to use the same file format. In the
> context of the question - which was about .so's which _are_ in -devel
> packages - it was clear the OP was asking about true shared libraries,
> not private plugins.

The OP also referred to something else: packaging multiple versions of a
library and making the packages parallel-installable. Now, at Fedora,
moving the .so symlink (here really the one used during development only)
to a -devel package indeed removes one potential file that would cause a
conflict,

libfoo.so.3 <- libfoo.so
  libfoo.so.4 <- libfoo.so

but those -devel packages would conflict, and that would not be permitted
by Fedora's packaging guidelines. It would be necessary to move the .so
symlink even further, e.g. into its own directory (requiring compilation
of programs to alter the build-time search path for libs).