Recompiling/relinking dependent applications/libraries on DSO change

[Florian Weimer]

binutils 2.29 introduced an optimization which requires that in the
general case, applications and libraries linking against a DSO will have
to be rebuilt when the DSO change the implementation of functions (i.e.,
changes to a function body can change ABI).  This is how many native
programming languages (such as Ada, Haskell/GHC, Go, Rust) handle DSOs,
but it's a material change for C and C++.

The question is: Do we want to move into that direction, or do we need
to ask binutils upstream to back out this change?

Thanks,
Florian

[Mark Wielaard]

Hi Florian,

On Fri, 2017-07-28 at 13:39 +0200, Florian Weimer wrote:
>  binutils 2.29 introduced an optimization which requires that in the
>  general case, applications and libraries linking against a DSO will have
>  to be rebuilt when the DSO change the implementation of functions (i.e.,
>  changes to a function body can change ABI).  This is how many native
>  programming languages (such as Ada, Haskell/GHC, Go, Rust) handle DSOs,
>  but it's a material change for C and C++.
>  
>  The question is: Do we want to move into that direction, or do we need
>  to ask binutils upstream to back out this change? 
Could you be a bit more specific? Normally that is why you use symbol
versioning isn't it? Does binutils now warn when it detects such an ABI
change? How does it know?

Thanks,

Mark

[Daniel P. Berrange]

On Fri, Jul 28, 2017 at 01:39:50PM +0200, Florian Weimer wrote:
>  binutils 2.29 introduced an optimization which requires that in the
>  general case, applications and libraries linking against a DSO will have
>  to be rebuilt when the DSO change the implementation of functions (i.e.,
>  changes to a function body can change ABI).  This is how many native
>  programming languages (such as Ada, Haskell/GHC, Go, Rust) handle DSOs,
>  but it's a material change for C and C++. 
Can you elaborate on what sort of change in the function body would
affect the ABI and thus require relinking ?  Having to relink every
time the internal impl of an ABI changes would seem to throw away one
of the main benefits of using shared libs, over static libs and be
pretty unpleasant as a result.

Libvirt, for example, has never changed any existing public API contract
or definition in 10 years of releases, but we do often change the internal
impl of these APIs. Apps have never had to relink against libvirt.so upon
new release, so I would not want that to see that change.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

[Florian Weimer]

On 07/28/2017 02:22 PM, Mark Wielaard wrote:
>  Hi Florian,
>  
>  On Fri, 2017-07-28 at 13:39 +0200, Florian Weimer wrote:
> > binutils 2.29 introduced an optimization which requires that in the
> > general case, applications and libraries linking against a DSO will have
> > to be rebuilt when the DSO change the implementation of functions (i.e.,
> > changes to a function body can change ABI).  This is how many native
> > programming languages (such as Ada, Haskell/GHC, Go, Rust) handle DSOs,
> > but it's a material change for C and C++.
> >
> > The question is: Do we want to move into that direction, or do we need
> > to ask binutils upstream to back out this change?
>  
>  Could you be a bit more specific? Normally that is why you use symbol
>  versioning isn't it? Does binutils now warn when it detects such an ABI
>  change? How does it know? 
Conceptually, this optimization inlines aspects of the called function
into the caller, across DSO boundaries.  In particular, as implemented
now on ppc64le, the net effect is that the ABI changes if you add a
global variable access to a function which previously did not have one.

I posted a more technical summary here:

  <https://sourceware.org/ml/libc-alpha/2017-07/msg00968.html>

It is theoretical possible to make this work with symbol versions, but
we don't have any tools for that right now.

Writing this message I realized that it's not worth to have a serious
discussion about this binutils change.  It's just wrong and breaks ELF
dynamic linking.  Sorry about the noise.

Thanks,
Florian

[Jakub Jelinek]

On Fri, Jul 28, 2017 at 03:06:29PM +0200, Florian Weimer wrote:
>  On 07/28/2017 02:22 PM, Mark Wielaard wrote:
>  > On Fri, 2017-07-28 at 13:39 +0200, Florian Weimer wrote:
>  >> binutils 2.29 introduced an optimization which requires that in the
>  >> general case, applications and libraries linking against a DSO will have
>  >> to be rebuilt when the DSO change the implementation of functions (i.e.,
>  >> changes to a function body can change ABI).  This is how many native
>  >> programming languages (such as Ada, Haskell/GHC, Go, Rust) handle DSOs,
>  >> but it's a material change for C and C++.
>  >>
>  >> The question is: Do we want to move into that direction, or do we need
>  >> to ask binutils upstream to back out this change?
>  > 
>  > Could you be a bit more specific? Normally that is why you use symbol
>  > versioning isn't it? Does binutils now warn when it detects such an ABI
>  > change? How does it know?
>  
>  Conceptually, this optimization inlines aspects of the called function
>  into the caller, across DSO boundaries.  In particular, as implemented
>  now on ppc64le, the net effect is that the ABI changes if you add a
>  global variable access to a function which previously did not have one.
>  
>  I posted a more technical summary here:
>  
>    <https://sourceware.org/ml/libc-alpha/2017-07/msg00968.html>
>  
>  It is theoretical possible to make this work with symbol versions, but
>  we don't have any tools for that right now.
>  
>  Writing this message I realized that it's not worth to have a serious
>  discussion about this binutils change.  It's just wrong and breaks ELF
>  dynamic linking.  Sorry about the noise. 
I think it is seriously flawed optimization that should be at least turned
off by default.  If people are willing to do this mess on ppc64le for some
libraries, they should request it explicitly, but we certainly shouldn't
make the implementation details part of exported ABI for all shared
libraries.  Then the ABI depends even on compiler flags used to compile the
routines etc., you recompile with -O0 and suddenly you've changed ABI.
Furthermore, how does it work with symbol interposition?  Say if malloc
happened to not clobber/use r2 and does not use r12, you suddenly couldn't
use ElectricFence or valgrind or AddressSanitizer where the implementation
would need to do that.

Can the decision if this braindamange should be used or not be done on a
per-shared library basis, then it might be useful for shared libraries
solely used by binaries in the same package where nothing else uses them.
But otherwise, just say no to this.  Ugh.

	Jakub

[Jakub Jelinek]

On Fri, Jul 28, 2017 at 03:20:49PM +0200, Jakub Jelinek wrote:
>  > Conceptually, this optimization inlines aspects of the called
> function
>  > into the caller, across DSO boundaries.  In particular, as implemented
>  > now on ppc64le, the net effect is that the ABI changes if you add a
>  > global variable access to a function which previously did not have one.
>  > 
>  > I posted a more technical summary here:
>  > 
>  >   <https://sourceware.org/ml/libc-alpha/2017-07/msg00968.html> 
Note link to the implementation and rationale is
https://sourceware.org/ml/binutils/2017-06/msg00007.html

	Jakub

[Florian Weimer]

On 07/28/2017 03:20 PM, Jakub Jelinek wrote:
>  I think it is seriously flawed optimization that should be at least
> turned
>  off by default.  If people are willing to do this mess on ppc64le for some
>  libraries, they should request it explicitly, but we certainly shouldn't
>  make the implementation details part of exported ABI for all shared
>  libraries.  Then the ABI depends even on compiler flags used to compile the
>  routines etc., you recompile with -O0 and suddenly you've changed ABI.
>  Furthermore, how does it work with symbol interposition?  Say if malloc
>  happened to not clobber/use r2 and does not use r12, you suddenly couldn't
>  use ElectricFence or valgrind or AddressSanitizer where the implementation
>  would need to do that. 
All the things you list are potentially broken.  As I said, it's not
worth discussing this.  It's not supportable.

I don't even want to ship this as an optional feature.

Thanks,
Florian

[Richard W.M. Jones]

On Fri, Jul 28, 2017 at 01:39:50PM +0200, Florian Weimer wrote:
>  binutils 2.29 introduced an optimization which requires that in the
>  general case, applications and libraries linking against a DSO will have
>  to be rebuilt when the DSO change the implementation of functions (i.e.,
>  changes to a function body can change ABI).  This is how many native
>  programming languages (such as Ada, Haskell/GHC, Go, Rust) handle DSOs,
>  but it's a material change for C and C++.
>  
>  The question is: Do we want to move into that direction, or do we need
>  to ask binutils upstream to back out this change? 
I think you summed up the proper response in your reply here:

  https://sourceware.org/ml/libc-alpha/2017-07/msg00974.html

"Ugh."

OCaml does cross-module function inlining.  It's a nice feature for
getting fast code, but from a packaging perspective it means you have
to rebuild the world every time a dependent library changes even
internally, and that's not nice.  It's only really possible for Fedora
because there are fewer than 100 packages.

Also as Dan mentioned, libvirt.so offers a stable ABI so existing
binaries must continue to work.  Avoiding use of globals in any future
libvirt (transitively if functions are called or inlined within the
DSO?) doesn't sound like it would be possible in general, or pleasant
if it is possible.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-df lists disk usage of guests without needing to install any
software inside the virtual machine.  Supports Linux and Windows.
http://people.redhat.com/~rjones/virt-df/