libtool issue with EPEL10

[Orion Poplawski]

I'm trying to build netcdf in EPEL10 but the test suite is failing to 
run with:

/builddir/build/BUILD/netcdf-c-4.9.2/build/ncgen3/.libs/lt-ncgen3: error 
while loading shared libraries: libnetcdf.so.19: cannot open shared 
object file: No such file or directory

Comparing EPEL10 to rawhide builds I notice that while the EPEL10 build 
produces an "lt-ncgen3" binary, rawhide does not.  The rawhide ncgen3 
wrapper script appears to set LD_LIBRARY_PATH, the EPEL10 one does not 
and instead seems to try to produce a "relinked" file.

I don't think I have every run into this issue before and have no idea 
why it's happening.

Any libtool experts out there with an idea?

Thanks,

Orion

[Orion Poplawski]

On 10/27/24 20:17, Orion Poplawski wrote:
> I'm trying to build netcdf in EPEL10 but the test suite is failing to 
> run with:
>
> /builddir/build/BUILD/netcdf-c-4.9.2/build/ncgen3/.libs/lt-ncgen3: error 
> while loading shared libraries: libnetcdf.so.19: cannot open shared 
> object file: No such file or directory
>
> Comparing EPEL10 to rawhide builds I notice that while the EPEL10 build 
> produces an "lt-ncgen3" binary, rawhide does not.  The rawhide ncgen3 
> wrapper script appears to set LD_LIBRARY_PATH, the EPEL10 one does not 
> and instead seems to try to produce a "relinked" file.
>
> I don't think I have every run into this issue before and have no idea 
> why it's happening.
>
> Any libtool experts out there with an idea?

Some more differences:

--- 
fedora-rawhide-x86_64/root/builddir/build/BUILD/netcdf-4.9.2-build/netcdf-c-4.9.2/build/libtool 
     2024-10-29 20:40:32.996721304 -0600
+++ 
centos-stream+epel-10-x86_64/root/builddir/build/BUILD/netcdf-c-4.9.2/build/libtool 
2024-10-29 20:35:41.753975205 -0600

# Whether or not to optimize for fast installation.
-fast_install=needless
+fast_install=yes

# Is shlibpath searched before the hard-coded library search path?
-shlibpath_overrides_runpath=yes
+shlibpath_overrides_runpath=no

From https://www.gnu.org/software/libtool/manual/libtool.html

Variable: shlibpath_overrides_runpath

Indicates whether it is possible to override the hard-coded library 
search path of a program with an environment variable. If this is set to 
no, libtool may have to create two copies of a program in the build 
tree, one to be installed and one to be run in the build tree only. When 
each of these copies is created depends on the value of fast_install. 
The default value is ‘unknown’, which is equivalent to ‘no’.

This appears to be the difference that I'm seeing.  It also seems to 
just be wrong.

There is no (real) difference between the libtool.spec file in rawhide 
and 10-stream.  I can't find any public build logs though for centos 
stream builds.

It's also very strange that this is the first package I've built that 
has had this issue.

I've filed https://issues.redhat.com/browse/RHEL-65261

[Orion Poplawski]

On 10/29/24 22:32, Orion Poplawski wrote:
> On 10/27/24 20:17, Orion Poplawski wrote:
> > I'm trying to build netcdf in EPEL10 but the test suite is failing to 
> > run with:
> >
> > /builddir/build/BUILD/netcdf-c-4.9.2/build/ncgen3/.libs/lt-ncgen3: 
> > error while loading shared libraries: libnetcdf.so.19: cannot open 
> > shared object file: No such file or directory
> >
> > Comparing EPEL10 to rawhide builds I notice that while the EPEL10 
> > build produces an "lt-ncgen3" binary, rawhide does not.  The rawhide 
> > ncgen3 wrapper script appears to set LD_LIBRARY_PATH, the EPEL10 one 
> > does not and instead seems to try to produce a "relinked" file.
> >
> > I don't think I have every run into this issue before and have no idea 
> > why it's happening.
> >
> > Any libtool experts out there with an idea?
>
> Some more differences:
>
> --- fedora-rawhide-x86_64/root/builddir/build/BUILD/netcdf-4.9.2-build/ 
> netcdf-c-4.9.2/build/libtool     2024-10-29 20:40:32.996721304 -0600
> +++ centos-stream+epel-10-x86_64/root/builddir/build/BUILD/netcdf- 
> c-4.9.2/build/libtool 2024-10-29 20:35:41.753975205 -0600
>
> # Whether or not to optimize for fast installation.
> -fast_install=needless
> +fast_install=yes
>
> # Is shlibpath searched before the hard-coded library search path?
> -shlibpath_overrides_runpath=yes
> +shlibpath_overrides_runpath=no
>
> From https://www.gnu.org/software/libtool/manual/libtool.html
>
> Variable: shlibpath_overrides_runpath
>
> Indicates whether it is possible to override the hard-coded library 
> search path of a program with an environment variable. If this is set to 
> no, libtool may have to create two copies of a program in the build 
> tree, one to be installed and one to be run in the build tree only. When 
> each of these copies is created depends on the value of fast_install. 
> The default value is ‘unknown’, which is equivalent to ‘no’.
>
> This appears to be the difference that I'm seeing.  It also seems to 
> just be wrong.
>
> There is no (real) difference between the libtool.spec file in rawhide 
> and 10-stream.  I can't find any public build logs though for centos 
> stream builds.
>
> It's also very strange that this is the first package I've built that 
> has had this issue.
>
> I've filed https://issues.redhat.com/browse/RHEL-65261

The issue appears to be caused by the rpath workaround from here:

https://docs.fedoraproject.org/en-US/packaging-guidelines/#_removing_rpath

sed -i 's|^hardcode_libdir_flag_spec=.*|hardcode_libdir_flag_spec=""|g' 
libtool
sed -i 's|^runpath_var=LD_RUN_PATH|runpath_var=DIE_RPATH_DIE|g' libtool

Removing that from the netcdf package allows the tests to run.  Still 
need to confirm that rpath does not return without it though.

[Yaakov Selkowitz]

On 10/30/24 10:22, Orion Poplawski wrote:
> On 10/29/24 22:32, Orion Poplawski wrote:
> > On 10/27/24 20:17, Orion Poplawski wrote:
> > > I'm trying to build netcdf in EPEL10 but the test suite is failing to 
> > > run with:
> > >
> > > /builddir/build/BUILD/netcdf-c-4.9.2/build/ncgen3/.libs/lt-ncgen3: 
> > > error while loading shared libraries: libnetcdf.so.19: cannot open 
> > > shared object file: No such file or directory
> > >
> > > Comparing EPEL10 to rawhide builds I notice that while the EPEL10 
> > > build produces an "lt-ncgen3" binary, rawhide does not.  The rawhide 
> > > ncgen3 wrapper script appears to set LD_LIBRARY_PATH, the EPEL10 one 
> > > does not and instead seems to try to produce a "relinked" file.
> > >
> > > I don't think I have every run into this issue before and have no 
> > > idea why it's happening.
> > >
> > > Any libtool experts out there with an idea?
> >
> > Some more differences:
> >
> > --- fedora-rawhide-x86_64/root/builddir/build/BUILD/netcdf-4.9.2- 
> > build/ netcdf-c-4.9.2/build/libtool     2024-10-29 20:40:32.996721304 
> > -0600
> > +++ centos-stream+epel-10-x86_64/root/builddir/build/BUILD/netcdf- 
> > c-4.9.2/build/libtool 2024-10-29 20:35:41.753975205 -0600
> >
> > # Whether or not to optimize for fast installation.
> > -fast_install=needless
> > +fast_install=yes
> >
> > # Is shlibpath searched before the hard-coded library search path?
> > -shlibpath_overrides_runpath=yes
> > +shlibpath_overrides_runpath=no
> >
> > From https://www.gnu.org/software/libtool/manual/libtool.html
> >
> > Variable: shlibpath_overrides_runpath
> >
> > Indicates whether it is possible to override the hard-coded 
> > library search path of a program with an environment variable. If this 
> > is set to no, libtool may have to create two copies of a program in 
> > the build tree, one to be installed and one to be run in the build 
> > tree only. When each of these copies is created depends on the value 
> > of fast_install. The default value is ‘unknown’, which is equivalent 
> > to ‘no’.
> >
> > This appears to be the difference that I'm seeing.  It also seems to 
> > just be wrong.
> >
> > There is no (real) difference between the libtool.spec file in rawhide 
> > and 10-stream.  I can't find any public build logs though for centos 
> > stream builds.
> >
> > It's also very strange that this is the first package I've built that 
> > has had this issue.
> >
> > I've filed https://issues.redhat.com/browse/RHEL-65261
>
> The issue appears to be caused by the rpath workaround from here:
>
> https://docs.fedoraproject.org/en-US/packaging-guidelines/#_removing_rpath
>
> sed -i 's|^hardcode_libdir_flag_spec=.*|hardcode_libdir_flag_spec=""|g' 
> libtool
> sed -i 's|^runpath_var=LD_RUN_PATH|runpath_var=DIE_RPATH_DIE|g' libtool
>
> Removing that from the netcdf package allows the tests to run.  Still 
> need to confirm that rpath does not return without it though.

The reason for this is 
https://src.fedoraproject.org/rpms/binutils/blob/rawhide/f/binutils.spec#_56 
IOW the lack of -Wl,--enable-new-dtags on RHEL combined with no RPATH 
means that you have to take full responsibility for assuring that the 
in-tree library is in LD_LIBRARY_PATH yourself.