Static library linking error

[Richard Shaw]

I'm working on some IIoT related packages in my COPR where I have a dynamic
library linking to a static library and getting the following error:

[ 18%] Linking C shared library libneuron-base.so
/usr/bin/cmake -E cmake_link_script CMakeFiles/neuron-base.dir/link.txt
--verbose=1
/usr/bin/gcc -fPIC -O2 -flto=auto -ffat-lto-objects -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1   -m64  -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wall
-Wextra -g -O2 -flto=auto -ffat-lto-objects -fexceptions -g
-grecord-gcc-switches -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1   -m64  -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wall
-Wextra -g -fsanitize=address   -fsanitize-recover=address
-fsanitize-address-use-after-scope   -fno-stack-protector
-fno-omit-frame-pointer -fno-var-tracking -Wl,-z,relro -Wl,--as-needed
 -Wl,-z,now -specs=/usr/lib/rpm/redhat/redhat-hardened-ld
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1
-Wl,-dT,/builddir/build/BUILD/neuron-2.0.1/.package_note-emqx-neuron-2.0.1-1.fc37.x86_64.ld
-shared -Wl,-soname,libneuron-base.so -o libneuron-base.so
"CMakeFiles/neuron-base.dir/src/types.c.o"
"CMakeFiles/neuron-base.dir/src/base/tag_table.c.o"
"CMakeFiles/neuron-base.dir/src/base/neu_tag.c.o"
"CMakeFiles/neuron-base.dir/src/base/neu_datatag_table.c.o"
"CMakeFiles/neuron-base.dir/src/base/neu_tag_group_config.c.o"
"CMakeFiles/neuron-base.dir/src/base/neu_plugin_common.c.o"
"CMakeFiles/neuron-base.dir/src/base/neu_data_expr.c.o"
"CMakeFiles/neuron-base.dir/src/base/tag_class.c.o"
"CMakeFiles/neuron-base.dir/src/connection/tcp_client.c.o"
"CMakeFiles/neuron-base.dir/src/connection/tcp_server.c.o"
"CMakeFiles/neuron-base.dir/src/connection/connection.c.o"
"CMakeFiles/neuron-base.dir/src/connection/mqtt_client_intf.c.o"
"CMakeFiles/neuron-base.dir/src/event/event_linux.c.o"
"CMakeFiles/neuron-base.dir/src/event/event_unix.c.o"
"CMakeFiles/neuron-base.dir/src/utils/atomic_data.c.o"
"CMakeFiles/neuron-base.dir/src/utils/idhash.c.o"
"CMakeFiles/neuron-base.dir/src/utils/hash_table.c.o"
"CMakeFiles/neuron-base.dir/src/utils/panic.c.o"
"CMakeFiles/neuron-base.dir/src/utils/log.c.o"
"CMakeFiles/neuron-base.dir/src/utils/json.c.o"
"CMakeFiles/neuron-base.dir/src/utils/neu_jwt.c.o"
"CMakeFiles/neuron-base.dir/src/utils/file.c.o"
"CMakeFiles/neuron-base.dir/src/utils/base64.c.o"
"CMakeFiles/neuron-base.dir/src/config.c.o"
"CMakeFiles/neuron-base.dir/src/connection/mqtt_c_client.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_error.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_fn.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_group_config.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_license.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_log.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_login.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_mqtt.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_node.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_plugin.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_rw.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_tag.c.o"
"CMakeFiles/neuron-base.dir/src/parser/neu_json_tty.c.o"   -L/usr/local/lib
 -Wl,-rpath,./:/usr/local/lib: -lzlog extern/vector/libvector-static.a
extern/libcsptr/libcsptr.a -ljansson -lcrypto -lssl -lmqttc -lyaml -ljwt
/usr/bin/ld:
/usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
warning: relocation against `mqtt_fixed_header_rules' in read-only section
`.text'
/usr/bin/ld:
/usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules' can not
be used when making a shared object; recompile with -fPIC

I added the following to the libmqttc library and verified -fPIC -pie is in
the build flags[1] per the recommendation from the hardening page[2] but
the error remains.

Any ideas?

Thanks,
Richard

[1]
https://download.copr.fedorainfracloud.org/results/hobbes1069/IIoT/fedora-ra...
[2] https://fedoraproject.org/wiki/Changes/Harden_All_Packages

[Mamoru TASAKA]

Richard Shaw wrote on 2022/05/10 12:07:
> I'm working on some IIoT related packages in my COPR where I have a dynamic
> library linking to a static library and getting the following error:
> /usr/bin/ld:
> /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
> warning: relocation against `mqtt_fixed_header_rules' in read-only section
> `.text'
> /usr/bin/ld:
> /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
> relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules' can not
> be used when making a shared object; recompile with -fPIC
>
> I added the following to the libmqttc library and verified -fPIC -pie is in
> the build flags[1] per the recommendation from the hardening page[2] but
> the error remains.
>
> Any ideas?
>
> Thanks,
> Richard
>
> [1]
> https://download.copr.fedorainfracloud.org/results/hobbes1069/IIoT/fedora-ra...

This log no longer seems to exist.
> [2] https://fedoraproject.org/wiki/Changes/Harden_All_Packages

Regards,
Mamoru

[John Reiser]

On 5/10/22 06:21 UTC, Mamoru TASAKA wrote:
> Richard Shaw wrote on 2022/05/10 12:07:
> > I'm working on some IIoT related packages in my COPR where I have a dynamic
> > library linking to a static library and getting the following error:
> > /usr/bin/ld:
> > /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
> > warning: relocation against `mqtt_fixed_header_rules' in read-only section
> > `.text'
> > /usr/bin/ld:
> > /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
> > relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules' can not
> > be used when making a shared object; recompile with -fPIC
> >
> > I added the following to the libmqttc library and verified -fPIC -pie is in
> > the build flags[1] per the recommendation from the hardening page[2] but
> > the error remains.
> >
> > Any ideas?
> >
> > Thanks,
> > Richard
> >
> > [1]
> > https://download.copr.fedorainfracloud.org/results/hobbes1069/IIoT/fedora-ra...
>
> This log no longer seems to exist.

I was able to access it just now.

Some relevant lines are:
=====
[ 18%] Building C object CMakeFiles/mqttc.dir/src/mqtt.c.o
/usr/bin/gcc -DMQTT_USE_BIO -I/builddir/build/BUILD/MQTT-C-1.1.5/include -O2 -flto=auto -ffat-lto-objects -fexceptions \
-g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS \
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  	 \
-m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC -pie -MD -MT \
CMakeFiles/mqttc.dir/src/mqtt.c.o -MF CMakeFiles/mqttc.dir/src/mqtt.c.o.d -o CMakeFiles/mqttc.dir/src/mqtt.c.o \
-c /builddir/build/BUILD/MQTT-C-1.1.5/src/mqtt.c
[ 27%] Linking C static library libmqttc.a
/usr/bin/cmake -P CMakeFiles/mqttc.dir/cmake_clean_target.cmake
/usr/bin/cmake -E cmake_link_script CMakeFiles/mqttc.dir/link.txt --verbose=1
/usr/bin/ar qc libmqttc.a CMakeFiles/mqttc.dir/src/mqtt_pal.c.o CMakeFiles/mqttc.dir/src/mqtt.c.o
/usr/bin/ranlib libmqttc.a
=====
which confirms that "-fPIC -pie" was used when compiling mqtt.c into CMakeFiles/mqttc.dir/src/mqtt.c.o .

Suggestion: extract mqtt.c.o from libmqttc.a, then run "readelf --all --wide mqtt.c.o  > foo"
and look in file foo for more information about:
    relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules'

Also, upstream should remedy complaints from the compiler:
=====
/builddir/build/BUILD/MQTT-C-1.1.5/examples/bio_publisher.c: In function 'main':
/builddir/build/BUILD/MQTT-C-1.1.5/examples/bio_publisher.c:47:5: warning: 'ERR_load_BIO_strings' is deprecated: \
Since OpenSSL 3.0 [-Wdeprecated-declarations]
    47 |     ERR_load_BIO_strings();
       |     ^~~~~~~~~~~~~~~~~~~~
In file included from /usr/include/openssl/cryptoerr.h:17,
                  from /usr/include/openssl/crypto.h:38,
                  from /usr/include/openssl/bio.h:30,
                  from /builddir/build/BUILD/MQTT-C-1.1.5/include/mqtt_pal.h:100,
                  from /builddir/build/BUILD/MQTT-C-1.1.5/include/mqtt.h:43,
                  from /builddir/build/BUILD/MQTT-C-1.1.5/examples/bio_publisher.c:10:
/usr/include/openssl/cryptoerr_legacy.h:31:27: note: declared here
    31 | OSSL_DEPRECATEDIN_3_0 int ERR_load_BIO_strings(void);
       |                           ^~~~~~~~~~~~~~~~~~~~
=====
and:
=====
/builddir/build/BUILD/MQTT-C-1.1.5/examples/simple_subscriber.c: In function 'main':
/builddir/build/BUILD/MQTT-C-1.1.5/examples/simple_subscriber.c:73:24: warning: passing argument 2 of 'mqtt_init' makes pointer from integer without a cast [-Wint-conversion]
    73 |     mqtt_init(&client, sockfd, sendbuf, sizeof(sendbuf), recvbuf, sizeof(recvbuf), publish_callback);
       |                        ^~~~~~
       |                        |
       |                        int
=====
plus several more int vs pointer conflicts.

[Richard Shaw]

On Tue, May 10, 2022 at 3:09 AM John Reiser  wrote:
> On 5/10/22 06:21 UTC, Mamoru TASAKA wrote:
> > Richard Shaw wrote on 2022/05/10 12:07:
> > > I'm working on some IIoT related packages in my COPR where I have a
>
> dynamic
> > > library linking to a static library and getting the following error:
> > > /usr/bin/ld:
>
> /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
> > > warning: relocation against `mqtt_fixed_header_rules' in read-only
>
> section
> > > `.text'
> > > /usr/bin/ld:
>
> /usr/lib/gcc/x86_64-redhat-linux/12/../../../../lib64/libmqttc.a(mqtt.c.o):
> > > relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules' can
>
> not
> > > be used when making a shared object; recompile with -fPIC
> > >
> > > I added the following to the libmqttc library and verified -fPIC -pie
>
> is in
> > > the build flags[1] per the recommendation from the hardening page[2] but
> > > the error remains.
> > >
> > > Any ideas?
> > >
> > > Thanks,
> > > Richard
> > >
> > > [1]
>
> https://download.copr.fedorainfracloud.org/results/hobbes1069/IIoT/fedora-ra...
> > This log no longer seems to exist.
>
> I was able to access it just now.
> Some relevant lines are:
>
> [ 18%] Building C object CMakeFiles/mqttc.dir/src/mqtt.c.o
> /usr/bin/gcc -DMQTT_USE_BIO -I/builddir/build/BUILD/MQTT-C-1.1.5/include
> -O2 -flto=auto -ffat-lto-objects -fexceptions \
> -g -grecord-gcc-switches -pipe -Wall -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS \
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1    \
> -m64  -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection
> -fcf-protection -fPIC -pie -MD -MT \
> CMakeFiles/mqttc.dir/src/mqtt.c.o -MF CMakeFiles/mqttc.dir/src/mqtt.c.o.d
> -o CMakeFiles/mqttc.dir/src/mqtt.c.o \
> -c /builddir/build/BUILD/MQTT-C-1.1.5/src/mqtt.c
> [ 27%] Linking C static library libmqttc.a
> /usr/bin/cmake -P CMakeFiles/mqttc.dir/cmake_clean_target.cmake
> /usr/bin/cmake -E cmake_link_script CMakeFiles/mqttc.dir/link.txt
> --verbose=1
> /usr/bin/ar qc libmqttc.a CMakeFiles/mqttc.dir/src/mqtt_pal.c.o
> CMakeFiles/mqttc.dir/src/mqtt.c.o
> /usr/bin/ranlib libmqttc.a
> =====
> which confirms that "-fPIC -pie" was used when compiling mqtt.c into
> CMakeFiles/mqttc.dir/src/mqtt.c.o .

I misread the hardening page but I got the same error without -fPIC -pie...
> Suggestion: extract mqtt.c.o from libmqttc.a, then run "readelf --all
> --wide mqtt.c.o  > foo"
> and look in file foo for more information about:
>     relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules'

I'll take a look, but this is one place where building in mock sucks... I
can shell in to the chroot but not everything "works" exactly the same,
especially vim, which I have to manually install :)

Also, upstream should remedy complaints from the compiler:
> =====
> /builddir/build/BUILD/MQTT-C-1.1.5/examples/bio_publisher.c: In function
> 'main':
> /builddir/build/BUILD/MQTT-C-1.1.5/examples/bio_publisher.c:47:5: warning:
> 'ERR_load_BIO_strings' is deprecated: \
> Since OpenSSL 3.0 [-Wdeprecated-declarations]
>     47 |     ERR_load_BIO_strings();
>        |     ^~~~~~~~~~~~~~~~~~~~
> In file included from /usr/include/openssl/cryptoerr.h:17,
>                   from /usr/include/openssl/crypto.h:38,
>                   from /usr/include/openssl/bio.h:30,
>                   from
> /builddir/build/BUILD/MQTT-C-1.1.5/include/mqtt_pal.h:100,
>                   from
> /builddir/build/BUILD/MQTT-C-1.1.5/include/mqtt.h:43,
>                   from
> /builddir/build/BUILD/MQTT-C-1.1.5/examples/bio_publisher.c:10:
> /usr/include/openssl/cryptoerr_legacy.h:31:27: note: declared here
>     31 | OSSL_DEPRECATEDIN_3_0 int ERR_load_BIO_strings(void);
>        |                           ^~~~~~~~~~~~~~~~~~~~
> =====
> and:
> =====
> /builddir/build/BUILD/MQTT-C-1.1.5/examples/simple_subscriber.c: In
> function 'main':
> /builddir/build/BUILD/MQTT-C-1.1.5/examples/simple_subscriber.c:73:24:
> warning: passing argument 2 of 'mqtt_init' makes pointer from integer
> without a cast [-Wint-conversion]
>     73 |     mqtt_init(&client, sockfd, sendbuf, sizeof(sendbuf), recvbuf,
> sizeof(recvbuf), publish_callback);
>        |                        ^~~~~~
>        |                        |
>        |                        int
> =====
> plus several more int vs pointer conflicts.

Yes, I'm surprised I ran into so many because they run with -Werror but
they are probably using a much older gcc.

Thanks,
Richard

[Florian Weimer]

* Richard Shaw:
> I misread the hardening page but I got the same error without -fPIC -pie...

Ahem, you need to compile with -fPIC if you intend to link with -shared.

Thanks,
Florian

Mock tips

[Vít Ondruch]

Dne 10. 05. 22 v 13:32 Richard Shaw napsal(a):
> On Tue, May 10, 2022 at 3:09 AM John Reiser  wrote:
> Suggestion: extract mqtt.c.o from libmqttc.a, then run "readelf
> --all --wide mqtt.c.o  > foo"
> and look in file foo for more information about:
>     relocation R_X86_64_PC32 against symbol `mqtt_fixed_header_rules'
>
>
> I'll take a look, but this is one place where building in mock 
> sucks... I can shell in to the chroot but not everything "works" 
> exactly the same, especially vim, which I have to manually install :)

You can use your local VIM outside of the mock to edit some file inside 
mock, e.g.:

~~~

$ vim 
/var/lib/mock/fedora-rawhide-x86_64/root/builddir/build/BUILD/your-project-1.2.3/your-file

~~~

You don't even need root privileges editing files in the /builddir 
directory.

BTW sometimes it might also be useful to bind mount your local directory 
into mock. There is even mock plugin which makes it automatic for 
specific config file. You need to add there something like:

~~~

config_opts['plugin_conf']['bind_mount_enable'] = True
config_opts['plugin_conf']['bind_mount_opts']['dirs'].append(('/home/vondruch/projects/your-project', 
'/your-project/' ))

~~~

Vít

[Florian Weimer]

* Richard Shaw:
> I added the following to the libmqttc library and verified -fPIC -pie
> is in the build flags[1] per the recommendation from the hardening
> page[2] but the error remains.

Code that is linked into a shared object (with -shared) must be compiled
as PIC, not PIE.

Thanks,
Florian

[John Reiser]

* Florian Weimer:
> Richard Shaw:
> > I added the following to the libmqttc library and verified -fPIC -pie
> > is in the build flags[1] per the recommendation from the hardening
> > page[2] but the error remains.
>
> Code that is linked into a shared object (with -shared) must be compiled
> as PIC, not PIE.

So using "-fPIC -pie" should elicit a warning from the compiler, something like:

warning: '-pie' turns off '-fPIC'

with an analogous warning whenever a command-line parameter conflicts with
an earlier command-line parameter.