Re: report on the "ELF package notes" status

[Luca Boccassi]

On Sun, 2022-02-13 at 20:05 +0100, Zbigniew Jędrzejewski-Szmek wrote:
>  On Fri, Feb 11, 2022 at 08:01:45PM +0100, Zbigniew
> Jędrzejewski-Szmek
>  wrote:
>  > Hi!
>  > 
>  > I did some quick'n'dirty statistics of how many ELF files
>  > successfully gained
>  > a .package.note section.
>  > 
>  > packages: 28742, see [1] for the list (*)
>  > ELF files: 72464, see [2]
>  > ELF files with .package.note: 47939, see [3]
>  > ELF files without: 24525, see [4]
>  > 
>  > It turns out that many of those are special files. I filtered
>  > out
>  > '\.(mod|o|h|cmxs|go|syso|c32|fas|wcx|wdx|dsx|wlx|wfx|out|dyn.*none)
>  > $' (**).
>  > Most of those seem irrelevant, in particular .o hasn't been linked,
>  > so it can't have the notes section…
>  > 
>  > ELF files after filtering: 15723, see [5]
>  > packages for those filtered files: 4659, see [6]
>  > 
>  > I did some quick analysis, and the reasons why the notes section is
>  > missing differ:
>  > - 460 packages don't have the .fc36 tag, so they weren't rebuilt
>  > - some packages I checked were built before the mass rebuild and
>  >   got the .fc36 disttag, but failed in the subsequent mass rebuild
>  > - opt-outs:
>  >   895 ghc packages
>  >   210+ ocaml packages
>  >   197 R packages
>  >   30 ruby packages
>  >   61 python packages
>  > 
>  > That still leaves quite a lot of packages without the notes, but I
>  > think
>  > more manual analysis would be necessary to figure out the reasons.
>  
>  Some other common failure modes:
>  - rust binaries (not rust-*-devel): it seems to be a general issue
>  with the cargo
>    build process.
>  
>    Manually inserting "-Clink-args=-Wl,-dT,…/.package_note-….ld" into
>  a rustc command
>    seems to work. So maybe it's just a matter of find the right flags
>  for cargo.
>  
>  - go binaries: same.
>  
>    E.g. for 3mux I see:
>  
>    + go build -buildmode pie -compiler gc '-tags=rpm_crashtraceback '
>  -ldflags ' -X github.com/aaronjanse/3mux/version=1.0.1 -B
>  0x383b831c21c2e07438685900d5a662a268d4e0ec -compressdwarf=false -
>  extldflags '\''-Wl,-z,relro -Wl,--as-needed  -Wl,-z,now -
>  specs=/usr/lib/rpm/redhat/redhat-hardened-ld -
>  specs=/usr/lib/rpm/redhat/redhat-annobin-cc1  -Wl,--build-id=sha1 -
>  Wl,-dT,/builddir/build/BUILD/3mux-1.0.1/.package_note-3mux-1.0.1-
>  4.fc36.x86_64.ld '\''' -a -v -x -o /builddir/build/BUILD/3mux-
>  1.0.1/_build/bin/3mux github.com/aaronjanse/3mux
>  
>    but 'readelf -n -W /usr/bin/3mux only' shows
>    .note.gnu.build-id and .note.go.buildid.
>  
>  I actually think it'd be useful to fix the note additions for rust
>  and
>  go.  Both rust and go binaries are fairly portable between different
>  systems, and the notes would help identify such binaries.
>  
>  I'll see if I can figure out rust… I don't know anything about go, so
>  any help would welcome.
>  
>  Zbyszek 
Hi,

For golang, a solution that works (but I am not a golang expert, so I
do not know whether it is the right solution in general) is to add the
'-linkmode=external' flag to the go linking step. As far as I
understand, this forces to use the default distro linker instead of the
internal one.

With this change to the rpm script:

--- a/macros.go-compilers-golang
+++ b/macros.go-compilers-golang
@@ -29,7 +29,7 @@
 #
 #    %make GOBUILDFLAGS="%gobuildflags"
 #
-%gobuildflags() %{expand:%{gocompilerflags} -tags="rpm_crashtraceback
${BUILDTAGS:-}" -ldflags "${LDFLAGS:-} %{?currentgoldflags} -B 0x$(head -c20
/dev/urandom|od -An -tx1|tr -d ' \\n') -compressdwarf=false -extldflags
'%__global_ldflags %{?__golang_extldflags}'" -a -v -x}
+%gobuildflags() %{expand:%{gocompilerflags} -tags="rpm_crashtraceback
${BUILDTAGS:-}" -ldflags "${LDFLAGS:-} %{?currentgoldflags} -B 0x$(head -c20
/dev/urandom|od -An -tx1|tr -d ' \\n') -compressdwarf=false -linkmode=external
-extldflags '%__global_ldflags %{?__golang_extldflags}'" -a -v -x}
 
 # Turn off Go modules
 %gomodulesmode GO111MODULE=off


A local rebuild of the 3mux package you cited shows the note:

$ readelf --notes -W rpmbuild/BUILDROOT/3mux-1.0.1-4.fc36.x86_64/usr/bin/3mux 

Displaying notes found in: .note.gnu.property
  Owner                Data size 	Description
  GNU                  0x00000010	NT_GNU_PROPERTY_TYPE_0	      Properties: x86 ISA needed:
x86-64-baseline

Displaying notes found in: .note.gnu.build-id
  Owner                Data size 	Description
  GNU                  0x00000014	NT_GNU_BUILD_ID (unique build ID bitstring)	    Build
ID: d95f4a446c52232a5f50d7cae20630fc358b5baf

Displaying notes found in: .note.package
  Owner                Data size 	Description
  FDO                  0x00000078	FDO_PACKAGING_METADATA	    Packaging Metadata:
{"type":"rpm","name":"3mux","version":"1.0.1-4.fc36","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:36"}

Displaying notes found in: .note.ABI-tag
  Owner                Data size 	Description
  GNU                  0x00000010	NT_GNU_ABI_TAG (ABI version tag)	    OS: Linux, ABI:
3.2.0

Displaying notes found in: .note.go.buildid
  Owner                Data size 	Description
  Go                   0x00000053	GO BUILDID	   description data: 62 56 61 67 68 7a 68 5f
53 6e 72 77 70 51 48 49 73 6f 36 41 2f 4a 35 53 4b 51 55 4b 46 59 4c 43 48 41 52 71 61 7a
6d 6c 73 2f 30 73 49 6b 69 77 39 62 6b 32 31 4f 75 7a 39 69 50 77 56 4d 2f 5f 39 74 55 41
36 31 6c 36 37 75 42 57 67 74 6a 2d 48 66 38 

Displaying notes found in: .gnu.build.attributes
  Owner                Data size 	Description
  GA$<version>3a1              0x00000010	OPEN	    Applies to region from 0x4f000 to
0x4f396
  GA$<version>3a1              0x00000010	OPEN	    Applies to region from 0x4f3a0 to
0x1ceb21

-- 
Kind regards,
Luca Boccassi