Re: F34 Change proposal: Debug Info Standardization (from DWZ to -fdebug-types-section) (System-Wide Change proposal)
On Thu, 24 Sep 2020 21:20:38 +0200, Dominique Martinet wrote:
Ben Cotton wrote on Thu, Sep 24, 2020:
> ** If the 3.3% size increase is a concern I can implement a different
> optimization ([https://whova.com/embedded/session/llvm_202010/1193947/
> talk (2)]) as a GCC post-processing phase which would require no
> changes in any DWARF consumers.
That talk doesn't load for me, sorry if I ask something answered in
there.
I have added a title there now but the URL loads for me even in lynx+wget.
Copy-pasted it at the bottom of this mail. I do not know the talk but TL;DR
existing DWARF contains some dead DIEs - unused/deduplicated functions and also
-fdebug-types-section declarations/skeletons which can be removed or converted
to direct DIE references respectively. That way one could reduce the size like
DWZ does but without needing any new complicated support in DWARF consumers.
How does this relate to debuginfo compression such as passing
-Wl,--compress-debug-sections=zlib ?
That is orthogonal - that is one can add it to DWZ or -fdebug-types-section
the same way. It would be for another Fedora Change proposal but I do not
think it matters for F-33 as it already implements:
https://fedoraproject.org/wiki/Changes/BtrfsByDefault#Compression
I haven't yet checked whether that applies to /usr/lib/debug/ by default.
btrfs is using zstd which has better performance than zlib. I was considering
adding an ELF section compression extension for zstd but with btrfs
transparent compression that looks as not useful.
Some people have concern about performance of debuggers/tools loading such
compressed *.debug files, my benchmarks show only at most 10% performance hit.
I have calculated for Fedora Rawhide -Wl,--compress-debug-sections=zlib saves
52.84% of on-disk *.debug size. But that does not apply to *-debuginfo.rpm as
rpms are already compressed.
That 3.3% size reduction=advantage of DWZ against -fdebug-types-section is
calculated for *-debuginfo.rpm (3.3% is for the whole distribution incl.
binaries, for debug/ itself it is 6.35%). Also it is calculated for DWARF-4,
F-34 will hopefully switch to DWARF-5 (which is smaller by 10-20%) but DWZ is
not yet ported to DWARF-5 so it is impossible to compare -fdebug-types-section
vs. DWZ size for DWARF-5.
As far as I can see, gdb and lldb both support it just fine; and the
linux kernel builds with it if CONFIG_DEBUG_INFO_COMPRESSED is set so it
looks widespread enough.
Yes, zlib ELF section compression is well supported.
Jan
https://whova.com/embedded/session/llvm_202010/1193947/
2) Fragmenting the DWARF to Enable Dead Debug Data Elimination
Speaker: James Henderson
Standard DWARF defines a series of sections in the output, with one of each
per object file. Each of these sections may have information about every
function and variable in that unit. Linkers typically leave this information
intact, referencing address 0 (or other tombstone value) when a function or
variable���������s section is discarded, as the debug sections still contain used
information. This approach has issues such as potential ambiguity and
excessive space usage.
This talk will present a solution to these issues, leveraging existing ELF
features, which enables linkers to discard dead pieces of DWARF without the
linker requiring any special knowledge of its structure. It will also include
performance figures to evaluate the approach.