Switching to DWARF5 default for GCC11 (and the default Fedora 34 toolchain)

[Mark Wielaard]

Hi Neal,

On Tue, 2020-09-29 at 19:59 -0400, Neal Gompa wrote:
>  For the record, Mark has started implementing DWARF-5 support in
> dwz:
>  https://sourceware.org/git/?p=dwz.git;a=log
>  
>  I think I would rather like to see a Change proposal to switch to
>  DWARF-5 for Fedora 34, especially since it looks like dwz will be
>  ready for it. 
That is indeed my goal, but I wasn't planning on filing a specific
Change Proposal for it.

First because as you observed in the past we did some of these
debuginfo things Fedora first and then it took years (!) for some of
the default settings we had changed and helper scripts to make it
upstream. So I am concentrating on getting everything ready upstream
first before making and proposing any changes for Fedora. 

Secondly I am hoping that because of the first point the GCC11 for
Fedora 34 Change Proposal will simply say "-gdwarf-5 is now the
default".

Lastly, and sadly, I find the whole Fedora change proposal debates
extremely hostile. They often seem to quickly result in people
attacking you because you made a choice to spend time to work on A and
not their favorite feature B, and if they cannot have feature B then
you should also not spend any more time on A.

So I am happy to describe the work I am doing to try to get DWARF5 the
default for GCC11 and by extension for the Fedora 34 default toolchain,
but I will mainly do that work upstream and then see whether it is all
ready on time to enable it for Fedora 34. But I am not interested in a
heated debate on how I should prioritize my time and energy.

= Why DWARF5 for GCC?

- A couple of new tags and attributes make it easier/more accurate to
  describe some of the newer language features (although most were
  already covered by various GNU extensions)

- A lot of GNU extensions to DWARF4 have been standardized in DWARF5.
  By adopting the standardized variant alternative toolchains will
  hopefully find it easier to support these features.

- The representation of various data structures in DWARF5 is much more
  efficient causing a 25% on-disk size reduction (before any other
  compression method) for the .debug sections:
  https://gcc.gnu.org/pipermail/gcc-patches/2020-September/553527.html

= DWARF5 for the (extended) GNU toolchain

- binutils (gas) is responsible for turning part of the assembly
  produced by the compiler into a line table (.debug_line) and the
  linker sometimes reads parts of the DWARF (for example
  when producing warnings about where a symbol was defined). The just
  released binutils 2.35.1 should have all fixes necessary to support
  DWARF5.

- gcc needs to use the new gas features. Jakub has a patch (not
  committed yet):
  https://gcc.gnu.org/pipermail/gcc-patches/2020-September/553992.html

- gdb seems ready except for one corner case with C++ static member
  variables in classes. This is because in DWARF5 these are represented
  not as variables, which might be optimized away when not used. In
  this case gcc probably shouldn't optimize out the unused variables
  (or gdb should not depend on being able to show optimized out static
  member variables). Ongoing debate how to resolve this:
  https://sourceware.org/bugzilla/show_bug.cgi?id=26525
  https://gcc.gnu.org/pipermail/gcc-patches/2020-September/553102.html

- The just released elfutils 0.181 seems to have all needed support,
  which should cover systemtap, dwarves, perf, systemd, libabigail.
  More testing ongoing.

- For valgrind I initially wanted to switch the DWARF reader to an
  external helper program based on elfutils libdw. But to get a
  solution faster I will tweak the internal reader to deal with just
  the minimal DWARF5 as generated by gcc for now. I haven't started on
  this yet.

= DWARF5 for the (Fedora) packaging tools

- rpm debugedit has patches to support DWARF5 but we have to make sure
  they have testcases to work with gcc:
  http://lists.rpm.org/pipermail/rpm-maint/2020-August/014833.html

- dwz is seeing active work towards supporting DWARF5:
  https://sourceware.org/pipermail/dwz/2020q3/000668.html
  I am hoping that by end of next week we have generic support.
  That might not do optimal compression yet and will probably need lots
  of testing (and bug fixing).

= What I am NOT working on

- We'll keep using .gdb_index for now, moving to .debug_names only
  when that is ready in gdb:
  https://sourceware.org/pipermail/gdb/2020-September/048879.html

- Optional DWARF5 features like debug-types, forms, operations or index
  tables only used for split-dwarf by GCC (e.g. DW_FORM_strx,
  DW_FORM_addrx, DW_FORM_loclistx, DW_FORM_rnglistx, DW_OP_addrx,
  DW_OP_constx).

- Any other tool, project not mentioned above or other
  native toolchains like golang, rust, clang/llvm or ocaml.
  I expect those to simply keep producing DWARF4.

That doesn't mean I won't help with any of the above if others propose
to do that work for the various pieces of the toolchain and packaging,
but I currently don't have time for it and I don't think it is
realistic for the Fedora 34 timeframe.

= Timeline

- I am hoping that all upstream work is ready by end of next month
  (end of October, start of November). Then we can backport any patches
  into Fedora for project which haven't had a release yet and start
  integration testing once GCC11 drops in rawhide (assuming GCC
  upstream defaults to DWARF5).