multi-CPU optimization inside a distribution
Hi all,
we were talking about this item for some time, so let's start a thread
for it to have the discussion and hopefully also a solution documented.
This is meant as discussion initiation based on the situation in
Fedora on POWER. I would like to ask the bigger experts than me to fill
the missing details and options.
Currently we set a minimal CPU level for an architecture (or use the
toolchain default) on the distribution level
(/usr/lib/rpm/redhat/rpmpc owned by redhat-rpm-config [1]). It allows
the distro to run on such CPU and any newer evolution of it (omitting
any kernel or hw issues), but it also means it doesn't generally take
advantage of features and improvements in the newer CPUs.
For ppc64 (the big endian POWER) the base is set by the toolchain
default which is Power4/ppc970. When Power7 came we were asked what are
the options to take the advantage of these CPUs, 3 generations newer
than the base. The solution was the introduction of ppc64p7 subarch into
the packaging and release engineering tools. But it showed more as a
hack than a solution, touching rpm, yum, koji, .... The list of packages
is manually maintained, requires manual updates to the buildsystem for
new releases, seems new packaging tools (dnf) don't understand it
correctly. Is there a way to make the subarch mechanism generic and
integrated with the other tools? So we could have ppc64p8 and ppc64p9
inside Fedora for POWER ...
Now I'm getting to an area where others are the experts :-) Glibc
allows to build and install multiple per-cpu optimized runtimes that
are selected based on the CPU. There is the IFUNC mechanism and
function multi-versioning in GCC (user-friendly IFUNC) [2] to allow
multiple implementations inside one library/binary.
Some packages do the CPU detection during runtime and select the
optimized functions themselves.
There is also an option to introduce a "tertiary architecture" and
rebuild everything for the new CPU, but keeping the rpm arch the same.
But it has its own costs too.
What do you think? Are there any recommendations for both the distro
and package maintainers and upstream developers? I suppose even primary
architectures are facing the same problem.
[1] http://pkgs.fedoraproject.org/cgit/rpms/redhat-rpm-config.git/tree/rpmrc
[2] https://lwn.net/Articles/691932/
Dan