[OS-BUILD PATCHv3 0/3] redhat/kernel.spec.template: Add global compression variables

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173

Most of the kernel and its packing is currently xz.  There are two
exceptions: the aarch64 boot an initrd images [1] and KABI.  Mark gzip as
only being necessary for aarch64, and change the KABI symbol compression
to xz.

There are other compression algorithms that are becoming more popular.  To
prepare for this change,  add a compression variable and compression file
extension variable.

Signed-off-by: Prarit Bhargava

[1] There is upstream work on implementing xz and other compression
algorithms for aarch64.  This work should be completed in the next few
upstream kernel and bootloader releases.

---
 redhat/kernel.spec.template |  40 +++++++++++++++++++++-------------------
 1 files changed, 21 insertions(+), 19 deletions(-)

[OS-BUILD PATCHv3 1/3] kernel.spec.template: Only use gzip with aarch64

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

kernel.spec.template: Only use gzip with aarch64

gzip is only needed for aarch64.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -578,7 +578,7 @@ Requires: kernel-modules-uname-r = %{KVERREL}
 # List the packages used during the kernel build
 #
 BuildRequires: kmod, bash, coreutils, tar, git-core, which
-BuildRequires: bzip2, xz, findutils, gzip, m4, perl-interpreter, perl-Carp, perl-devel, perl-generators, make, diffutils, gawk
+BuildRequires: bzip2, xz, findutils, m4, perl-interpreter, perl-Carp, perl-devel, perl-generators, make, diffutils, gawk
 BuildRequires: gcc, binutils, redhat-rpm-config, hmaccalc, bison, flex, gcc-c++
 BuildRequires: net-tools, hostname, bc, elfutils-devel
 BuildRequires: dwarves
@@ -612,6 +612,7 @@ BuildRequires: numactl-devel
 %endif
 %ifarch aarch64
 BuildRequires: opencsd-devel >= 1.0.0
+BuildRequires: gzip
 %endif
 %endif
 %if %{with_tools}
@@ -1712,14 +1713,11 @@ BuildKernel() {
         CopyKernel=cp
     fi

-    # Sign the image if we're using EFI
-    # aarch64 kernels are gziped EFI images
-    KernelExtension=${KernelImage##*.}
-    if [ "$KernelExtension" == "gz" ]; then
+    SignImage=$KernelImage
+    %ifarch aarch64
+        # aarch64 kernels are gziped EFI images
         SignImage=${KernelImage%.*}
-    else
-        SignImage=$KernelImage
-    fi
+    %endif

%ifarch x86_64 aarch64
     %pesign -s -i $SignImage -o vmlinuz.tmp -a %{secureboot_ca_0} -c %{secureboot_key_0} -n %{pesign_name_0}
@@ -1742,9 +1740,9 @@ BuildKernel() {
         exit 1
     fi
     mv vmlinuz.signed $SignImage
-    if [ "$KernelExtension" == "gz" ]; then
+    %ifarch aarch64
         gzip -f9 $SignImage
-    fi
+    %endif
     # signkernel
     %endif

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173

[OS-BUILD PATCHv3 2/3] kernel.spec.template: Use xz for KABI

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

kernel.spec.template: Use xz for KABI

Use xz for KABI instead of gzip.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -478,6 +478,7 @@ Summary: The Linux kernel
 %define all_arch_configs kernel-%{version}-aarch64*.config
 %define asmarch arm64
 %define hdrarch arm64
+# only aarch64 uses gzip compression for boot images
 %define make_target Image.gz
 %define kernel_image arch/arm64/boot/Image.gz
 %endif
@@ -1819,8 +1820,8 @@ BuildKernel() {
     # NOTENOTE: checksums to the rpm metadata provides list.
     # NOTENOTE: if you change the symvers name, update the backend too
     echo "**** GENERATING kernel ABI metadata ****"
-    gzip -c9 < Module.symvers > $RPM_BUILD_ROOT/boot/symvers-$KernelVer.gz
-    cp $RPM_BUILD_ROOT/boot/symvers-$KernelVer.gz $RPM_BUILD_ROOT/lib/modules/$KernelVer/symvers.gz
+    xz -c9 < Module.symvers > $RPM_BUILD_ROOT/boot/symvers-$KernelVer.xz
+    cp $RPM_BUILD_ROOT/boot/symvers-$KernelVer.xz $RPM_BUILD_ROOT/lib/modules/$KernelVer/symvers.xz

%if %{with_kabichk}
     echo "**** kABI checking is enabled in kernel SPEC file. ****"
@@ -2848,9 +2849,9 @@ fi\
 %endif\
 rm -f %{_localstatedir}/lib/rpm-state/%{name}/installing_core_%{KVERREL}%{?1:+%{1}}\
 /bin/kernel-install add %{KVERREL}%{?1:+%{1}} /lib/modules/%{KVERREL}%{?1:+%{1}}/vmlinuz || exit $?\
-if [[ ! -e "/boot/symvers-%{KVERREL}%{?1:+%{1}}.gz" ]]; then\
-    ln -s "/lib/modules/%{KVERREL}%{?1:+%{1}}/symvers.gz" "/boot/symvers-%{KVERREL}%{?1:+%{1}}.gz"\
-    command -v restorecon &>/dev/null && restorecon "/boot/symvers-%{KVERREL}%{?1:+%{1}}.gz" \
+if [[ ! -e "/boot/symvers-%{KVERREL}%{?1:+%{1}}.xz" ]]; then\
+    ln -s "/lib/modules/%{KVERREL}%{?1:+%{1}}/symvers.xz" "/boot/symvers-%{KVERREL}%{?1:+%{1}}.xz"\
+    command -v restorecon &>/dev/null && restorecon "/boot/symvers-%{KVERREL}%{?1:+%{1}}.xz" \
 fi\
 %{nil}

@@ -3105,9 +3106,9 @@ fi
 %endif\
 %attr(0600, root, root) /lib/modules/%{KVERREL}%{?3:+%{3}}/System.map\
 %ghost %attr(0600, root, root) /boot/System.map-%{KVERREL}%{?3:+%{3}}\
-/lib/modules/%{KVERREL}%{?3:+%{3}}/symvers.gz\
+/lib/modules/%{KVERREL}%{?3:+%{3}}/symvers.xz\
 /lib/modules/%{KVERREL}%{?3:+%{3}}/config\
-%ghost %attr(0600, root, root) /boot/symvers-%{KVERREL}%{?3:+%{3}}.gz\
+%ghost %attr(0600, root, root) /boot/symvers-%{KVERREL}%{?3:+%{3}}.xz\
 %ghost %attr(0600, root, root) /boot/initramfs-%{KVERREL}%{?3:+%{3}}.img\
 %ghost %attr(0644, root, root) /boot/config-%{KVERREL}%{?3:+%{3}}\
 %dir /lib/modules\

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173

[OS-BUILD PATCHv3 3/3] kernel.spec.template: Add global compression variables

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

kernel.spec.template: Add global compression variables

Add global compression variables.  'compression' is the compression
utility, and 'compext' is the compression extension.  In the case of xz,
these variables have the same value but that may not always be the
case.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -91,8 +91,11 @@ Summary: The Linux kernel
 %global zipmodules 1
 %endif

+# Default compression algorithm
+%global compression xz
+%global compext xz
 %if %{zipmodules}
-%global zipsed -e 's/.ko$/.ko.xz/'
+%global zipsed -e 's/.ko$/.ko.%compext/'
 %endif

%if 0%{?fedora}
@@ -1820,8 +1823,8 @@ BuildKernel() {
     # NOTENOTE: checksums to the rpm metadata provides list.
     # NOTENOTE: if you change the symvers name, update the backend too
     echo "**** GENERATING kernel ABI metadata ****"
-    xz -c9 < Module.symvers > $RPM_BUILD_ROOT/boot/symvers-$KernelVer.xz
-    cp $RPM_BUILD_ROOT/boot/symvers-$KernelVer.xz $RPM_BUILD_ROOT/lib/modules/$KernelVer/symvers.xz
+    %compression -c9 < Module.symvers > $RPM_BUILD_ROOT/boot/symvers-$KernelVer.%compext
+    cp $RPM_BUILD_ROOT/boot/symvers-$KernelVer.%compext $RPM_BUILD_ROOT/lib/modules/$KernelVer/symvers.%compext

%if %{with_kabichk}
     echo "**** kABI checking is enabled in kernel SPEC file. ****"
@@ -2439,7 +2442,7 @@ find Documentation -type d | xargs chmod u+w
     fi \
   fi \
   if [ "%{zipmodules}" -eq "1" ]; then \
-    find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r xz; \
+    find $RPM_BUILD_ROOT/lib/modules/ -type f -name '*.ko' | xargs -P${RPM_BUILD_NCPUS} -r %compression; \
   fi \
 %{nil}

@@ -2849,9 +2852,9 @@ fi\
 %endif\
 rm -f %{_localstatedir}/lib/rpm-state/%{name}/installing_core_%{KVERREL}%{?1:+%{1}}\
 /bin/kernel-install add %{KVERREL}%{?1:+%{1}} /lib/modules/%{KVERREL}%{?1:+%{1}}/vmlinuz || exit $?\
-if [[ ! -e "/boot/symvers-%{KVERREL}%{?1:+%{1}}.xz" ]]; then\
-    ln -s "/lib/modules/%{KVERREL}%{?1:+%{1}}/symvers.xz" "/boot/symvers-%{KVERREL}%{?1:+%{1}}.xz"\
-    command -v restorecon &>/dev/null && restorecon "/boot/symvers-%{KVERREL}%{?1:+%{1}}.xz" \
+if [[ ! -e "/boot/symvers-%{KVERREL}%{?1:+%{1}}.%compext" ]]; then\
+    ln -s "/lib/modules/%{KVERREL}%{?1:+%{1}}/symvers.%compext" "/boot/symvers-%{KVERREL}%{?1:+%{1}}.%compext"\
+    command -v restorecon &>/dev/null && restorecon "/boot/symvers-%{KVERREL}%{?1:+%{1}}.%compext" \
 fi\
 %{nil}

@@ -3106,9 +3109,9 @@ fi
 %endif\
 %attr(0600, root, root) /lib/modules/%{KVERREL}%{?3:+%{3}}/System.map\
 %ghost %attr(0600, root, root) /boot/System.map-%{KVERREL}%{?3:+%{3}}\
-/lib/modules/%{KVERREL}%{?3:+%{3}}/symvers.xz\
+/lib/modules/%{KVERREL}%{?3:+%{3}}/symvers.%compext\
 /lib/modules/%{KVERREL}%{?3:+%{3}}/config\
-%ghost %attr(0600, root, root) /boot/symvers-%{KVERREL}%{?3:+%{3}}.xz\
+%ghost %attr(0600, root, root) /boot/symvers-%{KVERREL}%{?3:+%{3}}.%compext\
 %ghost %attr(0600, root, root) /boot/initramfs-%{KVERREL}%{?3:+%{3}}.img\
 %ghost %attr(0644, root, root) /boot/config-%{KVERREL}%{?3:+%{3}}\
 %dir /lib/modules\

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173

[Patrick Talbert (via Email Bridge)]

From: Patrick Talbert on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1212511...

I think this is merged now?

[Justin M. Forbes (via Email Bridge)]

From: Justin M. Forbes on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1212551...

But is it in Fedora 36 - Rawhide built binaries?

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1216406...
> But is it in Fedora 36 - Rawhide built binaries?

Just checked.  It is NOT in F36 or rawhide yet.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1232441...

This is in rawhide now AFAICT.
https://koji.fedoraproject.org/koji/buildinfo?buildID=2103281

I will ping rharwood to see if we can get this into F36.

[John B. Wyatt IV (via Email Bridge)]

From: John B. Wyatt IV on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1261397...

I just checked my F36 machine. I do not see any code in 20-grub.install that
handles .xz compressed symvers compared to gz and bz.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1265045...

@jwyatt-rh the change was to deal with bz, not xz.  xz is only used in the
modules.  The bz change here is to jump from gz.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1265046...

It looks like this change is in F36 to F38.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2173#note_1265047...

Oops.  Forgot this link https://src.fedoraproject.org/rpms/grub2