[OS-BUILD PATCHv17 0/2] redhat/kernel.spec: add uki_addons to create UKI kernel cmdline addons

[Emanuele Giuseppe Esposito (via Email Bridge)]

From: Emanuele Giuseppe Esposito on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917

We want to enable kernel.spec to optionally ship UKI addons defined in a
common config file in redhat folder.
The folder redhat/uki_addons will contain all addons configs specifying the
UKI kernel cmdline addons to be created in the next build. An addon
config is simply a .addon plain text file, where any line
is taken as kernel cmdline, except for the ones starting with '#',
which will be automatically ignored.

redhat/scripts/uki_addons.py will take care of parsing all configs and
folders in redhat/uki_addons and call 'ukify' to create the actual addons.
The output addon filename will be a concatenation of all folders in
redhat/uki_addons that are part of the addon config path.

The folder hierarchy inside of redhat/uki_addons is similar to
redhat/configs: $distro/$UKI_NAME/%arch.

It is also possible to add .sbat to all the generated addons, by
populating redhat/addons/$distro/$UKI_NAME/%arch/sbat/sbat.conf.
Syntax is same as the addons config.

At build time, Makefile will create a tar.gz archive (uki_addons.tar.gz)
containing all the files in redhat/uki_addons. It will then passed to the
kernel specfile that will extract the addons from it and generate the
UKI kernel cmdline addons.

As an example of this feature, add the fips addon to optionally enable fips
(https://issues.redhat.com/browse/RHEL-23049).

---
 redhat/scripts/uki_addons.py             |  162 +++++++++++++++++++++++++++++++
 redhat/uki_addons/virt/common/fips.addon |    1 +
 redhat/Makefile                          |    3 +
 redhat/kernel.spec.template              |   44 ++++++++
 4 files changed, 210 insertions(+), 0 deletions(-)

[OS-BUILD PATCHv17 1/2] redhat/kernel.spec: add uki_addons to create UKI kernel cmdline addons

[Emanuele Giuseppe Esposito (via Email Bridge)]

From: Emanuele Giuseppe Esposito

redhat/kernel.spec: add uki_addons to create UKI kernel cmdline addons

Upstream Status: RHEL-Only

The folder redhat/uki_addons will contain all addons configs specifying the
UKI kernel cmdline addons to be created in the next build. An addon
config is simply a .addon plain text file, where any line
is taken as kernel cmdline, except for the ones starting with '#',
which will be automatically ignored.

redhat/scripts/uki_addons.py will take care of parsing all configs and
folders in redhat/uki_addons and call 'ukify' to create the actual addons.
The output addon filename will be a concatenation of all folders in
redhat/uki_addons that are part of the addon config path.

The folder hierarchy inside of redhat/uki_addons is similar to
redhat/configs: $distro/$UKI_NAME/%arch.

It is also possible to add .sbat to all the generated addons, by
populating redhat/uki_addons/$distro/$UKI_NAME/%arch/sbat/sbat.conf.
Syntax is same as the addons config.

At build time, Makefile will create a tar.gz archive (uki_addons.tar.gz)
containing all the files in redhat/uki_addons. It will then passed to the
kernel specfile that will extract the addons from it and generate the
UKI kernel cmdline addons.

Signed-off-by: Emanuele Giuseppe Esposito

diff --git a/redhat/Makefile b/redhat/Makefile
index blahblah..blahblah 100644
--- a/redhat/Makefile
+++ b/redhat/Makefile
@@ -685,6 +685,7 @@ sources-rh: $(TARBALL) $(KABI_TARBALL) $(KABIDW_TARBALL) generate-testpatch-tmp
    @sed -e "s/%%SPECKVERSION%%/$(SPECKVERSION)/" \
    	-e "s/%%SPECKPATCHLEVEL%%/$(SPECKPATCHLEVEL)/" \
    	scripts/gating/rpminspect.yaml > $(SOURCES)/rpminspect.yaml
+	@tar -czf uki_addons.tar.gz uki_addons
    @cp scripts/kernel-tools/kvm_stat.logrotate \
    	keys/rhel*.x509 \
    	keys/nvidia*.x509 \
@@ -695,6 +696,7 @@ sources-rh: $(TARBALL) $(KABI_TARBALL) $(KABIDW_TARBALL) generate-testpatch-tmp
    	scripts/filtermods.py \
    	scripts/mod/mod-denylist.sh \
    	scripts/mod/mod-sign.sh \
+		scripts/uki_addons.py \
    	configs/flavors \
    	configs/generate_all_configs.sh \
    	configs/merge.py \
@@ -703,6 +705,7 @@ sources-rh: $(TARBALL) $(KABI_TARBALL) $(KABIDW_TARBALL) generate-testpatch-tmp
    	README.rst \
    	kernel-local \
    	dracut-virt.conf \
+		uki_addons.tar.gz \
    	$(SOURCES)/
    @cat $$(ls -1 $(SPECPACKAGE_NAME).changelog-* | sort -t '.' -k 3 -n -r) \
    	> $(SOURCES)/kernel.changelog
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100644
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -791,6 +791,8 @@ BuildRequires: binutils
 BuildRequires: lvm2
 BuildRequires: systemd-boot-unsigned
 # For systemd-stub and systemd-pcrphase
+BuildRequires: systemd-ukify
+# For UKI kernel cmdline addons
 BuildRequires: systemd-udev >= 252-1
 # For TPM operations in UKI initramfs
 BuildRequires: tpm2-tools
@@ -920,6 +922,9 @@ Source86: dracut-virt.conf

Source87: flavors

+Source151: uki_addons.py
+Source152: uki_addons.tar.gz
+
 Source100: rheldup3.x509
 Source101: rhelkpatch1.x509
 Source102: nvidiagpuoot001.x509
@@ -1540,6 +1545,11 @@ Provides: kernel-%{?1:%{1}-}uname-r = %{KVERREL}%{uname_suffix %{?1:+%{1}}}\
 Requires: kernel%{?1:-%{1}}-modules-core-uname-r = %{KVERREL}%{uname_suffix %{?1:+%{1}}}\
 Requires(pre): %{kernel_prereq}\
 Requires(pre): systemd >= 254-1\
+%package %{?1:%{1}-}uki-virt-addons\
+Summary: %{variant_summary} unified kernel image addons for virtual machines\
+Provides: installonlypkg(kernel)\
+Requires: kernel%{?1:-%{1}}-uki-virt = %{specrpmversion}-%{release}\
+Requires(pre): systemd >= 254-1\
 %endif\
 %endif\
 %if %{with_gcov}\
@@ -1679,31 +1689,49 @@ input and output, etc.
 %if %{with_up} && %{with_debug} && %{with_efiuki}
 %description debug-uki-virt
 Prebuilt debug unified kernel image for virtual machines.
+
+%description debug-uki-virt-addons
+Prebuilt debug unified kernel image addons for virtual machines.
 %endif

%if %{with_up_base} && %{with_efiuki}
 %description uki-virt
 Prebuilt default unified kernel image for virtual machines.
+
+%description uki-virt-addons
+Prebuilt default unified kernel image addons for virtual machines.
 %endif

%if %{with_arm64_16k} && %{with_debug} && %{with_efiuki}
 %description 16k-debug-uki-virt
 Prebuilt 16k debug unified kernel image for virtual machines.
+
+%description 16k-debug-uki-virt-addons
+Prebuilt 16k debug unified kernel image addons for virtual machines.
 %endif

%if %{with_arm64_16k_base} && %{with_efiuki}
 %description 16k-uki-virt
 Prebuilt 16k unified kernel image for virtual machines.
+
+%description 16k-uki-virt-addons
+Prebuilt 16k unified kernel image addons for virtual machines.
 %endif

%if %{with_arm64_64k} && %{with_debug} && %{with_efiuki}
 %description 64k-debug-uki-virt
 Prebuilt 64k debug unified kernel image for virtual machines.
+
+%description 64k-debug-uki-virt-addons
+Prebuilt 64k debug unified kernel image addons for virtual machines.
 %endif

%if %{with_arm64_64k_base} && %{with_efiuki}
 %description 64k-uki-virt
 Prebuilt 64k unified kernel image for virtual machines.
+
+%description 64k-uki-virt-addons
+Prebuilt 64k unified kernel image addons for virtual machines.
 %endif

%if %{with_ipaclones}
@@ -2552,6 +2580,12 @@ BuildKernel() {
            --kernel-cmdline 'console=tty0 console=ttyS0' \
       $KernelUnifiedImage

+  tar -xvf %{SOURCE152} -C $KernelUnifiedImageDir
+  KernelAddonsDir="$KernelUnifiedImageDir/uki_addons"
+  KernelAddonsDirOut="$RPM_BUILD_ROOT/usr/lib/linux/extra.d/$KernelVer"
+  mkdir -p $KernelAddonsDirOut
+  python3 %{SOURCE151} $KernelAddonsDir $KernelAddonsDirOut virt %{primary_target} %{_target_cpu}
+
 %if %{signkernel}
    %{log_msg "Sign the EFI UKI kernel"}
    %pesign -s -i $KernelUnifiedImage -o $KernelUnifiedImage.tmp -a %{secureboot_ca_0} -c %{secureboot_key_0} -n %{pesign_name_0}
@@ -2564,9 +2598,17 @@ BuildKernel() {
     	fi
     	mv $KernelUnifiedImage.signed $KernelUnifiedImage

+      for addon in "$KernelAddonsDirOut"/*; do
+        %pesign -s -i $addon -o $addon.signed -a %{secureboot_ca_1} -c %{secureboot_key_1} -n %{pesign_name_1}
+        rm -f $addon
+        mv $addon.signed $addon
+      done
+
 # signkernel
 %endif

+  rm -rf $KernelUnifiedImageDir/uki_addons
+

# with_efiuki
 %endif
@@ -3861,6 +3903,8 @@ fi\
 /lib/modules/%{KVERREL}%{?3:+%{3}}/modules.builtin*\
 %attr(0644, root, root) /lib/modules/%{KVERREL}%{?3:+%{3}}/%{?-k:%{-k*}}%{!?-k:vmlinuz}-virt.efi\
 %ghost /%{image_install_path}/efi/EFI/Linux/%{?-k:%{-k*}}%{!?-k:*}-%{KVERREL}%{?3:+%{3}}.efi\
+%{expand:%%files %{?3:%{3}-}uki-virt-addons}\
+/usr/lib/linux/extra.d/%{KVERREL}%{?3:+%{3}}/*.addon.efi\
 %endif\
 %endif\
 %if %{?3:1} %{!?3:0}\
diff --git a/redhat/scripts/uki_addons.py b/redhat/scripts/uki_addons.py
new file mode 100644
index blahblah..blahblah 100644
--- /dev/null
+++ b/redhat/scripts/uki_addons.py
@@ -0,0 +1,162 @@
+#!/usr/bin/env python3
+#
+# This script recursively inspects a given uki directory, and creates an addon
+# for each file found.
+#
+# Usage: python uki_addons.py input_dir out_dir uki distro arch
+#
+# This tool requires the systemd-ukify and systemd-boot packages.
+#
+# Addon file
+#-----------
+# Each addon terminates with .addon
+# Each addon contains only two types of lines:
+# Lines beginning with '#' are description and thus ignored
+# All other lines are command line to be added.
+# The name of the end resulting addon is taken from the folder hierarchy.
+# For example, and addon in virt/rhel/x86_64/hello.addon will result in
+# an UKI addon file generated in out_dir called hello-virt.rhel.x86_64.addon.efi
+#
+# The common folder, present in any folder in redhat/uki_addons (except the leaf folders)
+# is used as place for default addons when the same addon is not defined deep
+# in the hierarchy. For example, if we define test.addon (text: test1) in
+# redhat/uki_addons/common and another test.addon (text: test2) in
+# redhat/uki_addons/virt/common, any other uki except virt will have a test.addon.efi
+# with text "test1", and virt will have a test.addon.efi with "test2"
+#
+# sbat.conf
+#----------
+# This file is containing the sbat string for *all* addons being created.
+# This file is optional, but when used has to be put in out_dir/sbat/sbat.conf.
+# It follows the same syntax as the addon files, meaning '#' is comment and
+# the rest is taken as sbat string and feed to ukify.
+#
+#
+# How to extend the script and kernel.spec with a new arch or uki or distro
+#--------------------------------------------------------------------------
+# A new distro has to be added by creating the folder in redhat/uki_addons.
+# See above to how the directory hierarchy in redhat/uki_addons is expected to be.
+# After that, if the distro is a different arch from the one already supported,
+# one needs to extend the %define with_efiuki in kernel.spec.template.
+# If a new UKI has to be created with a different name from the existing ones,
+# the logic to create the addons and call this script has to be implemented too
+# in kernel.spec.template. As an example, see how the 'virt' UKI addons are
+# created.
+
+import os
+import sys
+import collections
+import subprocess
+
+
+UKIFY_PATH = '/usr/lib/systemd/ukify'
+SBAT_RELATIVE_PATH = 'sbat/sbat.conf'
+
+def usage(err):
+    print(f'Usage: {os.path.basename(__file__)} input_dir output_dir uki distro arch')
+    print(f'Error:{err}')
+    sys.exit(1)
+
+def check_clean_arguments(in_dir, out_dir):
+    # Remove end '/'
+    if in_dir[-1:] == '/':
+        in_dir = in_dir[:-1]
+    if out_dir[-1:] == '/':
+        out_dir = out_dir[:-1]
+    if not os.path.isdir(in_dir):
+        usage(f'in_dir {in_dir} is not a dir, or does not exist!')
+    if not os.path.isdir(out_dir):
+        usage(f'out_dir_dir {out_dir} is not a dir, or does not exist!')
+    return in_dir, out_dir
+
+UKICmdlineAddon = collections.namedtuple('UKICmdlineAddon', ['name', 'cmdline'])
+uki_addons_list = []
+uki_addons = {}
+addon_sbat_string = None
+
+def parse_addon_conf(path, rstrip=True):
+    with open(path, 'r') as addons:
+        lines = addons.readlines()
+    cmdline = ''
+    for l in lines:
+        l = l.lstrip()
+        if not l:
+            continue
+        if l[0] == '#':
+            continue
+        if rstrip:
+            l = l.rstrip() + ' '
+        cmdline += l
+    if cmdline == '':
+        return ''
+    return cmdline
+
+def parse_all_addons_folder(in_dir):
+    global addon_sbat_string
+    sbat_loc = os.path.join(in_dir, SBAT_RELATIVE_PATH)
+
+    for el in os.listdir(in_dir):
+        el_path = os.path.join(in_dir, el)
+        # file found: save its path, replacing the old one
+        if os.path.isfile(el_path):
+            if el.endswith('.addon'):
+                uki_addons[el] = el_path
+
+    if os.path.isfile(sbat_loc):
+        # override sbat with the most specific one found
+        addon_sbat_string = parse_addon_conf(sbat_loc, rstrip=False)
+
+def recursively_find_addons(in_dir, i, folder_list):
+    # end of recursion, leaf directory. Search all addons here
+    if i == len(folder_list):
+        parse_all_addons_folder(in_dir)
+        return
+
+    # first, check for common folder
+    common_dir = os.path.join(in_dir, 'common')
+    if os.path.exists(common_dir):
+        parse_all_addons_folder(common_dir)
+
+    # second, check if there is a match with the searched folder
+    folder_dir = os.path.join(in_dir, folder_list[i])
+    if os.path.exists(folder_dir):
+        recursively_find_addons(folder_dir, i+1, folder_list)
+
+def parse_in_dir(in_dir, uki_name, distro, arch):
+    recursively_find_addons(in_dir, 0, [uki_name, distro, arch])
+
+    for addon_name, addon_path in uki_addons.items():
+        addon_name = addon_name.replace(".addon","")
+        addon_full_name = f'{addon_name}-{uki_name}.{distro}.{arch}.addon.efi'
+        cmdline = parse_addon_conf(addon_path)
+        if cmdline:
+            uki_addons_list.append(UKICmdlineAddon(addon_full_name, cmdline))
+
+def create_addons(out_dir):
+    for uki_addon in uki_addons_list:
+        out_path = os.path.join(out_dir, uki_addon.name)
+        cmd = [
+            f'{UKIFY_PATH}', 'build',
+            f'--cmdline="{uki_addon.cmdline}"',
+            f'--output={out_path}']
+        if addon_sbat_string:
+            cmd.append('--sbat="' + addon_sbat_string.rstrip() +'"')
+
+        subprocess.check_call(cmd, text=True)
+
+if __name__ == "__main__":
+    argc = len(sys.argv) - 1
+    if argc != 5:
+        usage('too few or too many parameters!')
+
+    in_dir = sys.argv[1]
+    out_dir = sys.argv[2]
+    uki_name = sys.argv[3]
+    distro = sys.argv[4]
+    arch = sys.argv[5]
+
+    in_dir, out_dir = check_clean_arguments(in_dir, out_dir)
+    parse_in_dir(in_dir, uki_name, distro, arch)
+    create_addons(out_dir)
+
+

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917

[OS-BUILD PATCHv17 2/2] redhat/uki_cmdline_addons.conf: add FIPS addon

[Emanuele Giuseppe Esposito (via Email Bridge)]

From: Emanuele Giuseppe Esposito

redhat/uki_cmdline_addons.conf: add FIPS addon

Upstream Status: RHEL-Only

The fips addon simply enable fips in the kernel command line.

Signed-off-by: Emanuele Giuseppe Esposito

diff --git a/redhat/uki_addons/virt/common/fips.addon b/redhat/uki_addons/virt/common/fips.addon
new file mode 100644
index blahblah..blahblah 100644
--- /dev/null
+++ b/redhat/uki_addons/virt/common/fips.addon
@@ -0,0 +1 @@
+fips=1

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917

[Justin M. Forbes (via Email Bridge)]

From: Justin M. Forbes on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917#note_1890590...

I would actually agree that I prefer the files broken out instead of a
tarball. We could create a script that generates a manifest of file locations
from the source repository, and a small script to properly place the files in
the correct place based on that manifest during rpmbuild.

[Jarod Wilson (via Email Bridge)]

From: Jarod Wilson on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917#note_2026569...

Can you wrap this with an `%if %{with_efiuki}` perhaps, so that people can
skip the requirement if building without the UKI bits of the build?

[Emanuele Giuseppe Esposito (via Email Bridge)]

From: Emanuele Giuseppe Esposito on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917#note_2027247...

It is already under that if condition. Please check line 789.

[Jarod Wilson (via Email Bridge)]

From: Jarod Wilson on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917#note_2027764...

Whoops, sorry, didn't look at enough context around the addition.

[Vitaly Kuznetsov (via Email Bridge)]

From: Vitaly Kuznetsov on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2917#note_2031010...

@jwilsonrh @ptalbert @jmflinuxtx I see that C9S counterpart
(https://gitlab.com/redhat/centos-stream/src/kernel/centos-
stream-9/-/merge_requests/4603) is already merged but this one got blocked on
CKI. The failures look unrelated to the change, would it be possible to waive
them or do we need to ask @eesposit to rebase? I'd like to avoid the
divergence.