[OS-BUILD PATCH 0/4] rpm builds: Fix warnings and output

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255

There are several groups of warnings and output that should be fixed in
the kernel's rpm build commands.

1) The output of pathfix.py is too noisy.  Users do not need to see the
directory traversal messages, and only need to see files that were changed
(or not changed) by the script.

2) There is a vmlinux_decompressor error in !s390x builds.  This code
must only be executed on s390x builds.

3) There are references to an old .tmp_versions directory that is no
longer used in the build.  These references can be removed entirely from
the tree.

4) There are "File listed twice" rpm build errors that occur because the
rpm spec's kernel-modules-internal section %files section lists them
twice.  Of course, they should only be listed once.

The four commits in this series fix the above four issues.

Test build: https://koji.fedoraproject.org/koji/taskinfo?taskID=96528106

Signed-off-by: Prarit Bhargava

---
 redhat/scripts/mod/mod-denylist.sh |  11 -----------
 redhat/kernel.spec.template        |  27 +++++++++++----------------
 2 files changed, 11 insertions(+), 27 deletions(-)

[OS-BUILD PATCH 1/4] redhat/kernel.spec.template: Remove unnecessary output from pathfix.py

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/kernel.spec.template: Remove unnecessary output from pathfix.py

pathfix.py outputs directory names as it traverses them looking for files
to fix, for example,

recursedown('tools')
recursedown('tools/accounting')
recursedown('tools/arch')

This information isn't really useful and can be omitted.  The output that
is interesting, such as files that were changed, and in some cases files
that were not changed will remain in the rpm build output.

Remove unnecessary output from pathfix.py.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -1490,6 +1490,7 @@ touch .scmversion
 # This fixes errors such as
 # *** ERROR: ambiguous python shebang in /usr/bin/kvm_stat: #!/usr/bin/python. Change it to python3 (or python2) explicitly.
 # We patch all sources below for which we got a report/error.
+echo "Fixing Python shebangs..."
 pathfix.py -i "%{__python3} %{py3_shbang_opts}" -p -n \
    tools/kvm/kvm_stat/kvm_stat \
    scripts/show_delta \
@@ -1498,7 +1499,7 @@ pathfix.py -i "%{__python3} %{py3_shbang_opts}" -p -n \
    scripts/jobserver-exec \
    tools \
    Documentation \
-	scripts/clang-tools
+	scripts/clang-tools 2> /dev/null

# only deal with configs if we are going to build for the arch
 %ifnarch %nobuildarches

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255

[OS-BUILD PATCH 2/4] redhat/kernel.spec.template: Fix vmlinux_decompressor on !s390x

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/kernel.spec.template: Fix vmlinux_decompressor on !s390x

The build log contains

eu-readelf: cannot open input file '%{vmlinux_decompressor}': No such file or directory

This should only execute on s390x.

Fix vmlinux_decompressor warnings for all other arches.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -2068,6 +2068,7 @@ BuildKernel() {
     mkdir -p $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
     mv vmlinux $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer
     ln -s $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer/vmlinux vmlinux
+%ifarch s390x
     if [ -n "%{vmlinux_decompressor}" ]; then
        eu-readelf -n  %{vmlinux_decompressor} | grep "Build ID" | awk '{print $NF}' > vmlinux.decompressor.id
        # Without build-id the build will fail. But for s390 the build-id
@@ -2078,6 +2079,7 @@ BuildKernel() {
    	    cp %{vmlinux_decompressor} $RPM_BUILD_ROOT%{debuginfodir}/lib/modules/$KernelVer/vmlinux.decompressor
        fi
     fi
+%endif
 %endif

find $RPM_BUILD_ROOT/lib/modules/$KernelVer -name "*.ko" -type f >modnames

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255

[OS-BUILD PATCH 2/4] redhat/kernel.spec.template: Fix vmlinux_decompressor on !s390x

[Jan Stancek (via Email Bridge)]

From: Jan Stancek on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255#note_1248824...

vmlinux_decompressor is already defined only on s390x, so it should be enough
to let it expand to empty string otherwise:
```
-    if [ -n "%{vmlinux_decompressor}" ]; then
+    if [ -n "%{?vmlinux_decompressor}" ]; then

```

[OS-BUILD PATCH 2/4] redhat/kernel.spec.template: Fix vmlinux_decompressor on !s390x

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255#note_1249648...

Changed in new version.

[OS-BUILD PATCH 3/4] redhat: Remove stale .tmp_versions code and comments

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat: Remove stale .tmp_versions code and comments

At sometime prior to the beginning of this repository, .tmp_versions
was removed as a temporary data store in the rpm build.  Unfortunately two
references to it remain: one as a comment and one in code.

The code remnant causes false errors like

rm: cannot remove '.tmp_versions/gamecon.mod': No such file or directory
rm: cannot remove '.tmp_versions/gf2k.mod': No such file or directory
rm: cannot remove '.tmp_versions/grip.mod': No such file or directory
rm: cannot remove '.tmp_versions/grip_mp.mod': No such file or directory
rm: cannot remove '.tmp_versions/guillemot.mod': No such file or directory

for every module listed in mod-extra.list, mod-internal.list, and
mod-partners.list.

The mentions of .tmp_versions are stale and should be removed.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -2449,16 +2449,10 @@ chmod -R a=rX Documentation
 find Documentation -type d | xargs chmod u+w
 %endif

-# In the modsign case, we do 3 things.  1) We check the "variant" and hard
-# code the value in the following invocations.  This is somewhat sub-optimal
-# but we're doing this inside of an RPM macro and it isn't as easy as it
-# could be because of that.  2) We restore the .tmp_versions/ directory from
-# the one we saved off in BuildKernel above.  This is to make sure we're
-# signing the modules we actually built/installed in that variant.  3) We
-# grab the arch and invoke mod-sign.sh command to actually sign the modules.
-#
-# We have to do all of those things _after_ find-debuginfo runs, otherwise
-# that will strip the signature off of the modules.
+# Module signing (modsign)
+#
+# This must be run _after_ find-debuginfo.sh runs, otherwise that will strip
+# the signature off of the modules.
 #
 # Don't sign modules for the zfcpdump variant as it is monolithic.

diff --git a/redhat/scripts/mod/mod-denylist.sh b/redhat/scripts/mod/mod-denylist.sh
index blahblah..blahblah 100755
--- a/redhat/scripts/mod/mod-denylist.sh
+++ b/redhat/scripts/mod/mod-denylist.sh
@@ -127,17 +127,6 @@ fi

popd

-# If we're signing modules, we can't leave the .mod files for the .ko files
-# we've moved in .tmp_versions/.  Remove them so the Kbuild 'modules_sign'
-# target doesn't try to sign a non-existent file.  This is kinda ugly, but
-# so are the modules-* packages.
-
-while IFS= read -r mod
-do
-  modfile=$(basename "$mod" | sed -e 's/.ko/.mod/')
-  rm .tmp_versions/"$modfile"
-done < "$Dir"/dep2.list
-
 if [ -z "$Dest" ]; then
    sed -e "s|^.|${ModDir}|g" "$Dir"/dep2.list > "$RpmDir/$ListName"
    echo "./$RpmDir/$ListName created."

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255

[OS-BUILD PATCH 4/4] redhat/kernel.spec.template: Fix internal "File listed twice" errors

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/kernel.spec.template: Fix internal "File listed twice" errors

Executing an rpm build results in a "File listed twice" error for every
file in the modules-internal subpackage.

The modules-internal subpackage's file list is created by mod-denylist.sh.
The file contains both the path to the internal directory and separately
lists out each file.

ie) the contents of the file list are

<snip>
/lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/net/mptcp/mptcp_crypto_test.ko.xz/lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/net/mptcp/mptcp_token_test.ko.xz
/lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/sound/soc/soc-topology-test.ko.xz/lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/sound/soc/soc-utils-test.ko.xz
/lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal

The rpm build interprets this as including each file individually and also
recursively including the files below the internal directory, ie) each
file is listed twice.  The rpm build then throws errors for each file:

File listed twice: /lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/net/mptcp/mptcp_crypto_test.ko.xz
    File listed twice: /lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/net/mptcp/mptcp_token_test.ko.xz
    File listed twice: /lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/sound/soc/soc-topology-test.ko.xz
    File listed twice: /lib/modules/6.2.0-0.rc4.7287904c8771.33.test.fc36.x86_64/internal/sound/soc/soc-utils-test.ko.xz

Fix the "File listed twice" errors by only including the internal directory in the
modules-internal subpackage file list.

Additional fix: Make the same fix for the modules-partner subpackge.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -2208,10 +2208,6 @@ BuildKernel() {
     sed -e 's/^lib*/%dir /lib/' %{?zipsed} $RPM_BUILD_ROOT/module-dirs.list > ../kernel${Variant:+-${Variant}}-modules-core.list
     sed -e 's/^lib*//lib/' %{?zipsed} $RPM_BUILD_ROOT/modules.list >> ../kernel${Variant:+-${Variant}}-modules-core.list
     sed -e 's/^lib*//lib/' %{?zipsed} $RPM_BUILD_ROOT/mod-extra.list >> ../kernel${Variant:+-${Variant}}-modules-extra.list
-    sed -e 's/^lib*//lib/' %{?zipsed} $RPM_BUILD_ROOT/mod-internal.list >> ../kernel${Variant:+-${Variant}}-modules-internal.list
-%if 0%{!?fedora:1}
-    sed -e 's/^lib*//lib/' %{?zipsed} $RPM_BUILD_ROOT/mod-partner.list >> ../kernel${Variant:+-${Variant}}-modules-partner.list
-%endif

# Cleanup
     rm -f $RPM_BUILD_ROOT/k-d.list
@@ -3176,9 +3172,11 @@ fi
 %{expand:%%files %{?3:%{3}-}devel-matched}\
 %{expand:%%files -f kernel-%{?3:%{3}-}modules-extra.list %{?3:%{3}-}modules-extra}\
 %config(noreplace) /etc/modprobe.d/*-blacklist.conf\
-%{expand:%%files -f kernel-%{?3:%{3}-}modules-internal.list %{?3:%{3}-}modules-internal}\
+%{expand:%%files %{?3:%{3}-}modules-internal}\
+/lib/modules/%{KVERREL}%{?3:+%{3}}/internal\
 %if 0%{!?fedora:1}\
-%{expand:%%files -f kernel-%{?3:%{3}-}modules-partner.list %{?3:%{3}-}modules-partner}\
+%{expand:%%files %{?3:%{3}-}modules-partner}\
+/lib/modules/%{KVERREL}%{?3:+%{3}}/partner\
 %endif\
 %if %{with_debuginfo}\
 %ifnarch noarch\

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2255