[OS-BUILD PATCH 0/2] redhat: update merge.py to handle merge.pl corner cases

[Clark Williams (via Email Bridge)]

From: Clark Williams on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2262

Change merge.py to handle input config files in  similar manner to
how merge.pl did it.

Signed-off-by: Clark Williams

---
 redhat/configs/merge.pl     |   72 -----------------------------
 redhat/configs/merge.py     |  107 ++++++++++++++++++++++++++-----------------
 redhat/Makefile             |    2 +-
 redhat/kernel.spec.template |   10 ++--
 4 files changed, 71 insertions(+), 120 deletions(-)

[OS-BUILD PATCH 1/2] redhat: update merge.py to handle merge.pl corner cases

[Clark Williams (via Email Bridge)]

From: Clark Williams

redhat: update merge.py to handle merge.pl corner cases

Change merge.py to handle input config files in  similar manner to
how merge.pl did it.

Signed-off-by: Clark Williams

diff --git a/redhat/configs/merge.py b/redhat/configs/merge.py
index blahblah..blahblah 100755
--- a/redhat/configs/merge.py
+++ b/redhat/configs/merge.py
@@ -5,54 +5,77 @@
 #
 # merge.py - a direct replacement for merge.pl in the redhat/configs directory
 #
-# invocation:   python merge.py overrides baseconfig
+# invocation:   python merge.py overrides baseconfig [arch]
 #
-# Both input files are kernel config files, where overides is config overides
-# to the baseconfig file. Both are read into python dictionaries with the
-# keys being the config name and the values being the config file text
-
-# The script iterates through the overrides keys adding/replacing the contents
-# of the baseconfig values and then outputs the new baseconfig to stdout.
+# This script merges two kernel configuration files, an override file and a
+# base config file and writes the results to stdout.
 #
+# The script reads the overrides into a dictionary, then reads the baseconfig
+# file, looking for overrides and replacing any found, then printing the result
+# to stdout. Finally any remaining (new) configs in the override are appended to the
+# end of the output

import sys
+import re
 import os.path

def usage(msg):
-    print(msg)
-    print("usage: merge.py overrides baseconfig")
+    sys.stderr.write(msg + "\n")
+    sys.stderr.write("usage: merge.py overrides baseconfig [arch]\n")
     sys.exit(1)

-# read a config file and return a dictionary of the contents
-def read_config_file(cfgfile):
-    configs = {}
-    with open(cfgfile) as f:
-        for l in [n.strip() for n in f.readlines()]:
-            if not l:  continue
-            if l.startswith("# CONFIG_"):
-                configs[l.split()[1]] = l
-                continue
-            if l.startswith("CONFIG_"):
-                configs[l.split('=')[0]] = l
-                continue
-    return configs
-
-
-if len(sys.argv) < 3: usage("must have two input files")
-
-# read in the overides file
-if not os.path.exists(sys.argv[1]):
-    usage("overrides config file %s does not exist!" % sys.argv[1])
-overrides = read_config_file(sys.argv[1])
-
-# read in the base config file
-if not os.path.exists(sys.argv[2]):
-    usage("base config file %s does not exist!" % sys.argv[2])
-baseconfigs = read_config_file(sys.argv[2])
-
-# update baseconfigs with the overrides values
-baseconfigs.update(overrides)
-
-# print the new config to stdout
-for v in baseconfigs.values():
-    print(v)
+isset = re.compile(r'^(CONFIG_\w+)=')
+notset = re.compile(r'^#\s+(CONFIG_\w+)\s+is not set')
+
+# search an input line for a config (set or notset) pattern
+# if we get a match return the config that is being changed
+def find_config(line):
+    if m := isset .match(line):
+        return m.group(1)
+    if m := notset.match(line):
+        return m.group(1)
+    return None
+
+#########################################################
+
+if len(sys.argv) < 3:
+    usage("must have two input files")
+
+override_file = sys.argv[1]
+baseconfig_file = sys.argv[2]
+
+if not os.path.exists(override_file):
+    usage("overrides config file %s does not exist!" % override_file)
+
+if not os.path.exists(baseconfig_file):
+    usage("base configs file %s does not exist" % baseconfig_file)
+
+if len(sys.argv) == 4:
+    print("# %s" % sys.argv[3])
+
+# read each line of the override file and store any configuration values
+# in the overrides dictionary, keyed by the configuration name.
+overrides = {}
+with open(override_file, "rt") as f:
+    for l in [n.strip() for n in f.readlines()]:
+        if c := find_config(l):
+            overrides[c] = l
+
+# now read and print the base config, checking each line
+# that defines a config value and printing the override if
+# it exists
+with open(baseconfig_file, "rt") as f:
+    for line in [ l.strip() for l in f.readlines() ]:
+        c = find_config(line)
+        if c and c in overrides:
+            print(overrides[c])
+            del overrides[c]
+        else:
+            print(line)
+
+# print out the remaining configs (new values)
+# from the overrides file
+for c in overrides:
+    print (overrides[c])
+
+sys.exit(0)

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2262

[OS-BUILD PATCH 1/2] redhat: update merge.py to handle merge.pl corner cases

[Patrick Talbert (via Email Bridge)]

From: Patrick Talbert on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2262#note_1254026...

```suggestion:-0+0
    if m := isset.match(line):
```

[OS-BUILD PATCH 2/2] redhat: remove merge.pl and references to it

[Clark Williams (via Email Bridge)]

From: Clark Williams

redhat: remove merge.pl and references to it

get rid of redhat/configs/merge.pl and any references to it
in redhat/kernel.spec.template and redhat/Makefile

Signed-off-by: Clark Williams

diff --git a/redhat/Makefile b/redhat/Makefile
index blahblah..blahblah 100644
--- a/redhat/Makefile
+++ b/redhat/Makefile
@@ -634,7 +634,7 @@ sources-rh: $(TARBALL) generate-testpatch-tmp setup-source dist-configs-check
    	scripts/mod/mod-sign.sh \
    	configs/flavors \
    	configs/generate_all_configs.sh \
-		configs/merge.pl \
+		configs/merge.py \
    	configs/process_configs.sh \
    	../Makefile.rhelver \
    	README.rst \
diff --git a/redhat/configs/merge.pl b/redhat/configs/merge.pl
deleted file mode 100755
index blahblah..blahblah 0
--- a/redhat/configs/merge.pl
+++ /dev/null
@@ -1,72 +0,0 @@
-#! /usr/bin/perl
-
-my @args=@ARGV;
-my %configvalues;
-my @configoptions;
-my $configcounter = 0;
-
-# optionally print out the architecture as the first line of our output
-my $arch = $args[2];
-if (defined $arch) {
-	print "# $arch\n";
-}
-
-# first, read the override file
-
-open (FILE,"$args[0]") || die "Could not open $args[0]";
-while (<FILE>) {
-	my $str = $_;
-	my $configname;
-
-	if (/# ([\w]+) is not set/) {
-		$configname = $1;
-	} elsif (/^#/) {
-		# fall through on comments like 'avoid CONFIG_FOO=y'
-		;
-	} elsif (/([\w]+)=/) {
-		$configname = $1;
-	}
-
-	if (defined($configname) && !exists($configvalues{$configname})) {
-		$configvalues{$configname} = $str;
-		$configoptions[$configcounter] = $configname;
-		$configcounter ++;
-	}
-};
-
-# now, read and output the entire configfile, except for the overridden
-# parts... for those the new value is printed.
-
-open (FILE2,"$args[1]") || die "Could not open $args[1]";
-while (<FILE2>) {
-	my $configname;
-
-	if (/# ([\w]+) is not set/) {
-		$configname = $1;
-	} elsif (/^#/) {
-		# fall through on comments like 'avoid CONFIG_FOO=y'
-		;
-	} elsif (/([\w]+)=/) {
-		$configname  = $1;
-	}
-
-	if (defined($configname) && exists($configvalues{$configname})) {
-		print "$configvalues{$configname}";
-		delete($configvalues{$configname});
-	} else {
-		print "$_";
-	}
-}
-
-# now print the new values from the overridden configfile
-my $counter = 0;
-
-while ($counter < $configcounter) {
-	my $configname = $configoptions[$counter];
-	if (exists($configvalues{$configname})) {
-		print "$configvalues{$configname}";
-	}
-	$counter++;
-}
-
-1;
diff --git a/redhat/kernel.spec.template b/redhat/kernel.spec.template
index blahblah..blahblah 100755
--- a/redhat/kernel.spec.template
+++ b/redhat/kernel.spec.template
@@ -850,7 +850,7 @@ Source2002: kvm_stat.logrotate
 # Some people enjoy building customized kernels from the dist-git in Fedora and
 # use this to override configuration options. One day they may all use the
 # source tree, but in the mean time we carry this to support the legacy workflow
-Source3000: merge.pl
+Source3000: merge.py
 Source3001: kernel-local
 %if %{patchlist_changelog}
 Source3002: Patchlist.changelog
@@ -1042,7 +1042,7 @@ This package provides debug information for package kernel-tools.
 %{expand:%%global _find_debuginfo_opts %{?_find_debuginfo_opts} -p '.*%%{_bindir}/centrino-decode(.debug)?|.*%%{_bindir}/powernow-k8-decode(.debug)?|.*%%{_bindir}/cpupower(.debug)?|.*%%{_libdir}/libcpupower.*|.*%%{_bindir}/turbostat(.debug)?|.*%%{_bindir}/x86_energy_perf_policy(.debug)?|.*%%{_bindir}/tmon(.debug)?|.*%%{_bindir}/lsgpio(.debug)?|.*%%{_bindir}/gpio-hammer(.debug)?|.*%%{_bindir}/gpio-event-mon(.debug)?|.*%%{_bindir}/gpio-watch(.debug)?|.*%%{_bindir}/iio_event_monitor(.debug)?|.*%%{_bindir}/iio_generic_buffer(.debug)?|.*%%{_bindir}/lsiio(.debug)?|.*%%{_bindir}/intel-speed-select(.debug)?|.*%%{_bindir}/page_owner_sort(.debug)?|.*%%{_bindir}/slabinfo(.debug)?|.*%%{_sbindir}/intel_sdsi(.debug)?|XXX' -o kernel-tools-debuginfo.list}

%package -n rtla
-Summary: RTLA: Real-Time Linux Analysis tools 
+Summary: RTLA: Real-Time Linux Analysis tools
 %description -n rtla
 The rtla tool is a meta-tool that includes a set of commands that
 aims to analyze the real-time properties of Linux. But, instead of
@@ -1512,7 +1512,7 @@ cd configs
 # Drop some necessary files from the source dir into the buildroot
 cp $RPM_SOURCE_DIR/kernel-*.config .
 cp %{SOURCE80} .
-# merge.pl
+# merge.py
 cp %{SOURCE3000} .
 # kernel-local
 cp %{SOURCE3001} .
@@ -1523,12 +1523,12 @@ FLAVOR=%{primary_target} SPECVERSION=%{version} ./generate_all_configs.sh %{debu
 for i in %{all_arch_configs}
 do
   mv $i $i.tmp
-  ./merge.pl %{SOURCE3001} $i.tmp > $i
+  ./merge.py %{SOURCE3001} $i.tmp > $i
 %if %{with_gcov}
   echo "Merging with gcov options"
   cat %{SOURCE75}
   mv $i $i.tmp
-  ./merge.pl %{SOURCE75} $i.tmp > $i
+  ./merge.py %{SOURCE75} $i.tmp > $i
 %endif
   rm $i.tmp
 done

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2262