[OS-BUILD PATCH 0/2] redhat/Makefile: Improve dist-buildreq-check target

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900

The dist-buildreq-check target can be improved in a few ways by allowing
users to convert the fatal error into a warning, and by indicating where
the system-sb-certs package can be found.

Signed-off-by: Prarit Bhargava

---
 redhat/Makefile |  14 ++++++++++++--
 1 files changed, 12 insertions(+), 2 deletions(-)

[OS-BUILD PATCH 1/2] redhat/Makefile: Provide better system-sb-certs missing package message

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/Makefile: Provide better system-sb-certs missing package message

Most of the kernel build dependencies come from widely known repositories
including EPEL, RHEL, and Fedora.  The system-sb-certs package comes from
a smaller RHEL repository called CodeReady Linux Builder (CRB).

Provide a better system-sb-certs missing package message that points users
to the CRB repository.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/Makefile b/redhat/Makefile
index blahblah..blahblah 100644
--- a/redhat/Makefile
+++ b/redhat/Makefile
@@ -456,6 +456,9 @@ dist-buildreq-check: setup-source
    done; \
    if [ -n "$$MISSING" ]; then \
    	echo "Error: please install $$MISSING"; \
+		if  echo "$$MISSING" | grep -q system-sb-certs ; then \
+			echo "Error: system-sb-certs can be installed from the CodeReady Linux Builder (CRB) repository."; \
+		fi; \
    	exit 1; \
    fi

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900

[OS-BUILD PATCH 2/2] redhat/Makefile: Add DIST_BUILDREQ_WARN variable

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/Makefile: Add DIST_BUILDREQ_WARN variable

dist-* targets will fail if packages required by the build are not
installed.  In some cases, such as dist-configs, the build packages are
not required for command execution.

Provide a DIST_BUILDREQ_WARN variable that allows users to skip dist-*
errors due to missing packages.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/Makefile b/redhat/Makefile
index blahblah..blahblah 100644
--- a/redhat/Makefile
+++ b/redhat/Makefile
@@ -454,12 +454,19 @@ dist-buildreq-check: setup-source
    for pkg in $$PKGLIST; do \
    	rpm -q --whatprovides $$pkg >/dev/null || MISSING="$$MISSING $$pkg"; \
    done; \
+	warn="Error"; \
+	if [ -n "$$DIST_BUILDREQ_WARN" ]; then \
+		warn="Warning"; \
+	fi; \
    if [ -n "$$MISSING" ]; then \
-		echo "Error: please install $$MISSING"; \
+		echo "$$warn: Please install: $$MISSING"; \
    	if  echo "$$MISSING" | grep -q system-sb-certs ; then \
-			echo "Error: system-sb-certs can be installed from the CodeReady Linux Builder (CRB) repository."; \
+			echo "$$warn: system-sb-certs can be installed from the CodeReady Linux Builder (CRB) repository."; \
+		fi; \
+		if  [ $$warn == "Error" ]; then \
+			echo "$$warn: These errors can be skipped by specifying DIST_BUILDREQ_WARN=1."; \
+			exit 1; \
    	fi; \
-		exit 1; \
    fi

setup-source: dist-git-version-check dist-clean-sources

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900

[Ondrej Mosnáček (via Email Bridge)]

From: Ondrej Mosnáček on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1012031...

There seems to be some overlap between this MR and !1786.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1012523...

@dzickusrh & I had a significant discussion about this MR and what the
"right" thing to do is.  There are several facets to this problem
including what users should do to install packages, how they should know
to install those packages, and how and when errors should be displayed.

As a result of this conversation I am putting this MR back into draft for
a major rework.  I am going to update the documentation with instructions
to run a new target dist-get-buildreqs that will output the set of build
requirement rpms for the kernel.  The documentation will include also
include a statement that explains that the list of packages may or may not
be necessary for all dist-* targets.

In addition to that, I will rename the dist-buildreq-check to the
previously mentioned dist-get-buildreqs, and add in a section to better
explain the need for the system-sb-certs package.  Instead of pointing
everyone to a Red Hat Repository, I will point people to the CentOS Stream
repository that contains the matching package.  In this way we can then
also include a URL to the repository.

Users requiring the list of packages (automated builds, etc.) can then
do something like:

```
make dist-get-buildreqs | grep "Please install:" | cut -d":" -f2-
```

or something similar to get the full list of required packages.

The direct calling of dist-buildreq-check will be removed from the
targets.  It is expected that the above, once documented, should be
sufficient for users to understand how to install the kernel-ark build
requirements.

[Justin M. Forbes (via Email Bridge)]

From: Justin M. Forbes on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1012576...

'sudo dnf builddep kernel' should install the list of required packages
without much hassle.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1013253...
> 'sudo dnf builddep kernel' should install the list of required packages

without much hassle.

Yep, that works but there's a subtle issue here.  This command will
install the dependencies for the kernel version that is running on the
system, not the local git repository's dependencies.

ie) if you run this command on RHEL8, the result would be the dependencies
for the RHEL8 kernel not kernel-ark.

In my prarit rpm I am careful to install a yum config file for the
targeted kernel, and then do

```
yum deplist --enablerepo latest-Base-OS-Source-${distnum} --srpm kernel | awk
'/provider:/ {print $2}' | grep -v ".src" | grep $(uname -m) | sort -u |
xargs -r yum ${yumargs} -y install
```

Prolly (horribly?) over-engineered though; I'll look into just using builddep.

[Justin M. Forbes (via Email Bridge)]

From: Justin M. Forbes on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1013293...

builddep can be passed a spec file or srpm as well, unfortunately it seems to
choke if passed the kernel.spec.template, so we would need a processed
template into a proper spec.

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1013368...
> builddep can be passed a spec file or srpm as well, unfortunately it seems to

choke if passed the kernel.spec.template, so we would need a processed
template into a proper spec.

hehe :)  Yeah, the double-percent issue.  I ran into that too :)

@dzickusrh had an objection to running the install from within the
makefile.  I tend to lean towards his viewpoint; let the user make a
decision if they want to install the packages or not.  Let's just output a
list of the packages and let the user decide what they need to do.  It's
also possible (see system-sb-certs) that a package may not exist on the OS.

I'm planning on calling setup-source, just like dist-buildreq-check does
today.  There's still a small problem that I'm worried about: The
setup-source target may require packages to be installed as well, and if
tell people to execute a target that calls setup-source we may end up a
catch-22.  AFAICT setup-source only seems to call bash and simple commands
so maybe it's not a problem.

Additionally, I need to think about how to test this target.  That's a bit
tricky and I'll need @veruu and the CKI team's help in setting up a test.
I need a "plain" builder_image without any dependencies installed.  The
first test to run would be to confirm a, for example, 'make dist-srpm'
failure.  The second test, would be to call the new dist-* target to
list the dependencies, install the dependencies, and then run 'make
dist-srpm' to confirm that the new dist-* target worked.

^^^ I suspect that @veruu and the CKI team are laughing at me for not
knowing how easy this is to do :) :) but I'm not overly familiar with
CI and will need a bit of hand-holding the first time I add a test like
this.

[Veronika Kabátová (via Email Bridge)]

From: Veronika Kabátová on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/1900#note_1013394...

You can grab any base container and have your fun :) the kernel build
dependencies are *not* installed by default. Depending on which variant you
want:

```
podman image pull quay.io/fedora/fedora:rawhide
podman image pull quay.io/fedora/fedora:eln-x86_64
```