[OS-BUILD PATCHv8 0/4] makefile: Add -Werror support for RHEL

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava on gitlab.com
Merge Request: https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240

The -Werror flag should be enabled for RHEL.  In ARK, however, it is a bit
tricky to enable -Werror as a build bug occasionally leaks through
upstream review and into Linus' tree, and the toolchain is rebased which
causes many new errors to be reported.  As a consequence, -Werror cannot
be  enabled in ARK but the code to enable it for RHEL can be added to the
top-level makefile.

Additional fixes: Alphabetize Makefile.variables (something I've wanted to
do for a while now) and remove a mention of Makefile.rhkpg.

Signed-off-by: Prarit Bhargava

---
 redhat/configs/build_configs.sh |    5 +
 redhat/Makefile.variables       |  184 ++++++++++++++++++++-------------------
 makefile                        |    7 +
 3 files changed, 105 insertions(+), 91 deletions(-)

[OS-BUILD PATCHv8 1/4] redhat/Makefile.variables: Alphabetize variables

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/Makefile.variables: Alphabetize variables

Alphabetize the variables in the file.  There is no functional change with
this commit.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/Makefile.variables b/redhat/Makefile.variables
index blahblah..blahblah 100644
--- a/redhat/Makefile.variables
+++ b/redhat/Makefile.variables
@@ -11,38 +11,27 @@
 # file that needs to be changed now, and is included in redhat/Makefile
 # and redhat/Makefile.common

-# This is the source branch that you are building out of typically this is
-# the current branch.
-DIST_BRANCH ?= "os-build"
-
-# This is the dist release suffix used in the package release, eg. .fc34,
-# .el8 etc. In a different branch this may be set to a fixed value.
-DIST ?= $(shell $(RPMBUILD) --eval '%{?dist}')
+# This appends 5 characters of the top-of-tree commit ID to the BUILD variable,
+# which is then reflected in the package NVR.
+ADD_COMMITID_TO_VERSION ?=

-# This is a human readable distribution variable that is used to determine the
-# OS version (fedora, centos, or rhel).
-DISTRO ?=
+# This sets the incremental build version value in the rpm NVR.  If unset, this
+# defaults to RHEL_RELEASE from Makefile.rhelver
+BUILD ?=

-# The branch used as upstream. This is what the upstream tarball is it
-# should be tracked in a local branch. This would be "master" for the
-# Linus master branch or linux-5.x.y for a stable branch. It can also be
-# any other upstream you have added as a branch locally.
-UPSTREAM_BRANCH ?= master
+# This sets the koji|brew build flags for the dist[g]-koji|brew targets.  These
+# options are passed directly to the brew and koji commands in those targets.
+BUILD_FLAGS ?=

-# If VERSION_ON_UPSTREAM is set, the versioning of the rpm package is based
-# on a branch tracking upstream. This allows for generating rpms
-# based on untagged releases.
-VERSION_ON_UPSTREAM:=1
+# This sets the koji/brew configuration profile variable.
+BUILD_PROFILE ?=

-# RELEASED_KERNEL swaps between the pre-release secureboot keys and
-# the release one, for vmlinux signing. It also controls whether Fedora
-# kernels are built as debug kernels or release kernels with a separate
-# kernel-debug build.
-# Set RELEASED_KERNEL to 1 when the upstream source tarball contains a
-#  kernel release. (This includes prepatch or "rc" releases.)
-# Set RELEASED_KERNEL to 0 when the upstream source tarball contains an
-#  unreleased kernel development snapshot.
-RELEASED_KERNEL:=0
+# This is the brew/koji scratch build target.  A list of targets can be found
+# by executing the 'brew list-targets' or 'koji list-targets' command.  The
+# default values can be found in redhat/Makefile and are dependent on the
+# target OS.  The target OS can be changed by setting the DIST or DISTRO
+# variables.
+BUILD_TARGET ?=

# BUMP_RELEASE determines whether the pkgrelease is bumped when you call
 # 'make dist-release'.  It should be set to "yes" if you expect to do
@@ -50,6 +39,27 @@ RELEASED_KERNEL:=0
 # when you expect every build to use a different base kernel version.
 BUMP_RELEASE:=yes

+# This is the dist release suffix used in the package release, eg. .fc34,
+# .el8 etc. In a different branch this may be set to a fixed value.
+DIST ?= $(shell $(RPMBUILD) --eval '%{?dist}')
+
+# This is the source branch that you are building out of typically this is
+# the current branch.
+DIST_BRANCH ?= "os-build"
+
+# This is a human readable distribution variable that is used to determine the
+# OS version (fedora, centos, or rhel).
+DISTRO ?=
+
+# This is the executable for 'git' commands.  It is used by the redhat/self-test
+# tests to provide testing information on git hashes (see redhat/self-test/egit.sh)
+GIT ?= git
+
+# This is the referenced HEAD commit for git commands and can be used to change
+# the "top of tree" commit ID.  This is useful in cases where users want to
+# build against an older commit but do not want to check out the tree.
+HEAD ?= HEAD
+
 # INCLUDE_*_FILES determines whether or not the specific OS flavor files will
 # be included as part of the source RPM.  Downstream projects, such as
 # centos-stream or RHEL, can disable file inclusion by setting these values
@@ -57,94 +67,84 @@ BUMP_RELEASE:=yes
 INCLUDE_FEDORA_FILES:=1
 INCLUDE_RHEL_FILES:=1

-# In case PATCHLIST_URL is not set to "none", Patchlist.changelog is added to
-# the kernel src.rpm, which will contain shas and commits not upstream. The
-# value of PATCHLIST_URL in this case should point to the git repository where
-# the commits are located, and will be added as a prefix to the shas listed.
-PATCHLIST_URL ?= "https://gitlab.com/cki-project/kernel-ark/-/commit"
+# This variable is the location of the KABI cross compilers
+KABI_CROSS_COMPILE_PREFIX ?= /usr/bin/
+
+# This variable is the suffix of the KABI cross compilers
+KABI_CROSS_COMPILE_SUFFIX ?= -linux-gnu-
+
+# This variable is a list of the arches that are protected under RHEL's KABI policy
+KABI_SUPPORTED_ARCHS ?=
+
+# This disables the CONFIG error checking in redhat/configs/process_configs.sh.
+NO_CONFIGCHECKS ?=

# This can be used to change the filename of the specfile.  For example, it can
 # be set to "kernel" for kernel.spec or "kernel-rt" for kernel-rt.spec.  This
 # is useful for other projects with different specfile names.
 PACKAGE_NAME ?= kernel

-# This is the executable for 'git' commands.  It is used by the redhat/self-test
-# tests to provide testing information on git hashes (see redhat/self-test/egit.sh)
-GIT ?= git
-
-# This is the referenced HEAD commit for git commands and can be used to change
-# the "top of tree" commit ID.  This is useful in cases where users want to
-# build against an older commit but do not want to check out the tree.
-HEAD ?= HEAD
+# In case PATCHLIST_URL is not set to "none", Patchlist.changelog is added to
+# the kernel src.rpm, which will contain shas and commits not upstream. The
+# value of PATCHLIST_URL in this case should point to the git repository where
+# the commits are located, and will be added as a prefix to the shas listed.
+PATCHLIST_URL ?= "https://gitlab.com/cki-project/kernel-ark/-/commit"

-# This is the brew/koji scratch build target.  A list of targets can be found
-# by executing the 'brew list-targets' or 'koji list-targets' command.  The
-# default values can be found in redhat/Makefile and are dependent on the
-# target OS.  The target OS can be changed by setting the DIST or DISTRO
-# variables.
-BUILD_TARGET ?=
+# RELEASED_KERNEL swaps between the pre-release secureboot keys and
+# the release one, for vmlinux signing. It also controls whether Fedora
+# kernels are built as debug kernels or release kernels with a separate
+# kernel-debug build.
+# Set RELEASED_KERNEL to 1 when the upstream source tarball contains a
+#  kernel release. (This includes prepatch or "rc" releases.)
+# Set RELEASED_KERNEL to 0 when the upstream source tarball contains an
+#  unreleased kernel development snapshot.
+RELEASED_KERNEL:=0

-# This sets the koji/brew configuration profile variable.
-BUILD_PROFILE ?=
+# This variable is used by the redhat/self-tests.  It should not be
+# considered stable and my be changed or removed without warning.
+RHDISTDATADIR ?=

-# This sets the koji|brew build flags for the dist[g]-koji|brew targets.  These
-# options are passed directly to the brew and koji commands in those targets.
-BUILD_FLAGS ?=
+# This variable is the local dist-git clone, and is used when dist-git is updated.
+RHDISTGIT_CACHE ?=

-# This sets the incremental build version value in the rpm NVR.  If unset, this
-# defaults to RHEL_RELEASE from Makefile.rhelver
-BUILD ?=
+# This variable is the directory in which the dist-git repo is cloned into.  This
+# can be changed to a different directory for performance reasons.
+RHDISTGIT_TMP ?= /tmp

-# Builds may include a s390x+zfcpdump arch build, which results in the build
-# failing because the UTS_RELEASE field exceeded 64 chars.  This variable can
-# be set to 1 to decrease the kernel UTS_RELEASE and SPECBUILD by 11 characters
-# (YYYYMMDDgit).
-UPSTREAMBUILD_GIT_ONLY ?=
+# This sets the base commit ID for distg-* targets, and must be set on the
+# commandline with distg-* targets.
+RHGITCOMMIT ?= $(HEAD)

# This overrides the default git repository for the distg-* targets.  See
 # the 'make dist-full-help' entry for distg-brew for information on setting
 # rhg.url.
 RHGITURL ?= $(shell $(GIT) config rhg.url || $(GIT) config remote.origin.url)

-# This sets the base commit ID for distg-* targets, and must be set on the
-# commandline with distg-* targets.
-RHGITCOMMIT ?= $(HEAD)
-
 # This sets the number of threads that scripts called by Makefile targets can
 # use.  By default, this variable is set by the makefile '-j' parameter.
 RHJOBS ?=

-# This disables the CONFIG error checking in redhat/configs/process_configs.sh.
-NO_CONFIGCHECKS ?=
-
 # This variable is only used to create self-test data.  It should not be
 # considered stable and may be changed or removed without warning.
 RHSELFTESTDATA ?=

-# This variable is used by the redhat/self-tests.  It should not be
-# considered stable and my be changed or removed without warning.
-RHDISTDATADIR ?=
-
 # If set to 1, this variable will cause the kernel package build to fail if
 # the selftests fail to build.
 SPECSELFTESTS_MUST_BUILD ?= 0

-# This variable is a list of the arches that are protected under RHEL's KABI policy
-KABI_SUPPORTED_ARCHS ?=
-
-# This variable is the location of the KABI cross compilers
-KABI_CROSS_COMPILE_PREFIX ?= /usr/bin/
-
-# This variable is the suffix of the KABI cross compilers
-KABI_CROSS_COMPILE_SUFFIX ?= -linux-gnu-
-
-# This appends 5 characters of the top-of-tree commit ID to the BUILD variable,
-# which is then reflected in the package NVR.
-ADD_COMMITID_TO_VERSION ?=
+# The branch used as upstream. This is what the upstream tarball is it
+# should be tracked in a local branch. This would be "master" for the
+# Linus master branch or linux-5.x.y for a stable branch. It can also be
+# any other upstream you have added as a branch locally.
+UPSTREAM_BRANCH ?= master

-# This variable is the directory in which the dist-git repo is cloned into.  This
-# can be changed to a different directory for performance reasons.
-RHDISTGIT_TMP ?= /tmp
+# If VERSION_ON_UPSTREAM is set, the versioning of the rpm package is based
+# on a branch tracking upstream. This allows for generating rpms
+# based on untagged releases.
+VERSION_ON_UPSTREAM:=1

-# This variable is the local dist-git clone, and is used when dist-git is updated.
-RHDISTGIT_CACHE ?=
+# Builds may include a s390x+zfcpdump arch build, which results in the build
+# failing because the UTS_RELEASE field exceeded 64 chars.  This variable can
+# be set to 1 to decrease the kernel UTS_RELEASE and SPECBUILD by 11 characters
+# (YYYYMMDDgit).
+UPSTREAMBUILD_GIT_ONLY ?=

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240

[OS-BUILD PATCHv8 2/4] redhat/Makefile.variables: Remove mention of Makefile.rhpkg

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/Makefile.variables: Remove mention of Makefile.rhpkg

The Makefile.rhpkg no longer exists.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/Makefile.variables b/redhat/Makefile.variables
index blahblah..blahblah 100644
--- a/redhat/Makefile.variables
+++ b/redhat/Makefile.variables
@@ -1,7 +1,6 @@
-# Only the variables listed in this file, ../Makefile.rhelver, and
-# Makefile.rhpkg are available for command line modification.  There is no
-# guarantee on the stability of variables found in this file, or any other
-# redhat/ Makefile.
+# Only the variables listed in this file and ../Makefile.rhelver are available
+# for command line modification.  There is no guarantee on the stability of
+# variables found in this file, or any other redhat/ Makefile.
 #
 # Merge requests to include additional variables in _this_ file must include
 # details of why the variable must be exported.

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240

[OS-BUILD PATCHv8 3/4] makefile: Add -Werror support for RHEL

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

makefile: Add -Werror support for RHEL

The -Werror flag should be enabled for RHEL.  In ARK, however, it is a bit
tricky to enable -Werror as a build bug occasionally leaks through
upstream review and into Linus' tree.  As a consequence, CONFIG_WERROR
cannot be enabled in ARK but the code to add the flag for RHEL commpiles
can be added to the top-level makefile.

This code could be placed in the top-level Makefile, around the -Werror
line added by Linus in 3fe617ccafd6 ("Enable '-Werror' by default for all
kernel builds"), but given the amount of RHEL Only code here it is best to
keep it separated in our top-level makefile.

Signed-off-by: Prarit Bhargava

diff --git a/makefile b/makefile
index blahblah..blahblah 100644
--- a/makefile
+++ b/makefile
@@ -14,3 +14,10 @@ dist-%::
 distg-%::
    $(MAKE) -C redhat $(@) $(_EXTRA_ARGS)

+ifeq (,$(filter $(ARCH), x86 x86_64 powerpc s390 aarch64))
+  ifneq ($(KBUILD_EXTMOD),)
+    # always strip out error flags for external modules
+    KBUILD_CPPFLAGS := $(filter-out -Werror,$(KBUILD_CPPFLAGS))
+    KBUILD_RUSTFLAGS := $(filter-out -Dwarnings,$(KBUILD_RUSTFLAGS))
+  endif
+endif

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240

[OS-BUILD PATCHv8 4/4] redhat/Makefile.variables: Add ENABLE_WERROR

[Prarit Bhargava (via Email Bridge)]

From: Prarit Bhargava

redhat/Makefile.variables: Add ENABLE_WERROR

kernel-ark requires CONFIG_WERROR be disabled due to the rate of change in
the upstream toolchain.  This differs from RHEL where they need
CONFIG_WERROR=y.  Instead of trying to remember to set CONFIG_WERROR=y
when RHEL is snapshotted from kernel-ark, add a Makefile.variable that
overrides CONFIG_WERROR to 'y'.

Add ENABLE_WERROR which will override CONFIG_WERROR to 'y' for all
architectures.

Signed-off-by: Prarit Bhargava

diff --git a/redhat/Makefile.variables b/redhat/Makefile.variables
index blahblah..blahblah 100644
--- a/redhat/Makefile.variables
+++ b/redhat/Makefile.variables
@@ -50,6 +50,9 @@ DIST_BRANCH ?= "os-build"
 # OS version (fedora, centos, or rhel).
 DISTRO ?=

+# This overrides CONFIG_WERROR to 'y' for all arches.  RHEL requires this variable to be set.
+ENABLE_WERROR ?=
+
 # This is the executable for 'git' commands.  It is used by the redhat/self-test
 # tests to provide testing information on git hashes (see redhat/self-test/egit.sh)
 GIT ?= git
diff --git a/redhat/configs/build_configs.sh b/redhat/configs/build_configs.sh
index blahblah..blahblah 100755
--- a/redhat/configs/build_configs.sh
+++ b/redhat/configs/build_configs.sh
@@ -110,6 +110,11 @@ function merge_configs()
    esac

sort config-merging."$count" >> "$name"
+
+	if [ -n "$ENABLE_WERROR" ]; then
+	       sed -i "1,$$s|# CONFIG_WERROR is not set|CONFIG_WERROR=y|g" "$name"
+	fi
+
    rm -f config-merged."$count" config-merging."$count"
    echo "Building $name complete"
 }

--
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240

[Justin M. Forbes (via Email Bridge)]

From: Justin M. Forbes on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240#note_1245811...

I will approve it, though honestly Makefile.variables was currently arranged
from most likely to be changed, meaning the further you go down the file, the
less likely it is that you will need to change a value, while things like
DIST_BRANCH, UPSTREAM_BRANCH, VERSION_ON_UPSTREAM, and BUMP_RELEASE are
frequently changed.

[Patrick Talbert (via Email Bridge)]

From: Patrick Talbert on gitlab.com
https://gitlab.com/cki-project/kernel-ark/-/merge_requests/2240#note_1249812...

@prarit it looks like the selftest dies because of a new environment variable?

https://gitlab.com/redhat/red-hat-ci-tools/kernel/cki-internal-pipelines/cki...
trusted-contributors/-/jobs/3647053165#L99