[atomic-wg] Issue #493: Kernel module support

[Joe Doss]

jdoss reported a new issue against the project: `atomic-wg` that you are following:
``
There seems at least one thread [1] on the mailing list that talks about getting kernel
modules built on Atomic, but nothing that I could find has been created here to figure out
the best path forward on supporting kernel modules.

The use case that I am personally trying to fix is getting WireGuard working on Atomic. I
maintain the RHEL/CentOS/Fedora Copr for WireGuard [2] and DKMS works pretty well for
providing the WireGuard kernel module on non-atomic based installs. I have had some
WireGuard users try and use the Copr RPMs on Atomic Host which resulted in failure. One of
them was manually copying the .ko file over to get things working [3] and this isn't a
really good solution long term. I now find myself in the same situation where I would like
to use Wireguard on Atomic, so I am trying to figure out how to come up with a short term
solution to this issue as the WireGuard devs work on getting WG pushed upstream into the
mainline kernel. Even if that does happen somewhat soon, this problem will still impact
others that need third party kernel modules working such as Nvidia drivers or third party
monitoring services such as Sysdig [4] 

I started looking at creating a system container / systemd unit file that builds a new
container on boot if a new kernel is detected. The current idea is it would do the
following:

1) Check to see if a new kernel module is needed
2) Bind mount the `kernel-devel` source from the Atomic Host into the container
3) Check out the current WireGuard source and build the module (manually or via DKMS) 
4) Enable's the newly built kernel module via system container [5]

Some issues with this idea come to light quickly as the current Fedora Atomic Host
snapshot has mismatching `kernel` and `kernel-devel` packages so building the module this
way might not always work.  I haven't fully figured out if this is the right path
forward on getting kernel modules built. It seems that some folks have done somewhat
similar things for CoreOS to get kernel module support for Nvidia based GPUs. [6] [7]

I turned to IRC to get some pointers on my above idea to solve this problem and @walters
recommended that I start an issue.


[1]
https://lists.projectatomic.io/projectatomic-archives/atomic-devel/2017-N...
[2] https://copr.fedorainfracloud.org/coprs/jdoss/wireguard/
[3] https://lists.zx2c4.com/pipermail/wireguard/2017-August/001656.html
[4] https://sysdig.com/blog/dig-into-atomic-host/
[5] https://github.com/giuseppe/hellomod
[6] https://github.com/ryanolson/CoreOS-GPU
[6] https://github.com/src-d/coreos-nvidi7
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/493

[Dusty Mabe]

dustymabe added a new comment to an issue you are following:
``
>  Some issues with this idea come to light quickly as the current
> Fedora Atomic Host snapshot has mismatching kernel and kernel-devel packages so building
> the module this way might not always work.
>  
A possible workaround here is to use the [koji
cli](https://docs.pagure.org/koji/#koji-client) inside the container to run `koji
download-build --arch=x86_64 $(rpm -q --qf '%{N}-%{V}-%{R}\n'  kernel)` where the
`rpm -q` command is querying the host's rpm db (or find some other way to pass that
information in). 
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/493

[Colin Walters]

walters added a new comment to an issue you are following:
``
The core thing here is that since there's no kABI, you are not guaranteed that a
module will work as the host evolves.  This is true of non-Atomic Fedora systems today,
and has been true basically forever - nothing new here.  What changes is how the host is
built/delivered.

Mechanically there's two parts to this.  First, there's the question of where the
module gets built.  That in turn affects how it's deployed (the second issue).

DKMS builds it on the client.   There are also "akmod" variants that have the
modules pre-built for multiple kernels.  

Particularly "at scale", I think it makes the most sense to pre-build kernel
modules rather than doing it per-host.  If you set up a COPR or whatever, you can query
what kernels "will be" available in the future by looking at Koji builds.    If
your built kernel module package then has a `Requires` on the exact kernel version it was
built for, then rpm-ostree client side layering should automatically select the right one
as the primary releases pick up updated kernels.

That's going to work fairly well today because FAH/Silverblue etc. do not carry
distinct kernels.

Now everything I just said is fundamentally distinct from making a DKMS-style system work
with rpm-ostree which it seems is what you were looking at.  For that...let's perhaps
talk in https://github.com/projectatomic/rpm-ostree/issues/1091 ?

``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/493

[Colin Walters]

walters added a new comment to an issue you are following:
``
Just to cross-link there's now work targeted mostly at the nVidia driver for
Silverblue but it's obviously somewhat generic to akmods:
https://bugzilla.redhat.com/show_bug.cgi?id=1667014

https://pagure.io/workstation-ostree-config/pull-request/119
``

To reply, visit the link below or just reply to this email
https://pagure.io/atomic-wg/issue/493