building the rawhide kernel with cgroup-v2 cpu controller patches

[Zbigniew Jędrzejewski-Szmek]

Unfortunately, due to the disagreements in the kernel development
community, CPU controller cgroup v2 support has not been merged and
enabling it requires applying two small out-of-tree kernel
patches. The situation is explained in the following documentation:

https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/tree/Documenta...

While it isn't clear what will happen with CPU controller cgroup v2
support, there are critical features which are possible only on cgroup
v2 such as buffered write control making cgroup v2 essential for a lot
of workloads. Also we finally have reliable empty cgroup reporting which
avoids various systemd issues with zombie scope units and such.

Systemd recently gained experimental support for the new cgroup v2 cpu
controller [1], which of course only works for people who
a) are running with systemd.unified_cgroup_hierarchy=1 and
b) have a patched kernel. We would like to move closer towards
switching to v2 hierarchy, with an eye towards using unified hierarchy
by default, but we have a chicken and egg problem where we cannot
encourage people to switch and test systemd with v2 hierarchy,
since the kernel support is incomplete. And the kernel support does
not get enough testing because it's incomplete and requires a
substantive effort.

Would it be possible to compile the rawhide kernels with the cgroup-v2-cpu
patches [2]? This should cause no issue for v1 users, and would let the
kernel and systemd functionality get wider testing, and hopefully nudge the
kernel upstream towards finalizing the unified hierarchy support.

Zbyszek

[1] https://github.com/systemd/systemd/commit/5f9a610ad2b3200a3
[2] https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/log/?h=cgroup-...

[Josh Boyer]

On Tue, Aug 16, 2016 at 9:31 PM, Zbigniew Jędrzejewski-Szmek
<zbyszek(a)in.waw.pl&gt; wrote:
>  Unfortunately, due to the disagreements in the kernel development
>  community, CPU controller cgroup v2 support has not been merged and
>  enabling it requires applying two small out-of-tree kernel
>  patches. The situation is explained in the following documentation:
>
>
> https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/tree/Documenta...
>
>  While it isn't clear what will happen with CPU controller cgroup v2
>  support, there are critical features which are possible only on cgroup
>  v2 such as buffered write control making cgroup v2 essential for a lot
>  of workloads. Also we finally have reliable empty cgroup reporting which
>  avoids various systemd issues with zombie scope units and such.
>
>  Systemd recently gained experimental support for the new cgroup v2 cpu
>  controller [1], which of course only works for people who
>  a) are running with systemd.unified_cgroup_hierarchy=1 and
>  b) have a patched kernel. We would like to move closer towards
>  switching to v2 hierarchy, with an eye towards using unified hierarchy
>  by default, but we have a chicken and egg problem where we cannot
>  encourage people to switch and test systemd with v2 hierarchy,
>  since the kernel support is incomplete. And the kernel support does
>  not get enough testing because it's incomplete and requires a
>  substantive effort.
>
>  Would it be possible to compile the rawhide kernels with the cgroup-v2-cpu
>  patches [2]? This should cause no issue for v1 users, and would let the
>  kernel and systemd functionality get wider testing, and hopefully nudge the
>  kernel upstream towards finalizing the unified hierarchy support. 
So the past 3 times we've done this, it didn't work out that way at
all.  Despite the kernel developers saying they want in-distro
testing, it hasn't actually helped the upstream merge conversation.
The utrace, Secure Boot, and kdbus code all was in Fedora for a very
long time and only portions of utrace were ever merged.  With kdbus is
wasn't a huge issue because it was self contained within a module, but
we're still carrying the Secure Boot patches today.  The cgroup-v2
patches aren't self contained and if the upstream community continues
to disagree we could be setting ourselves up for another round of
patches we're carrying that aren't upstream.

Whether or not Fedora carries these isn't my call, but I'd approach
this with caution.  If the systemd developers are looking to simply
have a kernel to test against, perhaps reviving the kernel-playground
COPR with these would be enough.

josh

[Laura Abbott]

On 08/17/2016 04:06 AM, Josh Boyer wrote:
>  On Tue, Aug 16, 2016 at 9:31 PM, Zbigniew Jędrzejewski-Szmek
>  <zbyszek(a)in.waw.pl&gt; wrote:
> > Unfortunately, due to the disagreements in the kernel development
> > community, CPU controller cgroup v2 support has not been merged and
> > enabling it requires applying two small out-of-tree kernel
> > patches. The situation is explained in the following documentation:
> >
> >
> https://git.kernel.org/cgit/linux/kernel/git/tj/cgroup.git/tree/Documenta...
> >
> > While it isn't clear what will happen with CPU controller cgroup v2
> > support, there are critical features which are possible only on cgroup
> > v2 such as buffered write control making cgroup v2 essential for a lot
> > of workloads. Also we finally have reliable empty cgroup reporting which
> > avoids various systemd issues with zombie scope units and such.
> >
> > Systemd recently gained experimental support for the new cgroup v2 cpu
> > controller [1], which of course only works for people who
> > a) are running with systemd.unified_cgroup_hierarchy=1 and
> > b) have a patched kernel. We would like to move closer towards
> > switching to v2 hierarchy, with an eye towards using unified hierarchy
> > by default, but we have a chicken and egg problem where we cannot
> > encourage people to switch and test systemd with v2 hierarchy,
> > since the kernel support is incomplete. And the kernel support does
> > not get enough testing because it's incomplete and requires a
> > substantive effort.
> >
> > Would it be possible to compile the rawhide kernels with the cgroup-v2-cpu
> > patches [2]? This should cause no issue for v1 users, and would let the
> > kernel and systemd functionality get wider testing, and hopefully nudge the
> > kernel upstream towards finalizing the unified hierarchy support.
>
>  So the past 3 times we've done this, it didn't work out that way at
>  all.  Despite the kernel developers saying they want in-distro
>  testing, it hasn't actually helped the upstream merge conversation.
>  The utrace, Secure Boot, and kdbus code all was in Fedora for a very
>  long time and only portions of utrace were ever merged.  With kdbus is
>  wasn't a huge issue because it was self contained within a module, but
>  we're still carrying the Secure Boot patches today.  The cgroup-v2
>  patches aren't self contained and if the upstream community continues
>  to disagree we could be setting ourselves up for another round of
>  patches we're carrying that aren't upstream.
>
>  Whether or not Fedora carries these isn't my call, but I'd approach
>  this with caution.  If the systemd developers are looking to simply
>  have a kernel to test against, perhaps reviving the kernel-playground
>  COPR with these would be enough.
>  
I echo everything Josh wrote. I'm also wary because this is adding a
userspace ABI. It's one thing to carry an in kernel work around or
set of APIs but exposing something to userspace is a bigger issue.
If the objections were "requires more testing" I'd be more willing
but it sounds like the disagreement is over design. I'd also like
to see a better long term plan than "hope the kernel community
reaches a consensus that matches with this code". Convince me this
will eventually get merged.


Thanks,
Laura