systemd and cgroups: heads up

[Lennart Poettering]

On Thu, 26.08.10 09:59, Matthew Miller (mattdm at mattdm.org) wrote:

> 
> On Wed, Aug 25, 2010 at 10:13:05PM -0400, Daniel J Walsh wrote:
> > > Hmm, why is libcgroup pulled in by policycoreutils? What's the
> > > rationale?
> > It is used for confining sandboxes.
> 
> Having now looked at both projects, it appears to me that they are in
> conflict. They could be made to work side by side, in the same way that
> systemd's cron replacement feature doesn't necessarily mean that you can't
> run traditional crond, but there is significant overlap in terms of
> categorization policy. That is, libcgroup uses cgclassify to put stuff into
> cgroups, whereas systemd uses pam_systemd for users and creates cgroups
> automatically for services.
> 
> This overlap doesn't seem good for the distribution.

While there indeed is some overlap, both projects make a lot of sense when
used in conjunction with each other (and independently anyway).

What I mean to say with this is the following: systemd's focus when
handling cgroups is clearly on services: i.e. that there is an implicit
1:1 relationship between each service and its respective cgroup. On the
other hand libcgroup and its tools allow you to set up arbitrary group
hierarchies in a much more flexible way. Example: with systemd you will
get one group for apache, and one for mysql and one for postfix. With
libcgroup you can set up a group hierarchy that would for example
seperate the web server part from the mail server part, i.e. make mysql
and apache go in one group and make postfix go in another. This would be
a less detailed view on things. However, you can use libcgroup to make
things more fine grained too: i.e. while systemd would put all apache
processes and its cgi scripts into one cgroup, with libcgroup you could
split them up into multiple cgroups.

So, even if both systemd and libcgroup create and maintain cgroups,
their focus is certainly different.

Note that systemd even has explicit support for cgroups created by other
software such as libcgroup: by using the ControlGroup= switch in service
files you can move your services into arbitrary groups in abitrary
hierarchies, and then use for example the libcgroup tools to set limits
or other properties of these groups. While systemd can work fine
without libcgroup we carefully made sure that if you want to use them
together you can do this nicely and systemd supports you in this. 

I plan to explain how this works in a later blog story in more detail. 

Regarding the compatiblity of libcgroup and systemd right now. I see
three issues:

1) The mount point for the cgroup hierarchies has recently changed in
   the upstream kernel, from /cgroup/ to /sys/fs/cgroup/. systemd 8 now
   follows that scheme, libcgroup stil needs some updating, to use
   this mount point out of the box for its hierarchies. (bug filed)

2) systemd mounts all hierarchies exposed by the kernel by default. The
   scheme how it does that follows the default configuration libcgroup
   installs (modulo the recent /sys/fs/cgroup root dir change). We mount
   all hierarchies because we can then make them available with the
   ControlGroup= switch, way before the libcgroup init script is even
   run. systemd only really insists on its own hierarchy to be around,
   i.e. /sys/fs/cgroup/systemd/, the other hiearchies can actually be
   remounted differently later on, and moved to other places if the user
   really wants that. However, I personally see little reason to encourage
   this, for the same reasons we don't allow people to mount /sys to a
   different place even if the kernel would be fine with that. Jan
   raised the issue that mounting things like this by default would make
   it imposible to use hierarchies with more than one
   controller. However, I am not sure we want to support this, since
   firstly libcgroup makes it unnecessary to mount hierarchies like that
   because it is able to synchronize hierarchies anyway, and secondly
   this combined mounting only takes away features, and doesn't add
   any. But again, the fact that systemd makes all hierarchies available
   to you ot of the box doesn't mean you couldnt change them -- with the
   exception of the systemd hierarchy itself.

3) libcgroup currently tempers with the systemd tree in some cases,
   where it shouldn't. Dhaval already agreed to change this, and make
   sure libcgroup always leaves the systemd tree unmodified.

I hope this clears things up a little. The summary:

There's not systemd vs. libcgroup; more a systemd + libcgroup = ♥

Lennart

-- 
Lennart Poettering - Red Hat, Inc.