2:08 p.m.
https://fedoraproject.org/wiki/Changes/LibvirtModularDaemons
== Summary ==
Historically all libvirt functionality was provided in the monolithic
libvirtd daemon. Upstream has developed a new modular architecture for
libvirt where each driver is run in its own daemon. Primarily this
provides better robustness as a flaw in a secondary daemon will not
affect the QEMU daemon and vica-versa. It should also have slightly
lower host startup overhead, because only the installed hypervisor
daemon(s) will need to be fully started on boot, the other daemons can
be socket activated on demand.
== Owner ==
* Name: [[User:berrange| Daniel Berrange]]
* Email: berrange(a)redhat.com
== Detailed Description ==
Historically all libvirt functionality was provided in the monolithic
libvirtd daemon. Upstream has developed a new modular architecture for
libvirt where each driver is run in its own daemon. Daemons include
virtqemud, virtlxcd, virtnetworkd, virtnodedevd, virtsecretd, etc, one
for each libvirt stateful hypervisor driver and secondary support
driver. The new daemons listen on new socket paths and only support
UNIX sockets. For remote off-host TCP/TLS access and for backwards
compatibility with clients expecting the libvirtd UNIX socket paths, a
virtproxyd daemon is also provided. The latter accepts RPC requests
and forwards to the appropriate modular daemon. Further information
is available in the [https://libvirt.org/daemons.html Upstream
documentation]
The work will primarily involve changes in the systemd presets related
to libvirt. Currently the Fedora systemd presets enable
libvirtd.service. While libvirtd does support socket activation this
is not used during host startup, because the libvirtd daemon needs to
be fully started in order to auto-start QEMU guests that may be
configured. The current libvirtd.service uses a timeout, however, that
causes libvirtd to shutdown after 2 minutes if no VM is running.
This requirement for fully starting the service on boot still exists
in the modular daemons, however, we do not need to start all the
modular daemons on boot. It is sufficient to enable just the service
units for the hypervisor drivers QEMU, Xen and LXC. These will again
this will use a 2 minute timeout so they shut down if no
guest/container are running. Further, a default install of libvirt in
Fedora is only expected to provide the QEMU libvirt daemon.
The remaining modular daemons for libvirt non-hypervisor drivers can
exclusively rely on socket activation, because while they do provide
autostart capabilities for non-VM resources, these resources are only
needed when a VM actually starts.
The current '/usr/lib/systemd/system-preset/90-default.preset' config has
libvirtd.service
and due to dependencies, this implies libvirtd.socket,
libvirtd-ro.socket, libvirtd-admin.socket, virtlogd.socket,
virtlockd.socket also get enabled by default
The intended change is to remove libvirtd.service and instead add
virtqemud.service
virtxend.service
virtlxcd.service
virtinterfaced.socket
virtnetworkd.socket
virtnodedevd.socket
virtnwfilterd.socket
virtproxyd.socket
virtsecretd.socket
virtstoraged.socket
== Benefit to Fedora ==
The primary benefit of the modular daemons is robustness of the
management stack. There are times when libvirt code has exhibited
hangs or crashes. By having one daemon per libvirt driver, the impact
of these bugs is limited. The most important benefit of this is that a
bug in the node device / storage daemons will not impact management of
any running QEMU VMs.
The secondary benefit of the modular daemons is improved host startup
overhead. When the libvirtd daemon starts, all drivers are initialized
and this can take a significant amount of CPU time. This is
particularly true for the node device driver as it extracts
information about all host hardware.
The third potential future benefit is that it enables development of
more useful SELinux policy. The current policy for libvirtd is no
better than unconfined due to the broad range of features that must be
allowed. At least some of the modular daemons can potentially have
useful SELinux policy written. Note there is no active work on this
aspect.
== Scope ==
* Proposal owners:
Rebuild libvirt to prefer connecting to modular daemons by default
Submit a patch updating 90-defaults.preset in the fedora-release
package to enable modular daemons by default
* Other developers:
Review the proposal for fedora-release preset change
* Release engineering: N/A
* Policies and guidelines: N/A
* Trademark approval: N/A
* Alignment with Objectives: N/A
== Upgrade/compatibility impact ==
Existing Fedora installs using libvirtd will be unaffected by the
change in presets and will continue to use monolithic libvirtd. To
opt-in to using the new modular daemons follow instructions at
https://libvirt.org/daemons.html#switching-to-modular-daemons
New Fedora installs will get modular daemons instead of libvirtd. This
is intended to be functionally transparent to applications using
libvirt APIs.
This may have an impact on automation tools such as ansible / puppet
which have been written to manage libvirtd. These will need adapting
to manage the new modular daemons, or the host can be re-configured to
use libvirtd again in the short term by applying the inverse of the
logic at https://libvirt.org/daemons.html#switching-to-modular-daemons
Administrators who wish to enable TCP/TLS sockets for remote access to
libvirt, for the sake of migration, will need to configure this using
the virtproxyd daemon systemd socket units instead of libvirtd units.
At some point in the future it is intended that the monolithic
libvirtd will be entirely removed by libvirt upstream. This is
predicated on prolonged successful deployment of the modular daemons
in one or more major distros. Thus the likely timeframe is Fedora 37
at the earliest (i.e. min 1 year from successful usage of modular
daemons in a distro).
== How To Test ==
Hardware:
No special hardware is required for testing, since QEMU can run in
emulated mode if hardware virtualization is not available
Preparation:
Deploy a completely fresh Fedora install, since systemd preset changes
only take effect on initial installation. Ensure that virtualization
is installed with QEMU/KVM.
Actions:
Boot the system and observice that 'virtqemud' is running immediately
on completion of boot up with an arg of '--timeout=120'.
After two minutes the daemon should exit, and will automatically
restart if 'virsh -c qemu:///system list' is run as root.
Deploy a new virtual machine using virt-install / virt-manager.
Set the new VM to autostart, eg 'virsh -c qemu:///system autostart $GUESTNAME'
Shutdown the guest and reboot the host
Upon completion of boot up observe that 'virtqemud' is running, and
also a QEMU guest is running. 'virtqemud' should remain running for as
long as the guest is running.
The 'virtnetworkd' will also be running if the guest is connected to
the 'virbr0' virtual network device provided by libvirt.
== User Experience ==
Users of libvirt applications should not see any functional changes in
their experience, since the new modular daemons are intended to be
fully backwards compatible with existing libvirtd daemon.
Administrators configuring hosts, however, will need to be aware that
a different set of libvirt daemons will be running than they have seen
in the past. See upgrade impact section for likely impact on
administrators experience.
There may be a small improvement in boot time performance when libvirt
is installed.
== Dependencies ==
The fedora-release package needs updating with the new systemd presets.
== Contingency Plan ==
* Contingency mechanism:
Should unexpected problems with the change arise, the changes in
libvirt and fedora-release will be reverted to the historical known
good behaviour of libvirt in previous Fedora.
* Contingency deadline: Beta freeze
* Blocks release? Yes
== Documentation ==
Primary upstream guidance is https://libvirt.org/daemons.html
== Release Notes ==
The libvirt package has switched to using modular daemons by default.
The monolithic libvirtd daemon will no longer be started. In its place
a hypervisor specific daemon will be started, along with one or more
secondary driver daemons.
Applications using libvirt APIs should not notice any difference in behaviour.
Administrators configuring a host manually, or using automation tools
such as ansible/puppet, may need to take account of the new daemons
that are providing libvirt functionality.
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis