10:47 a.m.
[Is there a Fedora-specific systemd list? Not that I can find.]
We'd like to request that virtio-serial devices (/dev/virtio-ports/*)
are tagged so we can use them as systemd devices. Dan Berrange thinks
the right incantation is to add:
SUBSYSTEM=="virtio-ports", TAG+="systemd"
to /lib/udev/rules.d/99-systemd.rules
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
10:54 a.m.
On Tue, Jul 05, 2011 at 04:47:18PM +0100, Richard W.M. Jones wrote:
[Is there a Fedora-specific systemd list? Not that I can find.]
We'd like to request that virtio-serial devices (/dev/virtio-ports/*)
are tagged so we can use them as systemd devices. Dan Berrange thinks
the right incantation is to add:
SUBSYSTEM=="virtio-ports", TAG+="systemd"
to /lib/udev/rules.d/99-systemd.rules
The goal is that we want to automagically run /usr/sbin/guestfsd
when /dev/virtio-ports/org.libguestfs.channel.0 is present. For
this I believe we need to have a '.device' unit for the virtio-port
device populated from the above udev rule, so we can in turn have a
guestfsd.service unit looking like:
[Unit]
Description=libguestfs management daemon
BindTo=dev-virtio\x2dports-org.libguestfs.channel.0.device
After=dev-virtio\x2dports-org.libguestfs.channel.0.device
[Service]
ExecStart=-/usr/sbin/guestfsd
Restart=always
RestartSec=0
[Install]
WantedBy=multi-user.target
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
1:36 p.m.
On Tue, 05.07.11 16:54, Daniel P. Berrange (berrange(a)redhat.com) wrote:
Heya,
On Tue, Jul 05, 2011 at 04:47:18PM +0100, Richard W.M. Jones wrote:
> [Is there a Fedora-specific systemd list? Not that I can find.]
>
> We'd like to request that virtio-serial devices (/dev/virtio-ports/*)
> are tagged so we can use them as systemd devices. Dan Berrange thinks
> the right incantation is to add:
>
> SUBSYSTEM=="virtio-ports", TAG+="systemd"
>
> to /lib/udev/rules.d/99-systemd.rules
The goal is that we want to automagically run /usr/sbin/guestfsd
when /dev/virtio-ports/org.libguestfs.channel.0 is present. For
this I believe we need to have a '.device' unit for the virtio-port
device populated from the above udev rule, so we can in turn have a
guestfsd.service unit looking like:
I think adding such a rule to systemd's unit file is not a bad idea, but
since the use case here is very specific to your app, another option
would be to add an app-specific udev rule for this. (See below)
[Unit]
Description=libguestfs management daemon
BindTo=dev-virtio\x2dports-org.libguestfs.channel.0.device
After=dev-virtio\x2dports-org.libguestfs.channel.0.device
[Service]
ExecStart=-/usr/sbin/guestfsd
Prefixing the binary path with "-" will result in the exit code of
guestfsd be ignored, i.e. we wouldn't put the service into failed state
if it crashes (or exits otherwise abnormally). I'd encourage never to
prefix with "-" unless you have a really good reason to.
Restart=always
RestartSec=0
[Install]
WantedBy=multi-user.target
If you use "WantedBy=multi-user.target" then this unit would be started
at boot, and delayed until the device in question shows up. If it never
shows up (for example because you boot on bare metal), then it would
have to timeout, which wouldn't be particularly pretty.
I wonder if it wouldn't be nicer to use the device showing up as _only_
trigger to spawn the service. This would be nicer I think because it
would spawn the service only if run in a VM. If you run the machine on
bare metal, then it wouldn't be started at all, and would not have to
timeout. (And if I grok this properly, then the virtio serial ports are
even hotpluggable, which makes this even more interesting) To implement
a scheme like that, you'd just ship a udev rules file which you install
to /lib/udev/rules/99-guestfs.rules with contents
like this:
SUBSYSTEM=="virtio-ports", ATTR{name}=="org.libguestfs.channel.0",
TAG+="systemd", ENV{SYSTEMD_WANTS}="guestfsd.service"
(untested, I hope the match is right)
It's up to you whether you prefer the "spawn unconditionally but wait
for the device to show up" approach or my suggestion of "spawn if and
when the device shows up".
Either way I have now added to git a patch that marks virtio ports for
exposure in systemd, by marking them with "systemd".
Lennart
--
Lennart Poettering - Red Hat, Inc.
1:40 p.m.
On Tue, 05.07.11 20:36, Lennart Poettering (mzerqung(a)0pointer.de) wrote:
> > We'd like to request that virtio-serial devices
(/dev/virtio-ports/*)
> > are tagged so we can use them as systemd devices. Dan Berrange thinks
> > the right incantation is to add:
> >
> > SUBSYSTEM=="virtio-ports", TAG+="systemd"
> >
> > to /lib/udev/rules.d/99-systemd.rules
>
> The goal is that we want to automagically run /usr/sbin/guestfsd
> when /dev/virtio-ports/org.libguestfs.channel.0 is present. For
> this I believe we need to have a '.device' unit for the virtio-port
> device populated from the above udev rule, so we can in turn have a
> guestfsd.service unit looking like:
I think adding such a rule to systemd's unit file is not a bad idea, but
since the use case here is very specific to your app, another option
would be to add an app-specific udev rule for this. (See below)
Oops, wanted to write "systemd's udev rules file" here, and not
"systemd's unit file".
Either way I have now added to git a patch that marks virtio ports
for
exposure in systemd, by marking them with "systemd".
So, I am a bit confused now after reading this:
http://fedoraproject.org/wiki/Features/VirtioSerial
How does /dev/hvcxxx relate to these virtio ports?
hvc ports are tagged "systemd" anyway in udev, so if this is the same
thing, why do we have to tag the virtio ports too?
Can you explain how hvc and virtio relate to each other and under which
kernel device names they show up in udev and how they correspond?
Lennart
--
Lennart Poettering - Red Hat, Inc.
4:22 a.m.
On Tue, Jul 05, 2011 at 08:40:55PM +0200, Lennart Poettering wrote:
On Tue, 05.07.11 20:36, Lennart Poettering (mzerqung(a)0pointer.de)
wrote:
> > > We'd like to request that virtio-serial devices (/dev/virtio-ports/*)
> > > are tagged so we can use them as systemd devices. Dan Berrange thinks
> > > the right incantation is to add:
> > >
> > > SUBSYSTEM=="virtio-ports", TAG+="systemd"
> > >
> > > to /lib/udev/rules.d/99-systemd.rules
> >
> > The goal is that we want to automagically run /usr/sbin/guestfsd
> > when /dev/virtio-ports/org.libguestfs.channel.0 is present. For
> > this I believe we need to have a '.device' unit for the virtio-port
> > device populated from the above udev rule, so we can in turn have a
> > guestfsd.service unit looking like:
>
> I think adding such a rule to systemd's unit file is not a bad idea, but
> since the use case here is very specific to your app, another option
> would be to add an app-specific udev rule for this. (See below)
Oops, wanted to write "systemd's udev rules file" here, and not
"systemd's unit file".
> Either way I have now added to git a patch that marks virtio ports for
> exposure in systemd, by marking them with "systemd".
So, I am a bit confused now after reading this:
http://fedoraproject.org/wiki/Features/VirtioSerial
How does /dev/hvcxxx relate to these virtio ports?
hvc ports are tagged "systemd" anyway in udev, so if this is the same
thing, why do we have to tag the virtio ports too?
Can you explain how hvc and virtio relate to each other and under which
kernel device names they show up in udev and how they correspond?
In QEMU, there is one PCI device "virtio-serial-pci". In QEMU's view this
device is a bus to which we then attach zero or more 'virtconsole' or
'virtserialport' devices.
The 'virtconsole' device is a paravirt text console, which appears as
/dev/hvcNNN in the guest. These are intended for interactive login and
would be expected to run a mingetty/agetty process.
You can enable this in a libvirt guest by changing the default
<console type='pty'/>
To
<console type='pty'>
<target type='virtio'/>
</console>
Which should create a /dev/hvc0 in the guest
The 'virtserialport' device is a paravirt serial port (though it has
no baud/parity settings). These are basically reliable, bi-directional
data channels between host & guest. These are intended for running
guest agents which need to communicate with some host service. These
appear as unstable named devices /dev/vportNNNN, and also stable named
symlinks /dev/virt-port/UNIQUE-CHANNEL-NAME
You can enables these in a libvirt guest by adding devices like
<channel type='unix'>
<source mode='connect'
path='/var/lib/libvirt/myguest-org.myservice'/>
<target type='virtio' name='org.myservice'/>
</channel>
This should create a character device /dev/virtio-port/org.myservice in
the guest, which is connected to a UNIX socket
'/var/lib/libvirt/myguest-org.myservice'
in the host OS.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
7:27 a.m.
On Wed, 06.07.11 10:22, Daniel P. Berrange (berrange(a)redhat.com) wrote:
> So, I am a bit confused now after reading this:
>
> http://fedoraproject.org/wiki/Features/VirtioSerial
>
> How does /dev/hvcxxx relate to these virtio ports?
>
> hvc ports are tagged "systemd" anyway in udev, so if this is the same
> thing, why do we have to tag the virtio ports too?
>
> Can you explain how hvc and virtio relate to each other and under which
> kernel device names they show up in udev and how they correspond?
In QEMU, there is one PCI device "virtio-serial-pci". In QEMU's view this
device is a bus to which we then attach zero or more 'virtconsole' or
'virtserialport' devices.
The 'virtconsole' device is a paravirt text console, which appears as
/dev/hvcNNN in the guest. These are intended for interactive login and
would be expected to run a mingetty/agetty process.
Hmm, but Unix does not really distuingish between serial ports and
ttys. So what precisely is the difference in behaviour when I have
opened a /dev/hvcXX and a /dev/vportXXX? What happens if start a getty
on /dev/vportXXX? And why couldn't I use /dev/hvcXXX for fast data
transfer, too?
Or is the only difference on the host side? i.e. the destinction between
access via pty and access via sockets?
Lennart
--
Lennart Poettering - Red Hat, Inc.
7:49 a.m.
On Wed, Jul 06, 2011 at 02:27:58PM +0200, Lennart Poettering wrote:
On Wed, 06.07.11 10:22, Daniel P. Berrange (berrange(a)redhat.com)
wrote:
> > So, I am a bit confused now after reading this:
> >
> > http://fedoraproject.org/wiki/Features/VirtioSerial
> >
> > How does /dev/hvcxxx relate to these virtio ports?
> >
> > hvc ports are tagged "systemd" anyway in udev, so if this is the
same
> > thing, why do we have to tag the virtio ports too?
> >
> > Can you explain how hvc and virtio relate to each other and under which
> > kernel device names they show up in udev and how they correspond?
>
> In QEMU, there is one PCI device "virtio-serial-pci". In QEMU's view
this
> device is a bus to which we then attach zero or more 'virtconsole' or
> 'virtserialport' devices.
>
> The 'virtconsole' device is a paravirt text console, which appears as
> /dev/hvcNNN in the guest. These are intended for interactive login and
> would be expected to run a mingetty/agetty process.
Hmm, but Unix does not really distuingish between serial ports and
ttys. So what precisely is the difference in behaviour when I have
opened a /dev/hvcXX and a /dev/vportXXX? What happens if start a getty
on /dev/vportXXX? And why couldn't I use /dev/hvcXXX for fast data
transfer, too?
There's not really any functional difference between them once open.
The difference is is really about providing hints to the guest OS as
to how to configure the devices "out of the box"
The goal for the virtconsole devices was that a guest OS can just
automatically start a getty on every /dev/hvcXXX device it finds.
So a host admin can just add console devices to a guest config and
get a login without any extra guest OS configuration work.
The virtserial devices meanwhile have a admin defined name associated
with them to (in my example 'org.libguestfs.channel.0') so guest OS
agents can have a predictable way to identify which ports have been
exposed from the host for their use.
eg, consider if two guest agents 'guestfsd' and 'matahari' both wanted
to run in a guest and we were using just /dev/hvcXXX devices. There'd
be no way for them to decide which of /dev/hvc0, /dev/hvc1 applied
was theirs. With virtserial devices having a stable name, they can see
which is which device immediately.
Or is the only difference on the host side? i.e. the destinction
between
access via pty and access via sockets?
No, that was just two example configs - the range of host side config
options is the same for both, with exception that virtserial devices
have an additional 'name' associated with them which maps to the device
/dev/virtio-ports/$NAME in Linux guests.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
4:23 a.m.
On Tue, Jul 05, 2011 at 08:36:42PM +0200, Lennart Poettering wrote:
On Tue, 05.07.11 16:54, Daniel P. Berrange (berrange(a)redhat.com)
wrote:
Heya,
> On Tue, Jul 05, 2011 at 04:47:18PM +0100, Richard W.M. Jones wrote:
> > [Is there a Fedora-specific systemd list? Not that I can find.]
> >
> > We'd like to request that virtio-serial devices (/dev/virtio-ports/*)
> > are tagged so we can use them as systemd devices. Dan Berrange thinks
> > the right incantation is to add:
> >
> > SUBSYSTEM=="virtio-ports", TAG+="systemd"
> >
> > to /lib/udev/rules.d/99-systemd.rules
>
> The goal is that we want to automagically run /usr/sbin/guestfsd
> when /dev/virtio-ports/org.libguestfs.channel.0 is present. For
> this I believe we need to have a '.device' unit for the virtio-port
> device populated from the above udev rule, so we can in turn have a
> guestfsd.service unit looking like:
I think adding such a rule to systemd's unit file is not a bad idea, but
since the use case here is very specific to your app, another option
would be to add an app-specific udev rule for this. (See below)
> [Unit]
> Description=libguestfs management daemon
> BindTo=dev-virtio\x2dports-org.libguestfs.channel.0.device
> After=dev-virtio\x2dports-org.libguestfs.channel.0.device
>
> [Service]
> ExecStart=-/usr/sbin/guestfsd
Prefixing the binary path with "-" will result in the exit code of
guestfsd be ignored, i.e. we wouldn't put the service into failed state
if it crashes (or exits otherwise abnormally). I'd encourage never to
prefix with "-" unless you have a really good reason to.
> Restart=always
> RestartSec=0
>
> [Install]
> WantedBy=multi-user.target
If you use "WantedBy=multi-user.target" then this unit would be started
at boot, and delayed until the device in question shows up. If it never
shows up (for example because you boot on bare metal), then it would
have to timeout, which wouldn't be particularly pretty.
Yeah, we don't really want a timeout on bare metal, or if the guest
doesn't have the device enabled.
I wonder if it wouldn't be nicer to use the device showing up as
_only_
trigger to spawn the service. This would be nicer I think because it
would spawn the service only if run in a VM. If you run the machine on
bare metal, then it wouldn't be started at all, and would not have to
timeout. (And if I grok this properly, then the virtio serial ports are
even hotpluggable, which makes this even more interesting) To implement
a scheme like that, you'd just ship a udev rules file which you install
to /lib/udev/rules/99-guestfs.rules with contents
like this:
SUBSYSTEM=="virtio-ports", ATTR{name}=="org.libguestfs.channel.0",
TAG+="systemd", ENV{SYSTEMD_WANTS}="guestfsd.service"
(untested, I hope the match is right)
It's up to you whether you prefer the "spawn unconditionally but wait
for the device to show up" approach or my suggestion of "spawn if and
when the device shows up".
We do in fact want the latter 'spawn if and when the device shows up',
so we'll have a go with the udev rule you suggest above.
Regards,
Daniel
--
|: http://berrange.com -o- http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o- http://virt-manager.org :|
|: http://autobuild.org -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o- http://live.gnome.org/gtk-vnc :|
6:21 a.m.
On Tue, Jul 05, 2011 at 08:36:42PM +0200, Lennart Poettering wrote:
On Tue, 05.07.11 16:54, Daniel P. Berrange (berrange(a)redhat.com)
wrote:
> [Service]
> ExecStart=-/usr/sbin/guestfsd
Prefixing the binary path with "-" will result in the exit code of
guestfsd be ignored, i.e. we wouldn't put the service into failed state
if it crashes (or exits otherwise abnormally). I'd encourage never to
prefix with "-" unless you have a really good reason to.
In this case, I think we do. The daemon only handles one connection
at a time (that is the nature of virtio-serial ports) and it will exit
with EXIT_FAILURE if an error is read in the protocol. This can
happen in some legitimate-ish cases, eg. if the host side disconnects
without "properly" closing the connection.
But what we'd want to avoid is the case where the daemon dies during
startup, and we get into a loop repeatedly relaunching the daemon.
The question is, does systemd implement respawn throttling like inetd?
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top
6:37 a.m.
On 07/06/2011 11:21 AM, Richard W.M. Jones wrote:
On Tue, Jul 05, 2011 at 08:36:42PM +0200, Lennart Poettering wrote:
> On Tue, 05.07.11 16:54, Daniel P. Berrange (berrange(a)redhat.com) wrote:
>> [Service]
>> ExecStart=-/usr/sbin/guestfsd
> Prefixing the binary path with "-" will result in the exit code of
> guestfsd be ignored, i.e. we wouldn't put the service into failed state
> if it crashes (or exits otherwise abnormally). I'd encourage never to
> prefix with "-" unless you have a really good reason to.
In this case, I think we do. The daemon only handles one connection
at a time (that is the nature of virtio-serial ports) and it will exit
with EXIT_FAILURE if an error is read in the protocol. This can
happen in some legitimate-ish cases, eg. if the host side disconnects
without "properly" closing the connection.
But what we'd want to avoid is the case where the daemon dies during
startup, and we get into a loop repeatedly relaunching the daemon.
The question is, does systemd implement respawn throttling like inetd?
Hum
Would Restart= not suffice as in..
Restart=on-failure
Or
Restart=on-abort
from man systemd.service
Restart=
Configures whether the main service process shall be
restarted when it exits. Takes one of no, on-success, on-failure,
on-abort or always. If set to no (the default) the service
will not be restarted when it exits. If set to on-success it will
be restarted only when it exited cleanly, i.e. terminated
with an exit code of 0. If set to on-failure it will be restarted
only when it exited with an exit code not equalling 0, or
when terminated by a signal. If set to on-abort it will be
restarted only if it exits due to reception of an uncaught
signal. If set to always the service will be restarted regardless
whether it exited cleanly or not, or got terminated
abnormally by a signal
JBG
7:35 a.m.
On Wed, 06.07.11 12:21, Richard W.M. Jones (rjones(a)redhat.com) wrote:
On Tue, Jul 05, 2011 at 08:36:42PM +0200, Lennart Poettering wrote:
> On Tue, 05.07.11 16:54, Daniel P. Berrange (berrange(a)redhat.com) wrote:
> > [Service]
> > ExecStart=-/usr/sbin/guestfsd
>
> Prefixing the binary path with "-" will result in the exit code of
> guestfsd be ignored, i.e. we wouldn't put the service into failed state
> if it crashes (or exits otherwise abnormally). I'd encourage never to
> prefix with "-" unless you have a really good reason to.
In this case, I think we do. The daemon only handles one connection
at a time (that is the nature of virtio-serial ports) and it will exit
with EXIT_FAILURE if an error is read in the protocol. This can
happen in some legitimate-ish cases, eg. if the host side disconnects
without "properly" closing the connection.
Wouldn't it make sense to fix the daemon to return EXIT_SUCCESS in those
legitimate-ish cases?
But what we'd want to avoid is the case where the daemon dies
during
startup, and we get into a loop repeatedly relaunching the daemon.
systemd does not distuingish between failure-on-start and
failure-during-runtime.
The question is, does systemd implement respawn throttling like
inetd?
Yes, we rate limit almost everything. There's even a patch floating
around on our mailing list right now which we will probably merge pretty
soon which allows you to configure in a service how often we should
retry respawning the service before giving up, if it fails.
Lennart
--
Lennart Poettering - Red Hat, Inc.
8:35 a.m.
On Wed, Jul 06, 2011 at 02:35:13PM +0200, Lennart Poettering wrote:
Wouldn't it make sense to fix the daemon to return EXIT_SUCCESS
in those
legitimate-ish cases?
It's probably just me but I find code that does this slightly
repugnant:
if (some_syscall() == -1) {
perror ("some_syscall");
exit (EXIT_SUCCESS);
}
There are a large number of places in the code which we would need to
examine and possibly change.
I think as long as systemd is doing rate-limiting, we should be fine.
The daemon is heavily tested anyway and hasn't so far failed at
start-up time ...
Anyway, I'm going to test it.
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top
6:28 a.m.
I haven't tested this yet .. will do later today.
But comments welcome on:
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=99-guestf...
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=guestfsd....
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=libguestf...
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=libguestf...
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=libguestf...
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
New in Fedora 11: Fedora Windows cross-compiler. Compile Windows
programs, test, and build Windows installers. Over 70 libraries supprt'd
http://fedoraproject.org/wiki/MinGW http://www.annexia.org/fedora_mingw
7:31 a.m.
On Wed, 06.07.11 12:28, Richard W.M. Jones (rjones(a)redhat.com) wrote:
I haven't tested this yet .. will do later today.
But comments welcome on:
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=99-guestf...
http://pkgs.fedoraproject.org/gitweb/?p=libguestfs.git;a=blob;f=guestfsd....
You can drop the [Install] line, too.
Looks good!
Lennart
--
Lennart Poettering - Red Hat, Inc.
4702
days inactive
4703
days old
13 comments
4 participants
participants (4)
-
Daniel P. Berrange -
Jóhann B. Guðmundsson -
Lennart Poettering -
Richard W.M. Jones