libvirt configuration for cluster of qemu/kvm hosts

[Gianluca Cecchi]

Hello,
is there any pointer about how to set up a cluster of Qemu/KVM hosts?
I'm on F11 x86_64 for the hosts.
I read requirements for KVM migration in general highlighted at
http://www.linux-kvm.org/page/Migration but I keep having doubts, in
particular regarding libvirt configuration in cluster environments...
Some questions about libvirt configuration:
- /etc/libvirt/qemu/networks
do the files inside here be identical between hosts (default.xml and
other custom configured network ones)?
What are the uuid tags into the xml for? Do they have to be identical
for clusters or do they have to be absolutely different for a sort of
"identification" of host (as the term seems to suggest)?
If I have guests that use virbr0 and use dhcp for their network
config, and want two different qemu/kvm hosts able to migrate these
guests between them,  need to have same config or what?
- the same for storage configuration and other parameters.

Suppose I clone a pre-config F11 host configured with libvirt, and
suppose I have to differentiate uuid for networks/storage (see
questions above), do I have a method to regenerate these uuid values
inside xml files?

Thanks in advance for any pointer.
Gianluca

[Richard W.M. Jones]

On Mon, Jul 20, 2009 at 10:45:56AM +0200, Gianluca Cecchi wrote:
> Hello,
> is there any pointer about how to set up a cluster of Qemu/KVM hosts?
> I'm on F11 x86_64 for the hosts.

I was going to answer this in detail, but I'm not really an expert on
this.  You may be better off looking at http://ovirt.org/, which is an
open source management tool designed precisely for looking after
networks of virt hosts.  It is based on libvirt, and they have looked
at and solved many of the issues you raise.

Rich.

[Dwight Hubbard]

The question I keep bumping into is where are the cluster resource scripts for 
libvirt or kvm virtual machines.  Migration is nice, but it doesn't do much 
good in the event a server decides to up and die instead of going down 
cleanly...

On Monday 20 July 2009 01:45:56 Gianluca Cecchi wrote:
> Hello,
> is there any pointer about how to set up a cluster of Qemu/KVM hosts?
> I'm on F11 x86_64 for the hosts.
> I read requirements for KVM migration in general highlighted at
> http://www.linux-kvm.org/page/Migration but I keep having doubts, in
> particular regarding libvirt configuration in cluster environments...
> Some questions about libvirt configuration:
> /etc/libvirt/qemu/networks
>
> do the files inside here be identical between hosts (default.xml and
> other custom configured network ones)?
> What are the uuid tags into the xml for? Do they have to be identical
> for clusters or do they have to be absolutely different for a sort of
> "identification" of host (as the term seems to suggest)?
> If I have guests that use virbr0 and use dhcp for their network
> config, and want two different qemu/kvm hosts able to migrate these
> guests between them,  need to have same config or what?
> the same for storage configuration and other parameters.
>
> Suppose I clone a pre-config F11 host configured with libvirt, and
> suppose I have to differentiate uuid for networks/storage (see
> questions above), do I have a method to regenerate these uuid values
> inside xml files?
>
> Thanks in advance for any pointer.
> Gianluca
>
> Fedora-virt mailing list
> [e-mail]
> https://www.redhat.com/mailman/listinfo/fedora-virt

[Richard W.M. Jones]

On Mon, Jul 20, 2009 at 11:18:25AM -0700, Dwight Hubbard wrote:
> The question I keep bumping into is where are the cluster resource scripts for 
> libvirt or kvm virtual machines.  Migration is nice, but it doesn't do much 
> good in the event a server decides to up and die instead of going down 
> cleanly...

I guess the answer is that libvirt is just an abstraction layer above
various hypervisors.  We leave these higher level operations to
management tools, of which there are several (eg. oVirt - see my
previous email).

Rich.

[Guido Günther]

On Mon, Jul 20, 2009 at 10:45:56AM +0200, Gianluca Cecchi wrote:
> Hello,
> is there any pointer about how to set up a cluster of Qemu/KVM hosts?
> I'm on F11 x86_64 for the hosts.
> I read requirements for KVM migration in general highlighted at
> http://www.linux-kvm.org/page/Migration but I keep having doubts, in
> particular regarding libvirt configuration in cluster environments...
> Some questions about libvirt configuration:
> /etc/libvirt/qemu/networks
>
> do the files inside here be identical between hosts (default.xml and
> other custom configured network ones)?
> What are the uuid tags into the xml for? Do they have to be identical
> for clusters or do they have to be absolutely different for a sort of
> "identification" of host (as the term seems to suggest)?
> If I have guests that use virbr0 and use dhcp for their network
> config, and want two different qemu/kvm hosts able to migrate these
> guests between them,  need to have same config or what?
> the same for storage configuration and other parameters.

You can use rhcs or pacemaker for this and share the vm configuration
via e.g. gfs.
Cheers,
 -- Guido

[Gianluca Cecchi]

I' using rhcs, but I'm trying to setup a sort of actve active config.
Where I have also clvmd and where I can have for example some LVs of a
clustered VG active as disks of some VMs hosted on one server,
while other LVs of this VG represent disks hosted on another host.
Also, as I have no SAN in my test env I'm using drbd 8.3.2 in dual
primary mode...

Before starting with the cluster arch I had been done tests with
Qemu/Kvm and virt-manager using a standalone host.
Then without restarting from zero and deleting things, I transformed
my local VG into a drbd based one and I cloned the initial host

So the question arised about uuid values inside xml files... and for
example if they have to be the same for storage config and network
config or not...

I would like not to use GFS, so that I don't add another layer
impacting performance and complexity....
But your saying I can use GFS for config does implicitly mean same
config files, also in active-active config?
Gianluca

On Sat, Jul 25, 2009 at 3:17 AM, Guido Günther wrote:
> On Mon, Jul 20, 2009 at 10:45:56AM +0200, Gianluca Cecchi wrote:
> > Hello,
> > is there any pointer about how to set up a cluster of Qemu/KVM hosts?
> > I'm on F11 x86_64 for the hosts.
> > I read requirements for KVM migration in general highlighted at
> > http://www.linux-kvm.org/page/Migration but I keep having doubts, in
> > particular regarding libvirt configuration in cluster environments...
> > Some questions about libvirt configuration:
> > /etc/libvirt/qemu/networks
> >
> > do the files inside here be identical between hosts (default.xml and
> > other custom configured network ones)?
> > What are the uuid tags into the xml for? Do they have to be identical
> > for clusters or do they have to be absolutely different for a sort of
> > "identification" of host (as the term seems to suggest)?
> > If I have guests that use virbr0 and use dhcp for their network
> > config, and want two different qemu/kvm hosts able to migrate these
> > guests between them,  need to have same config or what?
> > the same for storage configuration and other parameters.
>
> You can use rhcs or pacemaker for this and share the vm configuration
> via e.g. gfs.
> Cheers,
>  -- Guido

[Guido Günther]

On Sat, Jul 25, 2009 at 12:11:40PM +0200, Gianluca Cecchi wrote:
[..snip..]
> So the question arised about uuid values inside xml files... and for
> example if they have to be the same for storage config and network
> config or not...

In principle they don't have to be the same across hosts since you can
identify the network by name and the volumes by their path but I prefer
to keep them in sync (using shared nfs in my case).
> I would like not to use GFS, so that I don't add another layer
> impacting performance and complexity....

Keeping the storge in LVM without filesystem should be fine.
> But your saying I can use GFS for config does implicitly mean same
> config files, also in active-active config?

Yes sure. I share configs across hosts and reload libvirtd's
configuration when adding new kvm vms so it picks up the new
configuration.
Hope that helps,
 -- Guido