Re: [fedora-virt] Building a new cluster of VM hosts
On Wed, Apr 01, 2015 at 10:00:43AM -0700, Scott Baker wrote:
We're in the process of building a new VM cluster that look
similar to this:
----- -----
|VM1| |VM2|
----- -----
\ NFS /
---------
|Backend|
---------
Are there any official recommendations on:
Not "official" but ...
1) NFS export/mount options
I used all defaults.
If you wish to migrate VMs, then you *must* use cache=none for the
VMs, as explained here: http://wiki.qemu.org/Migration/Storage
2) File system choice
I used ext4, but any decent fs should work fine. Avoid btrfs.
3) VM image format (qcow vs raw)
It turned out that qcow2 has an amazing susceptibility to corruption,
especially when a node crashes. So I converted my VMs to raw, because
at least if you get corruption it doesn't make the entire VM file
unreadable.
4) How to backup
rsync -av /var/lib/libvirt/images /mnt/backups
- - -
I was using nfs on my cluster[1], but I'm no longer using it. Why?
Because it's just way too slow over standard gigabit ethernet.
[1] https://rwmj.wordpress.com/2014/04/28/caseless-virtualization-cluster-par...
I also used iscsi for a while, backed with logical volumes. The iscsi
management tools are absolutely hideous, but the performance and
reliability were far better (using the precise same hardware).
- - -
Anyway, I'm now using local disks attached to each node, and just
storing my VMs there. Of course this means I can't easily migrate
VMs, but the performance is far superior. My VMs are now fully
transitioned to cloud-like "cattle", using NFS mounts for shared data
(basically NFS-mounted /home and a few other mounts), so migration
isn't really much of an issue for me now.
My cloud management software in case anyone is interested:
http://git.annexia.org/?p=mclu.git;a=summary
Rich.
--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top