On Wed, Apr 01, 2015 at 10:00:43AM -0700, Scott Baker wrote:
We're in the process of building a new VM cluster that look similar to this:
|VM1| |VM2|
\ NFS /
|Backend|
Are there any official recommendations on:
Not "official" but ...
- NFS export/mount options
I used all defaults.
If you wish to migrate VMs, then you *must* use cache=none for the VMs, as explained here: http://wiki.qemu.org/Migration/Storage
- File system choice
I used ext4, but any decent fs should work fine. Avoid btrfs.
- VM image format (qcow vs raw)
It turned out that qcow2 has an amazing susceptibility to corruption, especially when a node crashes. So I converted my VMs to raw, because at least if you get corruption it doesn't make the entire VM file unreadable.
- How to backup
rsync -av /var/lib/libvirt/images /mnt/backups
- - -
I was using nfs on my cluster[1], but I'm no longer using it. Why? Because it's just way too slow over standard gigabit ethernet.
[1] https://rwmj.wordpress.com/2014/04/28/caseless-virtualization-cluster-part-5...
I also used iscsi for a while, backed with logical volumes. The iscsi management tools are absolutely hideous, but the performance and reliability were far better (using the precise same hardware).
- - -
Anyway, I'm now using local disks attached to each node, and just storing my VMs there. Of course this means I can't easily migrate VMs, but the performance is far superior. My VMs are now fully transitioned to cloud-like "cattle", using NFS mounts for shared data (basically NFS-mounted /home and a few other mounts), so migration isn't really much of an issue for me now.
My cloud management software in case anyone is interested:
http://git.annexia.org/?p=mclu.git;a=summary
Rich.