We're in the process of building a new VM cluster that look similar to this:
----- ----- |VM1| |VM2| ----- ----- \ NFS / --------- |Backend| ---------
Are there any official recommendations on:
1) NFS export/mount options 2) File system choice 3) VM image format (qcow vs raw) 4) How to backup
On 04/01/2015 01:00 PM, Scott Baker wrote:
We're in the process of building a new VM cluster that look similar to this:
|VM1| |VM2|
\ NFS /
|Backend|
Are there any official recommendations on:
- NFS export/mount options
- File system choice
- VM image format (qcow vs raw)
- How to backup
If there's any specific document covering that usecase, I don't know about it. Sorry.
Personally I haven't heard about any specific NFS mount options or FS choice/options that are catered expressly to virt. Well, besides 'don't use btrfs' :)
Use qcow2 if you care about any of the specific qcow2 features like internal snapshots or overlay files. Otherwise use raw to maximize performance.
For backup, you can use live disk snapshots to get manual live backup:
https://fedoraproject.org/wiki/Features/Virt_Live_Snapshots
But if you don't want to worry about all these details, let a higher level tool handle it for you and check out ovirt. Maybe it's overkill for your needs, but they have all this stuff figured out already.
- Cole
On Wed, Apr 01, 2015 at 10:00:43AM -0700, Scott Baker wrote:
We're in the process of building a new VM cluster that look similar to this:
|VM1| |VM2|
\ NFS /
|Backend|
Are there any official recommendations on:
Not "official" but ...
- NFS export/mount options
I used all defaults.
If you wish to migrate VMs, then you *must* use cache=none for the VMs, as explained here: http://wiki.qemu.org/Migration/Storage
- File system choice
I used ext4, but any decent fs should work fine. Avoid btrfs.
- VM image format (qcow vs raw)
It turned out that qcow2 has an amazing susceptibility to corruption, especially when a node crashes. So I converted my VMs to raw, because at least if you get corruption it doesn't make the entire VM file unreadable.
- How to backup
rsync -av /var/lib/libvirt/images /mnt/backups
- - -
I was using nfs on my cluster[1], but I'm no longer using it. Why? Because it's just way too slow over standard gigabit ethernet.
[1] https://rwmj.wordpress.com/2014/04/28/caseless-virtualization-cluster-part-5...
I also used iscsi for a while, backed with logical volumes. The iscsi management tools are absolutely hideous, but the performance and reliability were far better (using the precise same hardware).
- - -
Anyway, I'm now using local disks attached to each node, and just storing my VMs there. Of course this means I can't easily migrate VMs, but the performance is far superior. My VMs are now fully transitioned to cloud-like "cattle", using NFS mounts for shared data (basically NFS-mounted /home and a few other mounts), so migration isn't really much of an issue for me now.
My cloud management software in case anyone is interested:
http://git.annexia.org/?p=mclu.git;a=summary
Rich.