[fedora-virt] Virt Disk Performance

[Richard W.M. Jones]

On Mon, Oct 31, 2011 at 04:26:33PM +0100, Kevin Wolf wrote:
> Am 31.10.2011 16:07, schrieb Richard W.M. Jones:
> > On Mon, Oct 31, 2011 at 10:18:50AM +0100, Kevin Wolf wrote:
> >> Am 30.10.2011 21:59, schrieb Richard W.M. Jones:
> >>>> I've pulled most of the information you requested. See above. I
> >>>> don't see any specific cache settings on either machine. Is there
> >>>> somewhere I would see the default setting on the machine?
> >>>
> >>> I'm out of the loop on what the default caching policy is these days.
> >>> Hopefully libvirt is at least choosing a safe one.
> >>
> >> cache=writethrough is the default. And indeed this is the safest option,
> >> and at the same time by far the slowest option. I would expect that you
> >> get much better write performance with cache=none.
> >>
> >> Note however that older guests OSes cannot deal correctly with disks
> >> that have a volatile write cache, like cache=none emulates. This means
> >> that in case of a host crash and with some bad luck, your guest might
> >> experience file system corruption.
> >>
> >> Recent guest OSes are safe in this respect and can be used with
> >> cache=none or cache=writeback in order to improve performance without
> >> any such risk.
> > 
> > Just FYI, here is how libvirt XML types are mapped to qemu types,
> > found by examining the libvirt code:
> > 
> >   libvirt          old qemu      modern qemu
> >   
> >   none             cache=off     cache=none
> >   writeback        cache=on      cache=writeback
> >   writethrough     cache=off     cache=writethrough
> >   directsync       cache=off     cache=directsync (if supported by qemu)
> >   unsafe           cache=off     cache=unsafe (if supported by qemu)
> 
> cache=off is a strange fallback for unsafe...

(CC'd to Dan)

That was based on my reading of the libvirt code, but it might be
wrong.  In any case that would only apply to some very old version of
qemu.

> > I tested this to see what libvirt would actually use with my fairly
> > recent qemu-kvm, and what sort of timings I would get:
> 
> What does fairly recent mean? cache=unsafe is supported since 0.13. If
> it doesn't work this might indicate a libvirt bug.

libvirt seems to think that cache=unsafe isn't supported.  However it
clearly is in my version of qemu:

$ qemu-kvm -help | grep cache
       [,cache=writethrough|writeback|none|unsafe][,format=f]

qemu-0.15.0-5.fc17.x86_64

But:

$ sudo virsh edit F16x64
error: internal error unknown disk cache mode 'unsafe'

I need to dig into libvirt code a bit deeper to see what this means.

> >   libvirt          MB/s          modern qemu command line
> >   
> >   (no setting)     29            (nothing)
> >   none             39            cache=none
> >   writeback        24*           cache=writeback
> >   writethrough     28            cache=writethrough
> >   directsync       [did not work for me -- "unknown disk cache mode"]
> >   unsafe           [did not work for me -- "unknown disk cache mode"]
> > 
> > * = large variability in this result
> > 
> > What's also interesting is that performance today with no cache
> > setting is slower than it was yesterday, even though I'm not running
> > anything significantly different on my laptop.
> > 
> > Kevin, I'm using 'dd ... conv=fsync' for these tests.  What's a good,
> > reliable method for testing read and write speeds?
> 
> Depends on what you want to test, obviously. conv=fsync will be very
> much in favour of cache=writethrough because you force all other caching
> options to do the disk flushes as well that make writethrough so slow.
> 
> When I use dd for testing, I usually use oflag=direct to bypass the
> guest's page cache, but don't issue any fsyncs. You should see that the
> writeback modes (none/writeback/unsafe) performs quite a bit better with
> that, while I'd expect writethrough to stay on about the same level.

OK, that's useful to know.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top