Benchmarking Report (Xen Virtualiztion , Para vs Fully)

[Kiranjeet Kaur Beniwal]

Hi everyone ,

We are the students of seneca college toronto . We have developed a project which performs the benchmarking tests over Xen machine and compare Para and fully virtualiztions . Here is our final report of results of benchmarking Xen tests. We used three different tools to perform the tests and posted the data and graphs in our report.

The html format of our project is 
http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml

Any comment to out report is appreciable.

Thanks
Kiranjeet Beniwal

[Paul Wouters]

On Sat, 14 Apr 2007, Kiranjeet Kaur Beniwal wrote:
> Hi everyone ,
>
> We are the students of seneca college toronto . We have developed a project which performs the benchmarking tests over Xen machine and compare Para and fully virtualiztions . Here is our final report of results of benchmarking Xen tests. We used three different tools to perform the tests and posted the data and graphs in our report.
>
> The html format of our project is
> http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml

You only compared full vs para. Could you run the same benchmarks on the
same server in Iron mode, eg a non-xen kernel?

Most of our customers want to know the impact of "normal" vs "virtual",
which this benchmark did not report on at all.

Paul
ps. say hi to Mike for me.

[Till Maas]

On Sa April 14 2007, Kiranjeet Kaur Beniwal wrote:
> The html format of our project is
> http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml
> Any comment to out report is appreciable.

Thanks, I looked at it only superficial and one thing irritated me: you 
switched the colors for Fully/Para in the "Disk/Filesystem IO" Diagramm, 
which I guess was not intentional.

Regards,
Till

[Daniel P. Berrange]

On Sat, Apr 14, 2007 at 01:03:42PM -0400, Kiranjeet Kaur Beniwal wrote:
> Hi everyone ,
>
> We are the students of seneca college toronto . We have developed a project 
> which performs the benchmarking tests over Xen machine and compare Para and 
> fully virtualiztions . Here is our final report of results of benchmarking 
> Xen tests. We used three different tools to perform the tests and posted the
> data and graphs in our report.
>
> The html format of our project is 
> http://tux.senecac.on.ca/~mshaver/xenbench/xenbench.xhtml

The pure-compute results look reasonable, but I don't believe those results for
Disk I/O in fully-virt for a second - Section 6, Disk I/O Unixbench is the exact
opposite of expectation - were results flipped?.  Based knowledge of the architecture
of paravirt/ fullyvirt I/O subsystems, there is no way in the world fullyvirt
will ever outperform paravirt for network or disk. In previous benchmarks I've
typically seen Paravirt have x2 the disk performance of fullyvirt, and have
x10 the network performance.

I'm not familiar with UnixBench enough to say whether it is good benchmark
to use, or whether these were just runtime anomolies. I'd recommend doing
some disk I/O tests using IOZone which is the pretty common standard for
disk benchmarking.

I'd also recommend doing some scaling tests using multiple concurrent guests,
since people rarely use virtualization for just a single guest. When adding
additional concurrently executing guests I'd expect to see the gap between
paravirt and full-virt increase even further.

Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston.  [phone] -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=|

[Curtis Doty]

8:33pm Daniel P. Berrange said:
> The pure-compute results look reasonable, but I don't believe those results for
> Disk I/O in fully-virt for a second - Section 6, Disk I/O Unixbench is the exact
> opposite of expectation - were results flipped?.  Based knowledge of the architecture

And is it even fair to compare tap:aio: to file: disks without even a 
mention of the technical difference?

../C

[Daniel P. Berrange]

On Sun, Apr 15, 2007 at 06:28:10PM -0700, Curtis Doty wrote:
> 8:33pm Daniel P. Berrange said:
> > The pure-compute results look reasonable, but I don't believe those 
> > results for
> > Disk I/O in fully-virt for a second - Section 6, Disk I/O Unixbench is the 
> > exact
> > opposite of expectation - were results flipped?.  Based knowledge of the 
> > architecture
>
> And is it even fair to compare tap:aio: to file: disks without even a 
> mention of the technical difference?

It is as fair as you'll get for file-backed VMs. 'file:' for fully-virt
doesn't have the same disadvantages / problems that 'file:' does for
paravirt, since its actually better to compare 'tap:aio:' for PV with
'file:' for FV that to compare 'file:' with PV with 'file:' for FV.

It would certainly be worth doing 2 sets of disk I/O tests though, one
with file backed disks, and the other with raw partition / LVM backed
disks.

Dan.