Fedora Cloud questions and proposal

[Miroslav Suchý]

I have few questions regarding how we plan to use our new Fedora Cloud:

1) Do we want to use Swift there? How much space we should allocate for Swift storage?

2) How should I configure /var/lib/nova/instances?
This is directory where RootFS, swap and ephemeral storages of instances are stored.
Right now fed-cloud09 have this path on root mount point where is 15 GB available. But it can be extended to whole size
of "vg_server" volume group, which is 383GB.
fed-cloud1X have now:
 /dev/mapper/vg_guests-nova      3.2T   32G  3.0T   2% /var/lib/nova
This gives us accumulated (over controller and 6 compute nodes) storage of 18.4 TB storage.
However this setup does not allows live migration, because /var/lib/nova is not on shared FS.

So I propose that /var/lib/nova/instances is NFS exported from fed-cloud09 to all compute nodes. This will allow live
migrations of VMs. However this means that /var/lib/nova/instances on fed-cloud09 should be big enough to host all
rootfs, swaps and ephemeral devices for all VMs in Fedora Cloud. This is 3-20 GB per instance. so 383 GB is not enough.
Other disadvantage is that ephemeral devices will be on network storage and not so fast as local device. However I
suppose we do not use ephemeral storage too much (maybe not at all).
I would use /dev/md127 from fed-cloud09 (2.89 TB) for /var/lib/nova. This should be enough for approximately 287
instances. Which is cca accumulated capacity of our compute nodes. However it may slightly limit us in future if we ever
add more compute nodes.
I would suggest to use /dev/md127 (2.89TB raid6 storage) on fed-cloud1X(s) as storage for Swift. And configure Glance to
store images in Swift.
This have advantage that Swift can be set up to store more replicas. On the other hand this is little bit overkill. As
we would have 3TB+ (it depend how much replicas we will use) storage for Swift. However biggest consumer would be
Glance. And Glance in fed-cloud02 only consume 100GB currently.

What you like most, hove more space for /var/lib/nova as in current setup but without live migrations? Or smaller
storage, which on the other hand allows live migration?
Or you have better idea how to split disks and machines to different services?


P.S. I am putting aside normal volumes for data, which will reside in DellEqualogic which is 20TB. And it is perfectly fine.



-- 
Miroslav Suchy, RHCE, RHCDS
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys