Fedora Cloud questions and proposal
I have few questions regarding how we plan to use our new Fedora Cloud:
1) Do we want to use Swift there? How much space we should allocate for Swift storage?
2) How should I configure /var/lib/nova/instances?
This is directory where RootFS, swap and ephemeral storages of instances are stored.
Right now fed-cloud09 have this path on root mount point where is 15 GB available. But it
can be extended to whole size
of "vg_server" volume group, which is 383GB.
fed-cloud1X have now:
/dev/mapper/vg_guests-nova 3.2T 32G 3.0T 2% /var/lib/nova
This gives us accumulated (over controller and 6 compute nodes) storage of 18.4 TB
storage.
However this setup does not allows live migration, because /var/lib/nova is not on shared
FS.
So I propose that /var/lib/nova/instances is NFS exported from fed-cloud09 to all compute
nodes. This will allow live
migrations of VMs. However this means that /var/lib/nova/instances on fed-cloud09 should
be big enough to host all
rootfs, swaps and ephemeral devices for all VMs in Fedora Cloud. This is 3-20 GB per
instance. so 383 GB is not enough.
Other disadvantage is that ephemeral devices will be on network storage and not so fast as
local device. However I
suppose we do not use ephemeral storage too much (maybe not at all).
I would use /dev/md127 from fed-cloud09 (2.89 TB) for /var/lib/nova. This should be enough
for approximately 287
instances. Which is cca accumulated capacity of our compute nodes. However it may slightly
limit us in future if we ever
add more compute nodes.
I would suggest to use /dev/md127 (2.89TB raid6 storage) on fed-cloud1X(s) as storage for
Swift. And configure Glance to
store images in Swift.
This have advantage that Swift can be set up to store more replicas. On the other hand
this is little bit overkill. As
we would have 3TB+ (it depend how much replicas we will use) storage for Swift. However
biggest consumer would be
Glance. And Glance in fed-cloud02 only consume 100GB currently.
What you like most, hove more space for /var/lib/nova as in current setup but without live
migrations? Or smaller
storage, which on the other hand allows live migration?
Or you have better idea how to split disks and machines to different services?
P.S. I am putting aside normal volumes for data, which will reside in DellEqualogic which
is 20TB. And it is perfectly fine.
--
Miroslav Suchy, RHCE, RHCDS
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys