On Fri, 10 Apr 2015 09:01:58 +0200
Miroslav Suchý <msuchy(a)redhat.com> wrote:
I have few questions regarding how we plan to use our new Fedora
Cloud:
1) Do we want to use Swift there? How much space we should allocate
for Swift storage?
I think it might be a good idea to have some swift space setup, but I
am not sure what use cases we fully have for it, so I would say it
should be somewhat small. 100GB or something?
This would also be backed by the equalogics? Or would it be
distributed on the nodes?
2) How should I configure /var/lib/nova/instances?
This is directory where RootFS, swap and ephemeral storages of
instances are stored. Right now fed-cloud09 have this path on root
mount point where is 15 GB available. But it can be extended to whole
size of "vg_server" volume group, which is 383GB. fed-cloud1X have
now: /dev/mapper/vg_guests-nova 3.2T 32G 3.0T
2% /var/lib/nova This gives us accumulated (over controller and 6
compute nodes) storage of 18.4 TB storage. However this setup does
not allows live migration, because /var/lib/nova is not on shared FS.
So I propose that /var/lib/nova/instances is NFS exported from
fed-cloud09 to all compute nodes. This will allow live migrations of
VMs. However this means that /var/lib/nova/instances on fed-cloud09
should be big enough to host all rootfs, swaps and ephemeral devices
for all VMs in Fedora Cloud. This is 3-20 GB per instance. so 383 GB
is not enough. Other disadvantage is that ephemeral devices will be
on network storage and not so fast as local device. However I suppose
we do not use ephemeral storage too much (maybe not at all). I would
use /dev/md127 from fed-cloud09 (2.89 TB) for /var/lib/nova. This
should be enough for approximately 287 instances. Which is cca
accumulated capacity of our compute nodes. However it may slightly
limit us in future if we ever add more compute nodes.
I guess that would be ok. Or could we also use the equalogics for this?
Make say a 5TB volume for this? That would limit volume space however.
Another option would be gluster, but thats going to add more complexity
for sure.
One other option: we have another smaller equalogics we could use for
instances perhaps. It has about 1/2 the space of the current one in the
cloud, but that still might be better if we could use it instead of
local?
I would suggest
to use /dev/md127 (2.89TB raid6 storage) on fed-cloud1X(s) as storage
for Swift. And configure Glance to store images in Swift. This have
advantage that Swift can be set up to store more replicas. On the
other hand this is little bit overkill. As we would have 3TB+ (it
depend how much replicas we will use) storage for Swift. However
biggest consumer would be Glance. And Glance in fed-cloud02 only
consume 100GB currently.
Yeah, we really don't have too many images. On the old/existing cloud
we used the head node (fed-cloud02) and then added one of the compute
nodes to cinder (fed-cloud08 I think) to add more room. But thats not
the part we a limited on in the new cloud. ;)
What you like most, hove more space for /var/lib/nova as in current
setup but without live migrations? Or smaller storage, which on the
other hand allows live migration? Or you have better idea how to
split disks and machines to different services?
Well, I am not sure I care too much about live migrations. I can see
some cases where it would be nice, but in most of those we could also
just spin up a new instance and reconfigure.
Are there specific cases where live migrations would help us out a lot?
P.S. I am putting aside normal volumes for data, which will reside in
DellEqualogic which is 20TB. And it is perfectly fine.
Yeah.
kevin