Cloud image lifetimes
Garrett Holmstrom
gholms at fedoraproject.org
Mon Mar 23 04:44:24 UTC 2015
On 2015-03-18 15:38, David Gay wrote:
> What are your thoughts on AMI lifetimes? That is to say, how long
> should EC2 AMIs exist before they're deleted? A few points to
> consider:
>
> - AMIs only cost us for storage, so it's not a *huge* cost to
> maintain a public AMI - At the same time, there are a lot of AMIs,
> since we build 2-4 per AWS region per build, and that number is
> growing - There are 9 regions now, and we have 2 virtualization
> types, and 2 volume types, as well (9 regions * 2 * 2 = 36 AMIs per
> Base image build, 18 for Atomic builds (since they are only available
> in HVM format)) - This total number will only grow larger as we add
> instance-store AMIs, and so on - This isn't even taking into account
> any costs we'll have once we secure a deal with other providers like
> HP, Rackspace, and GCE, to maintain public images on their services
>
> I propose we have some sort of discussion regarding how long cloud
> image builds should be available on services like AWS. I suspect this
> will resolve to having different lifetimes for scratch, test, RC,
> final, and maybe other build types.
Different clouds have different norms. AWS is a cloud where anyone can
share an image with the world without having to "secure a deal" with
anyone. People there are used to sifting through the resulting giant
lists of images. AFAIK, that isn't the case in any of the other clouds
you listed -- those all use much shorter, curated lists, so for those it
may certainly make sense to prune unsupported releases after a period of
time.
In AWS people seem to essentially expect images to last forever and
hardcode their IDs into templates and launch configurations all over the
place that will break when the image goes away, so removing them is not
a decision to take lightly. The best data I am aware of for that are
the MirrorManager statistics for Fedora 8, which can show how long that
release remained popular after we created the Cloud SIG and began
releasing newer images ourselves. Perhaps someone with the appropriate
level of infrastructure log access would be able to shed some more light
on that. Also keep in mind that the storage costs for AWS images are
somewhat nuanced, as some images are stored compressed or sparsely, some
can share storage, and so on.
Now, for pre-release images, I think rotating them out after a period of
time makes sense no matter which cloud it is. Exactly how long that
should be could reasonably be cloud-dependent for stuff like betas, but
probably not as much for nightlies or TCs.
--
Garrett Holmstrom
More information about the cloud
mailing list