Cloud image lifetimes

Garrett Holmstrom gholms at fedoraproject.org
Mon Mar 23 04:44:24 UTC 2015


On 2015-03-18 15:38, David Gay wrote:
> What are your thoughts on AMI lifetimes? That is to say, how long
> should EC2 AMIs exist before they're deleted? A few points to
> consider:
>
> - AMIs only cost us for storage, so it's not a *huge* cost to
> maintain a public AMI - At the same time, there are a lot of AMIs,
> since we build 2-4 per AWS region per build, and that number is
> growing - There are 9 regions now, and we have 2 virtualization
> types, and 2 volume types, as well (9 regions * 2 * 2 = 36 AMIs per
> Base image build, 18 for Atomic builds (since they are only available
> in HVM format)) - This total number will only grow larger as we add
> instance-store AMIs, and so on - This isn't even taking into account
> any costs we'll have once we secure a deal with other providers like
> HP, Rackspace, and GCE, to maintain public images on their services
>
> I propose we have some sort of discussion regarding how long cloud
> image builds should be available on services like AWS. I suspect this
> will resolve to having different lifetimes for scratch, test, RC,
> final, and maybe other build types.

Different clouds have different norms.  AWS is a cloud where anyone can 
share an image with the world without having to "secure a deal" with 
anyone.  People there are used to sifting through the resulting giant 
lists of images.  AFAIK, that isn't the case in any of the other clouds 
you listed -- those all use much shorter, curated lists, so for those it 
may certainly make sense to prune unsupported releases after a period of 
time.

In AWS people seem to essentially expect images to last forever and 
hardcode their IDs into templates and launch configurations all over the 
place that will break when the image goes away, so removing them is not 
a decision to take lightly.  The best data I am aware of for that are 
the MirrorManager statistics for Fedora 8, which can show how long that 
release remained popular after we created the Cloud SIG and began 
releasing newer images ourselves.  Perhaps someone with the appropriate 
level of infrastructure log access would be able to shed some more light 
on that.  Also keep in mind that the storage costs for AWS images are 
somewhat nuanced, as some images are stored compressed or sparsely, some 
can share storage, and so on.

Now, for pre-release images, I think rotating them out after a period of 
time makes sense no matter which cloud it is.  Exactly how long that 
should be could reasonably be cloud-dependent for stuff like betas, but 
probably not as much for nightlies or TCs.

-- 
Garrett Holmstrom


More information about the cloud mailing list