EC2 cloud image size

[Dmitry Makovey]

I just tried to spin up "cloud" instance of F21 on EC2 only to discover
it's requirements for 12G volume (???). Am I doing something wrong or
did F21 jumped from moderate 2-3G (F20) over to 12G "overnight". I
assumed "cloud image" would be a tiny set of packages with very little
requirements to fit into all flavors offered by EC2 and alike. Was this
done to accommodate Docker?

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[Matthew Miller]

On Tue, Dec 16, 2014 at 02:40:46PM -0700, Dmitry Makovey wrote:
>  I just tried to spin up "cloud" instance of F21 on EC2 only
> to discover
>  it's requirements for 12G volume (???). Am I doing something wrong or
>  did F21 jumped from moderate 2-3G (F20) over to 12G "overnight". I
>  assumed "cloud image" would be a tiny set of packages with very little
>  requirements to fit into all flavors offered by EC2 and alike. Was this
>  done to accommodate Docker? 
This is something in the upload process — it was supposed to be fixed
in the final and maybe that got overlooked.

It is in fact actually quite small — under 400MB on disk.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Dmitry Makovey]

On 12/16/2014 03:00 PM, Matthew Miller wrote:
>  On Tue, Dec 16, 2014 at 02:40:46PM -0700, Dmitry Makovey wrote:
> > I just tried to spin up "cloud" instance of F21 on EC2 only to discover
> > it's requirements for 12G volume (???). Am I doing something wrong or
> > did F21 jumped from moderate 2-3G (F20) over to 12G "overnight". I
> > assumed "cloud image" would be a tiny set of packages with very little
> > requirements to fit into all flavors offered by EC2 and alike. Was this
> > done to accommodate Docker?
>  
>  This is something in the upload process — it was supposed to be fixed
>  in the final and maybe that got overlooked.
>  
>  It is in fact actually quite small — under 400MB on disk.
>   
that is what I thought - looking inside the instance not much space was
utilized, however lots got "reserved".

Does it mean that "soon enough" we'll see an updated AMI with moderate
disk demands?

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[Matthew Miller]

On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey wrote:
>  Does it mean that "soon enough" we'll see an updated
> AMI with moderate
>  disk demands? 
For some value of "soon enough", yeah. Sorry about that -- I'm checking
with David Gay, who wrote the uploader.


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[David Gay]

>  On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey wrote:
>  > Does it mean that "soon enough" we'll see an updated AMI with
> moderate
>  > disk demands?
>  
>  For some value of "soon enough", yeah. Sorry about that -- I'm checking
>  with David Gay, who wrote the uploader. 
Yeah, this was my oversight. Seems the cloud base image is only 425 MB. I can produce new
AMIs right now. Do you think a 1 GB or 500 MB drive would be best, or something else?

-- David

[Garrett Holmstrom]

On 2014-12-17 13:09, David Gay wrote:
> > On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey wrote:
> >> Does it mean that "soon enough" we'll see an updated AMI with
> moderate
> >> disk demands?
> >
> > For some value of "soon enough", yeah. Sorry about that -- I'm
> checking
> > with David Gay, who wrote the uploader.
>
>  Yeah, this was my oversight. Seems the cloud base image is only 425 MB. I can produce new
> AMIs right now. Do you think a 1 GB or 500 MB drive would be best, or something else?
EBS images can only use 1 GB size increments, and FWIW, the stock images 
that AWS produce come in 2 GB and 8 GB variants.  How does 2 GB sound?

-- 
Garrett Holmstrom

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 17 Dec 2014 16:09:03 -0500 (EST)
David Gay <dgay(a)redhat.com&gt; wrote:

>  > On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey
> wrote:
>  > > Does it mean that "soon enough" we'll see an updated AMI with
>  > > moderate disk demands?
>  > 
>  > For some value of "soon enough", yeah. Sorry about that -- I'm
>  > checking with David Gay, who wrote the uploader.
>  
>  Yeah, this was my oversight. Seems the cloud base image is only 425
>  MB. I can produce new AMIs right now. Do you think a 1 GB or 500 MB
>  drive would be best, or something else?
>  
Anaconda requires at least a 3G disk to install so the disk image
backing the cloud image is 3G you need to be at least that

Dennis
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBAgAGBQJUkmcDAAoJEH7ltONmPFDRNN8QANDCMGNsDRPx4e4m0uFwqkDu
> iE1rFHrcdUp6gTnapqVnFuvFegCAlTA2sHdHrXejWYVptdYfmVIFn3tWx9ssuLj3
> qpgNUxaPjSM7CTHpqOqBkyp4oNZkGkYvUxo285gUJXZ40VhYM95yo1FrkwlIS2Uf
> bftWQ0BgMfmwtPjUAwm2zrWPA+MIEnUE9plMi14HCRTaBlLulehDz788ODZ3Abws
> yqlANpQCDeEbyZUkyJRCchj/FqaLbBWVxOJEmmR+dcVVJAvvq8b/3H75Ft6D5wSa
> BU9HWLdYFSPfPrej92EVv6wi3NT3NweClBSDFHbWO5tOu8NqM3Mg21Vv4pxZ3OJ4
> uF8/THcwblKEszgd15bKimykBqj3v0Mp6fPMGZ/vaJR0mscATzCyK+VIUv+2lZHj
> TCXlCAGEnGPzB+qWY06ltVcC3jIjuXU7H0SArqgHw5lTLOenEzFAAINYsxsAlH+T
> xR401Cu8mnLDVTNbhhgYiUTIyoHza8ZfSzH9yruCf46cY9w67E4vtDQhXUHC4Mgu
> /O/iLpzZmEa5KyLy8+y8pn4PdkJlgn9o996TVWRPK3r8CCmf0BQa5KvpoQPxIfNJ
> IgKKsWjDBZkEyW5CZLBv2i6QZ/rCjRgl7UENPtAaCQs+ML7+n+1FqRoY8GRzM0hh
> t5Gdv6Jv3TKSNJ6fhVn+
> =Oc0g
> -----END PGP SIGNATURE-----
>

[Dmitry Makovey]

On 12/17/2014 10:32 PM, Dennis Gilmore wrote:
>  Anaconda requires at least a 3G disk to install so the disk image
>  backing the cloud image is 3G you need to be at least that 
are you saying that this has been introduced in F21? I have just spun up
F20 on a 2G disk with no issues.

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thu, 18 Dec 2014 08:03:12 -0700
Dmitry Makovey <dmitry(a)athabascau.ca&gt; wrote:

>  On 12/17/2014 10:32 PM, Dennis Gilmore wrote:
>  > Anaconda requires at least a 3G disk to install so the disk image
>  > backing the cloud image is 3G you need to be at least that
>  
>  are you saying that this has been introduced in F21? I have just spun
>  up F20 on a 2G disk with no issues.
>  
>  
yes, we changed how we made disk images in f21 and as a result we had
to make the minimal disk image bigger. it could be made smaller again
with changes to anaconda

Dennis
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2
>
> iQIcBAEBAgAGBQJUkx8NAAoJEH7ltONmPFDR7SYP/1E62tf5GCfvL1fL7zYwFt0h
> qiiA80i/okQ6Yu19IK6LHm5lNjsAfHNlmo6SL0z2dCh51e6elWCa8604VneGOENh
> 2PoNNfAIVfZFd2uRIekCjau47Zm0VzqA5fAexHgWndC1x6Px0bXfaivC/0R3iNUu
> E8cBY9CvxmkeSCVO3IdMA9jc8N7kxlBykmZtRsBeHZVVuaw+uBjT58uWMTayao6a
> FAPHndcAGtyn1P4ttU6vdMVMnjNu9/gbJxcAgv324YCCHBlBVJfx5sLfZNEZFdNv
> GYNTNBAtnfj9/XcnQCJNBNEZ1XU4Da3Nojodt8MNO/wkXrhGWYtmpnw1Hs7IhUiX
> mPswDoAFUQ76DtA2+0gQP+Ao6UoT5ejkJGRWqYBzHmzT1c3c7W0xvNhjqxmWUA1A
> QeNbqHTSbzgTHII/75yVDemp9Q/CXpohV6Ca/GrzCSd7rubFVAgW8jI42nzKX5OQ
> KhVhxdG/28/UV5d2C/cGt0AvxGpAEI17bnkcwLdzC60SehgralB9qs7hkxd/X5oT
> VcpS0oENGKyORhWVxDKILHvlMIHVXeHNoReqs/BOf+bITjMTvmMPh3YPv6XP6lE/
> L3zxHcOSHRBAG1yIRWx4qpujMt0HuLC8z9d/lhjtHsXQ2p8OfRuGBchY8GapBDVS
> uwJpIRubU61yKltonSy7
> =Pny4
> -----END PGP SIGNATURE-----
>

[Dmitry Makovey]

On 12/17/2014 02:09 PM, David Gay wrote:
>  Yeah, this was my oversight. Seems the cloud base image is only 425
>  MB. I can produce new AMIs right now. Do you think a 1 GB or 500 MB
>  drive would be best, or something else? 
A bit of a side-track: is there a documentation on workflow you use for
creating/uploading those? I haven't seen any detailed step-by-step
guides for creation and upload of Fedora images so far...

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[Dmitry Makovey]

On 12/17/2014 02:09 PM, David Gay wrote:
> > On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey wrote:
> >> Does it mean that "soon enough" we'll see an updated AMI with
> >> moderate disk demands?
> > 
> > For some value of "soon enough", yeah. Sorry about that -- I'm
> > checking with David Gay, who wrote the uploader.
>  
>  Yeah, this was my oversight. Seems the cloud base image is only 425
>  MB. I can produce new AMIs right now. Do you think a 1 GB or 500 MB
>  drive would be best, or something else? 
/gentle/ bump?

Is any help required to get F21 "smaller" images to AWS? I'd love to
help (if I can) since I'm interested in the outcome myself.

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[David Gay]

----- Original Message -----
>  From: "Dmitry Makovey" <dmitry(a)athabascau.ca&gt;
>  To: "Fedora Cloud SIG" <cloud(a)lists.fedoraproject.org&gt;
>  Sent: Friday, January 2, 2015 3:02:20 PM
>  Subject: Re: EC2 cloud image size
>  
>  On 12/17/2014 02:09 PM, David Gay wrote:
>  >> On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey wrote:
>  >>> Does it mean that "soon enough" we'll see an updated AMI with
>  >>> moderate disk demands?
>  >> 
>  >> For some value of "soon enough", yeah. Sorry about that -- I'm
>  >> checking with David Gay, who wrote the uploader.
>  > 
>  > Yeah, this was my oversight. Seems the cloud base image is only 425
>  > MB. I can produce new AMIs right now. Do you think a 1 GB or 500 MB
>  > drive would be best, or something else?
>  
>  /gentle/ bump?
>  
>  Is any help required to get F21 "smaller" images to AWS? I'd love to
>  help (if I can) since I'm interested in the outcome myself.
>  
>  --
>  Dmitry Makovey
>  Web Systems Administrator
>  Athabasca University
>  [phone]
>  ---
>  Confidence is what you have before you understand the problem
>      Woody Allen
>  
>  When in trouble when in doubt run in circles scream and shout
>       http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330
>  
>  
>  _______________________________________________
>  cloud mailing list
>  cloud(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/cloud
>  Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>   
Hey -- I actually solved this problem during the holidays. Lowered the size to 3 GB as
suggested. There just haven't been any new builds that have gone through, and I
haven't cut a new version just yet anyway (due to the holidays). It's no problem
for me to send someone a list of fresh AMIs of the F21 cloud build with the lowered size
tomorrow, if someone can toss them on the official website, as well. But yeah, this is all
done. Just had to tweak two numbers is all.

-- oddshocks

[Dmitry Makovey]

On 01/02/2015 05:46 PM, David Gay wrote:
>  Hey -- I actually solved this problem during the holidays. Lowered
>  the size to 3 GB as suggested. There just haven't been any new builds
>  that have gone through, and I haven't cut a new version just yet
>  anyway (due to the holidays). It's no problem for me to send someone
>  a list of fresh AMIs of the F21 cloud build with the lowered size
>  tomorrow, if someone can toss them on the official website, as well.
>  But yeah, this is all done. Just had to tweak two numbers is all. 
Yay David! :)


-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[David Gay]

----- Original Message -----
>  From: "David Gay" <dgay(a)redhat.com&gt;
>  To: "Fedora Cloud SIG" <cloud(a)lists.fedoraproject.org&gt;
>  Sent: Friday, January 2, 2015 4:46:44 PM
>  Subject: Re: EC2 cloud image size
>  
>  ----- Original Message -----
>  > From: "Dmitry Makovey" <dmitry(a)athabascau.ca&gt;
>  > To: "Fedora Cloud SIG" <cloud(a)lists.fedoraproject.org&gt;
>  > Sent: Friday, January 2, 2015 3:02:20 PM
>  > Subject: Re: EC2 cloud image size
>  > 
>  > On 12/17/2014 02:09 PM, David Gay wrote:
>  > >> On Tue, Dec 16, 2014 at 03:07:12PM -0700, Dmitry Makovey wrote:
>  > >>> Does it mean that "soon enough" we'll see an updated AMI
> with
>  > >>> moderate disk demands?
>  > >> 
>  > >> For some value of "soon enough", yeah. Sorry about that --
> I'm
>  > >> checking with David Gay, who wrote the uploader.
>  > > 
>  > > Yeah, this was my oversight. Seems the cloud base image is only 425
>  > > MB. I can produce new AMIs right now. Do you think a 1 GB or 500 MB
>  > > drive would be best, or something else?
>  > 
>  > /gentle/ bump?
>  > 
>  > Is any help required to get F21 "smaller" images to AWS? I'd love to
>  > help (if I can) since I'm interested in the outcome myself.
>  > 
>  > --
>  > Dmitry Makovey
>  > Web Systems Administrator
>  > Athabasca University
>  > [phone]
>  > ---
>  > Confidence is what you have before you understand the problem
>  >     Woody Allen
>  > 
>  > When in trouble when in doubt run in circles scream and shout
>  >      http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330
>  > 
>  > 
>  > _______________________________________________
>  > cloud mailing list
>  > cloud(a)lists.fedoraproject.org
>  > https://admin.fedoraproject.org/mailman/listinfo/cloud
>  > Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>  > 
>  
>  Hey -- I actually solved this problem during the holidays. Lowered the size
>  to 3 GB as suggested. There just haven't been any new builds that have gone
>  through, and I haven't cut a new version just yet anyway (due to the
>  holidays). It's no problem for me to send someone a list of fresh AMIs of
>  the F21 cloud build with the lowered size tomorrow, if someone can toss them
>  on the official website, as well. But yeah, this is all done. Just had to
>  tweak two numbers is all.
>  
>  -- oddshocks
>  _______________________________________________
>  cloud mailing list
>  cloud(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/cloud
>  Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>   
Regarding this, I've published a list of new F21 cloud x86_64 base AMIs -- in both HVM
and PV -- to this list[1]. These AMIs have been registered with our minimum size
requirement, 3 GB. Much smaller than the previous 12 GB registrations. Hopefully, this
solves everyone's issues for now. If desired, I can send out a list of fresh Atomic
AMIs with the lower 3 GB requirement, too. That email was mostly an announcement that base
images are now available in HVM, though.

[1]: https://lists.fedoraproject.org/pipermail/cloud/2015-January/004831.html

-- oddshocks

[Dmitry Makovey]

On 01/02/2015 05:46 PM, David Gay wrote:
>  Hey -- I actually solved this problem during the holidays. Lowered
>  the size to 3 GB as suggested. There just haven't been any new builds
>  that have gone through, and I haven't cut a new version just yet
>  anyway (due to the holidays). It's no problem for me to send someone
>  a list of fresh AMIs of the F21 cloud build with the lowered size
>  tomorrow, if someone can toss them on the official website, as well.
>  But yeah, this is all done. Just had to tweak two numbers is all.
>   
OK, So David's part is done, however AWS still shows old AMI's. Does it
mean we have to nudge/bump someone to push new images over?

-- 
Dmitry Makovey
Web Systems Administrator
Athabasca University
[phone]
---
Confidence is what you have before you understand the problem
    Woody Allen

When in trouble when in doubt run in circles scream and shout
     http://www.wordwizard.com/phpbb3/viewtopic.php?f=16&t=19330

[David Gay]

----- Original Message -----
>  From: "Dmitry Makovey" <dmitry(a)athabascau.ca&gt;
>  To: "Fedora Cloud SIG" <cloud(a)lists.fedoraproject.org&gt;
>  Sent: Thursday, January 29, 2015 11:50:52 AM
>  Subject: Re: EC2 cloud image size
>  
>  On 01/02/2015 05:46 PM, David Gay wrote:
>  > Hey -- I actually solved this problem during the holidays. Lowered
>  > the size to 3 GB as suggested. There just haven't been any new builds
>  > that have gone through, and I haven't cut a new version just yet
>  > anyway (due to the holidays). It's no problem for me to send someone
>  > a list of fresh AMIs of the F21 cloud build with the lowered size
>  > tomorrow, if someone can toss them on the official website, as well.
>  > But yeah, this is all done. Just had to tweak two numbers is all.
>  > 
>  
>  OK, So David's part is done, however AWS still shows old AMI's. Does it
>  mean we have to nudge/bump someone to push new images over? 
The AMIs need to be updated on the web team's side. I'm told I'd probably have
to send a patch in rather than get commit access. I'd like to provide them with a list
of entirely fresh AMIs with the smaller volume size, including the new base HVM images. Do
we have someone who's looped in with the web team (maybe robyduck?) who can manage
this? I could write a patch myself, but I've looked at the code and I'm unfamiliar
with the particular templating language and JS used on that area of the page, and also, we
need a new button/selector/something so that users can choose paravirtual or HVM when they
select "Fedora Cloud base". I figured the implementation of that new
button/selector/whatever should be left up to the web team.

All that being said, if someone can connect us to the web team on this one, I can send out
a fresh, signed list of AMIs to this list. From there, the web team can update the page
with the new AMI IDs however they see fit.

-- David