F20 image building plan and decisions we need to make

[Matthew Miller]

I had planned to post this earlier in the week, but... well. Not a good
week. So here we go.

We'd like to get the the automatic service for uploading images to EC2 and
alt.fedoraproject.org) running, and ideally also anaconda-based image
generation. In order to do that, we need to decide and to do a number of
things:

1. We need a process for non-scratch builds of AMIs

   a. We'll get the arch/name bug fixed, but in the meantime, I suggest we
   go ahead with the uglier approach of creating a package in koji with the
   arch in the name, as a temporary workaround.

   b. We need to configure the expiration policy for these builds. I suggest
   we choose three weeks for weekly builds, or one week for nightlies.
   Alpha, Beta, and Final release images should be kept forever. If that
   means we need different koji packages for the automatically-generated
   weekly or nightly builds, let's do that.
   
2. We need a place to run the cron job that runs the scripted automatic builds
  
   - and someone to put those scripts into that place
   
3. We need a system to run the uploader service -- ideally a new isolated
   system for security, as this will need to hold EC2 credentials and have
   write access to alt.fedoraproject.org.
   
4. We need to finalize the Oz/ImageFactory integration.

   - timeline, work to be done?

   - how to deal with possible oz bugs and fixes in production

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  <mattdm(a)fedoraproject.org&gt;

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 12 Jul 2013 14:05:48 -0400
Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:

>  I had planned to post this earlier in the week, but... well. Not a
>  good week. So here we go.
>  
>  We'd like to get the the automatic service for uploading images to
>  EC2 and alt.fedoraproject.org) running, and ideally also
>  anaconda-based image generation. In order to do that, we need to
>  decide and to do a number of things:
>  
>  1. We need a process for non-scratch builds of AMIs
>  
>     a. We'll get the arch/name bug fixed, but in the meantime, I
>  suggest we go ahead with the uglier approach of creating a package in
>  koji with the arch in the name, as a temporary workaround.
>  
As I have said to you privately I don't find that acceptable. the fix
should be fairly trivial.

>  
>     b. We need to configure the expiration policy for these builds. I
>  suggest we choose three weeks for weekly builds, or one week for
>  nightlies. Alpha, Beta, and Final release images should be kept
>  forever. If that means we need different koji packages for the
>  automatically-generated weekly or nightly builds, let's do that.
>  
we just need to work out the right koji-gc policy to write. 
    
>  2. We need a place to run the cron job that runs the scripted
>  automatic builds 
>     - and someone to put those scripts into that place
>  
it needs to all be put into ansible and credentials in ansible-private
the host would likely be releng03.phx2.fedoraproject.org

>  3. We need a system to run the uploader service -- ideally a new
>  isolated system for security, as this will need to hold EC2
>  credentials and have write access to alt.fedoraproject.org.
>  
it would also be releng03.phx2.fedoraproject.org

>  4. We need to finalize the Oz/ImageFactory integration.
>  
>     - timeline, work to be done?
>  
https://fedoraproject.org/wiki/Releases/20/Schedule
we are due to branch F20 on August 6th I would say that is the deadline,
that will give us time to test and fix any issues before Alpha Change
deadline of August 20th, the fedora 20 schedule is insanely tight
however you have had since we branched off f19 to work on things.

>     - how to deal with possible oz bugs and fixes in production
>  
We deal with them as we hit them, its a bit hard to plan

Dennis 

> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
>
> iEYEARECAAYFAlHgbe4ACgkQkSxm47BaWff1wACdFSwzkxOg2LcokOPJ3tKjz7Xc
> YHIAoJ2Ew3YZbxKEdxzX8lcSQ8EoCqEC
> =bUMf
> -----END PGP SIGNATURE-----
>

[Matthew Miller]

On Fri, Jul 12, 2013 at 03:58:17PM -0500, Dennis Gilmore wrote:
>  > 1. We need a process for non-scratch builds of AMIs
>  >    a. We'll get the arch/name bug fixed, but in the meantime, I
>  > suggest we go ahead with the uglier approach of creating a package in
>  > koji with the arch in the name, as a temporary workaround.
>  As I have said to you privately I don't find that acceptable. the fix
>  should be fairly trivial. 
If we can land the fix right now, let's do it. If it's going to take another
week or so, does it really hurt to have the hack there temporarily until
that time?

>  >    b. We need to configure the expiration policy for these
> builds. I
>  > suggest we choose three weeks for weekly builds, or one week for
>  > nightlies. Alpha, Beta, and Final release images should be kept
>  > forever. If that means we need different koji packages for the
>  > automatically-generated weekly or nightly builds, let's do that.
>  we just need to work out the right koji-gc policy to write.  
Okay. Let's do it. Does the above (3 weeks for the automatic weekly builds)
sound right?


>  > 2. We need a place to run the cron job that runs the scripted
>  > automatic builds 
>  >    - and someone to put those scripts into that place
>  it needs to all be put into ansible and credentials in ansible-private
>  the host would likely be releng03.phx2.fedoraproject.org 
Okay, cool. Who can take care of that? Andrew has (or will make / finalize)
the scripts and put them into

  https://git.fedorahosted.org/cgit/cloud-image-service.git/

What are the next steps to get them live?

>  > 3. We need a system to run the uploader service -- ideally a
> new
>  > isolated system for security, as this will need to hold EC2
>  > credentials and have write access to alt.fedoraproject.org.
>  it would also be releng03.phx2.fedoraproject.org 
Is that adequate for the security requirements? What else does that system
do? Are we sure it's not better to have it as a separate system?


>  > 4. We need to finalize the Oz/ImageFactory integration.
>  >    - timeline, work to be done?
>  https://fedoraproject.org/wiki/Releases/20/Schedule
>  we are due to branch F20 on August 6th I would say that is the deadline,
>  that will give us time to test and fix any issues before Alpha Change
>  deadline of August 20th, the fedora 20 schedule is insanely tight
>  however you have had since we branched off f19 to work on things. 
Yes, it's my understanding that the work Jay has is basically ready to go. I
just want to plan out exactly _how_ it's going to go.


>  >    - how to deal with possible oz bugs and fixes in production
>  We deal with them as we hit them, its a bit hard to plan 
Okay, I can live with that.

-- 
Matthew Miller           mattdm(a)mattdm.org          <http://mattdm.org/>

[Matthew Miller]

>  > > 4. We need to finalize the Oz/ImageFactory integration.
>  > >    - timeline, work to be done? 
[...]
>  Yes, it's my understanding that the work Jay has is basically
> ready to go. I
>  just want to plan out exactly _how_ it's going to go. 
And, if this part doesn't land, the end-user impact is low, because you've
managed to cobble appliance-creator into moderately functional shape for
another release, so that's good.

I just really think we have to get there eventually.

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  <mattdm(a)fedoraproject.org&gt;

[Matthew Miller]

*ping* on this -- any further feedback Dennis or anyone else involved in
rel-eng? We do need to get this moving.

-- 
Matthew Miller  ☁☁☁  Fedora Cloud Architect  ☁☁☁  <mattdm(a)fedoraproject.org&gt;

[Dennis Gilmore]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 17 Jul 2013 21:28:59 -0400
Matthew Miller <mattdm(a)fedoraproject.org&gt; wrote:

>  *ping* on this -- any further feedback Dennis or anyone else
> involved
>  in rel-eng? We do need to get this moving.
>  
>  
sorry ive been out having surgery. What we need now is the new tooling
for image creation. its really all in your court to deliver, upload
scripts and koji integration of new tooling.

Dennis
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.19 (GNU/Linux)
>
> iEYEARECAAYFAlHnYQYACgkQkSxm47BaWfcd+gCfUpEnXk+o3OcdYU6JY81n5ioB
> KRgAnjc9jSfDcZhexsbzIzhV2FLjKGe9
> =TZel
> -----END PGP SIGNATURE-----
>