I had planned to post this earlier in the week, but... well. Not a good week. So here we go.
We'd like to get the the automatic service for uploading images to EC2 and alt.fedoraproject.org) running, and ideally also anaconda-based image generation. In order to do that, we need to decide and to do a number of things:
1. We need a process for non-scratch builds of AMIs
a. We'll get the arch/name bug fixed, but in the meantime, I suggest we go ahead with the uglier approach of creating a package in koji with the arch in the name, as a temporary workaround.
b. We need to configure the expiration policy for these builds. I suggest we choose three weeks for weekly builds, or one week for nightlies. Alpha, Beta, and Final release images should be kept forever. If that means we need different koji packages for the automatically-generated weekly or nightly builds, let's do that.
2. We need a place to run the cron job that runs the scripted automatic builds
- and someone to put those scripts into that place
3. We need a system to run the uploader service -- ideally a new isolated system for security, as this will need to hold EC2 credentials and have write access to alt.fedoraproject.org.
4. We need to finalize the Oz/ImageFactory integration.
- timeline, work to be done?
- how to deal with possible oz bugs and fixes in production
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, 12 Jul 2013 14:05:48 -0400 Matthew Miller mattdm@fedoraproject.org wrote:
I had planned to post this earlier in the week, but... well. Not a good week. So here we go.
We'd like to get the the automatic service for uploading images to EC2 and alt.fedoraproject.org) running, and ideally also anaconda-based image generation. In order to do that, we need to decide and to do a number of things:
We need a process for non-scratch builds of AMIs
a. We'll get the arch/name bug fixed, but in the meantime, I
suggest we go ahead with the uglier approach of creating a package in koji with the arch in the name, as a temporary workaround.
As I have said to you privately I don't find that acceptable. the fix should be fairly trivial.
b. We need to configure the expiration policy for these builds. I suggest we choose three weeks for weekly builds, or one week for nightlies. Alpha, Beta, and Final release images should be kept forever. If that means we need different koji packages for the automatically-generated weekly or nightly builds, let's do that.
we just need to work out the right koji-gc policy to write.
- We need a place to run the cron job that runs the scripted
automatic builds
- and someone to put those scripts into that place
it needs to all be put into ansible and credentials in ansible-private the host would likely be releng03.phx2.fedoraproject.org
- We need a system to run the uploader service -- ideally a new
isolated system for security, as this will need to hold EC2 credentials and have write access to alt.fedoraproject.org.
it would also be releng03.phx2.fedoraproject.org
We need to finalize the Oz/ImageFactory integration.
- timeline, work to be done?
https://fedoraproject.org/wiki/Releases/20/Schedule we are due to branch F20 on August 6th I would say that is the deadline, that will give us time to test and fix any issues before Alpha Change deadline of August 20th, the fedora 20 schedule is insanely tight however you have had since we branched off f19 to work on things.
- how to deal with possible oz bugs and fixes in production
We deal with them as we hit them, its a bit hard to plan
Dennis
On Fri, Jul 12, 2013 at 03:58:17PM -0500, Dennis Gilmore wrote:
- We need a process for non-scratch builds of AMIs a. We'll get the arch/name bug fixed, but in the meantime, I
suggest we go ahead with the uglier approach of creating a package in koji with the arch in the name, as a temporary workaround.
As I have said to you privately I don't find that acceptable. the fix should be fairly trivial.
If we can land the fix right now, let's do it. If it's going to take another week or so, does it really hurt to have the hack there temporarily until that time?
b. We need to configure the expiration policy for these builds. I suggest we choose three weeks for weekly builds, or one week for nightlies. Alpha, Beta, and Final release images should be kept forever. If that means we need different koji packages for the automatically-generated weekly or nightly builds, let's do that.
we just need to work out the right koji-gc policy to write.
Okay. Let's do it. Does the above (3 weeks for the automatic weekly builds) sound right?
- We need a place to run the cron job that runs the scripted
automatic builds
- and someone to put those scripts into that place
it needs to all be put into ansible and credentials in ansible-private the host would likely be releng03.phx2.fedoraproject.org
Okay, cool. Who can take care of that? Andrew has (or will make / finalize) the scripts and put them into
https://git.fedorahosted.org/cgit/cloud-image-service.git/
What are the next steps to get them live?
- We need a system to run the uploader service -- ideally a new
isolated system for security, as this will need to hold EC2 credentials and have write access to alt.fedoraproject.org.
it would also be releng03.phx2.fedoraproject.org
Is that adequate for the security requirements? What else does that system do? Are we sure it's not better to have it as a separate system?
- We need to finalize the Oz/ImageFactory integration.
- timeline, work to be done?
https://fedoraproject.org/wiki/Releases/20/Schedule we are due to branch F20 on August 6th I would say that is the deadline, that will give us time to test and fix any issues before Alpha Change deadline of August 20th, the fedora 20 schedule is insanely tight however you have had since we branched off f19 to work on things.
Yes, it's my understanding that the work Jay has is basically ready to go. I just want to plan out exactly _how_ it's going to go.
- how to deal with possible oz bugs and fixes in production
We deal with them as we hit them, its a bit hard to plan
Okay, I can live with that.
- We need to finalize the Oz/ImageFactory integration.
- timeline, work to be done?
[...]
Yes, it's my understanding that the work Jay has is basically ready to go. I just want to plan out exactly _how_ it's going to go.
And, if this part doesn't land, the end-user impact is low, because you've managed to cobble appliance-creator into moderately functional shape for another release, so that's good.
I just really think we have to get there eventually.
*ping* on this -- any further feedback Dennis or anyone else involved in rel-eng? We do need to get this moving.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Wed, 17 Jul 2013 21:28:59 -0400 Matthew Miller mattdm@fedoraproject.org wrote:
*ping* on this -- any further feedback Dennis or anyone else involved in rel-eng? We do need to get this moving.
sorry ive been out having surgery. What we need now is the new tooling for image creation. its really all in your court to deliver, upload scripts and koji integration of new tooling.
Dennis
rel-eng@lists.fedoraproject.org
-
Dennis Gilmore -
Matthew Miller