basic plan for two-week atomic images

Wed Jun 17 14:17:59 UTC 2015

On Mon, Jun 15, 2015 at 2:23 PM, Michael P. McGrath <mmcgrath at redhat.com> wrote:
> ----- Original Message -----
>> From: "Matthew Miller" <mattdm at fedoraproject.org>
>> To: "Fedora Cloud SIG" <cloud at lists.fedoraproject.org>
>> Sent: Monday, June 15, 2015 10:17:31 AM
>> Subject: Re: basic plan for two-week atomic images
>>
>> On Wed, May 27, 2015 at 11:37:18AM -0500, Adam Miller wrote:
>> > >> > 2. Images built from those nightly (I think right now only rawhide?)
>> > >> once f22 is final only rawhide.
>> > > What would it take to make those happen post-release?
>> > If we could get a clearly defined list of what these items are I would
>> > be happy to work towards getting them done. I just don't know how that
>>
>> Going back to this old thread. :) What's the list you need here? I
>> assume it's:
>>
>> - downloadable Atomic qcow2 (and raw.xz)
>> - Same image uploaded to EC2 (and hopefully others)
>> - installer ISO?
>>
>
> At this point I would settle for anything I could put here:
>
> http://dl.fedoraproject.org/pub/alt/fedora-atomic/images/testing/
>
> and blog about.

To satisfy this in the near term as we sort out a more appropriate
long term solution, we now have F22 + Upddates nightly ISO builds[0]
as well as re-enabled vagrant and qcow builds[1].

For anyone interested in the "secret sauce" here:

- the ISOs are built in a mock chroot with this script:
https://github.com/maxamillion/fedora-atomic-nightly/blob/master/build-atomic.sh

- vagrant/qcow images were re-enabled in the releng script here:
https://pagure.io/releng/blob/master/f/scripts/build-cloud-images

Once we have pungi4 in shape and/or the run-root koji plugin live in
production, we will likely be able to move these sort of tasks into a
more well defined process better suited for the long term but my hope
is that we'll no longer block the projectatomic team.

-AdamM

[0] - http://atomic-nightly.cloud.fedoraproject.org/composes
[1] - http://koji.fedoraproject.org/koji/tasks?start=0&state=all&view=tree&method=image&order=-id

>
>>
>> > >> > 3. Images go through automated tests, automatically (tunir; not fully
>> > >> >    automatic yet, right?)
>> > >> there is zero automated testing available, but we really need it, afaik
>> > >> it is
>> > >> all manual
>> > > See <https://fedoraproject.org/wiki/Changes/tunir>. Plus glorious
>> > > Taskotron future, I'm sure. :)
>> > I'm not sure where in the phases this should fit, it seems like a
>> > giant forklift of work to me given that the base Fedora distro that
>> > Atomic is going to be pulling it's bits from doesn't even have this
>> > functionality. Am I mistaken or possibly missing something? (Please
>> > correct me if I am)
>>
>> I think initially, this will be basic smoketests. Currently, I believe
>> those are https://github.com/kushaldas/tunirtests — Kushal, can you
>> correct me if I'm wrong. So, we don't need to forklift the whole
>> thing... just get these running automatically.
>>
>>
>>
>> > >> We need people to test, and a process to move things through stages.
>> > >> build all the images, etc. a location to put things, i,e lots of
>> > >> things
>> > This largely piggy backs off my previous point. Automatic "graduation"
>> > will require a decent amount of work and probably commitment from the
>> > QA group. I won't speak to their capacity to cater to this but I think
>> > what is ultimately being asked of them should be brought to their
>> > attention as early as possible to get their input.
>>
>> I don't think we can ask a lot more of the QA group — we may need more,
>> new bodies.
>>
>>
>> > > Let's work at building the full list of lots of things. If we want to
>> > > have people testing, I'd say that'd go something like:
>> > >   - every two weeks, latest image to pass is flagged as needing human
>> > >     testing, a la a package update in bodhi
>> > >   - some sort of karma system — maybe it _is_ bodhi, but don't want to
>> > >     block on that
>> > If it were to be Bodhi, would the Atomic tree (or image) then be just
>> > another build artifact pushed through and sync'd like an rpm? So for
>> > at least the near term, it would require human intervention of
>> > submitting an update?
>>
>> Yeah. Although maybe that update could be summitted by a bot of some
>> sort? Presumably the same bot running Tunir. This would have the logic
>> for "two weeks mark has passed; find the latest image to pass tests
>> with this period, submit as update (or raise alarm if nothing has
>> succeeded)".
>>
>> > > Atomic devs have expressed a strong worry about the lack of
>> > > something like this, unless I'm misunderstanding. Possibly an
>> > > alternate image including updates-testing would suffice (or, maybe,
>> > > is actually separately necessary, since atomic doesn't let
>> > > individual testers just pull in individual RPMs to test).
>> > If I remember correctly, there are also concerns about things ending
>> > up in Atomic trees that actually never see the light of day in Fedora
>> > because they might never make it to stable. What's the policy on that
>> > sort of thing? (Or is there one? I suspect this could be uncharted
>> > territory because Atomic is effectively a new Operating System product
>> > with it's own lifecycle aiming to operate within the Fedora umbrella)
>>
>> It's a little bit uncharted territory if we're using F22 as a base
>> rather than Rawhide. For Rawhide, it's happened before (and
>> theoretically should be SOP for a Change proposal which doesn't work
>> out). But even with F22 base, as long as name-epoch-version-release
>> marches forward, and the packages which are subject to such churn are
>> clearly described as volatile, I don't see a problem.
>>
>>
>> > I suspect that if/when the testing automation is in place the double
>> > gate wouldn't be needed, especially if this is just considered a
>> > development release. But I think that's where part of the problem lies
>> > is that the Atomic dev teams wants to use a stable OS as something
>> > that is re-rev'd with components updated and replaced but still call
>> > it "Fedora $VERSION" when in fact, it is not the same as what others
>> > will get from Fedora $VERSION unless they install from either the
>> > Atomic image or this dev tree. (depending on what the decisions are
>> > there)
>>
>> I think mostly the components that will get updated and replaced are
>> under the general Atomic umbrella, right? (Kubernetes, Atomic,
>> Docker...) Do we have concrete examples of things outside of that? I
>> know there was a theoretical example of systemd, but maybe that can be
>> coordinated with a mainline update. And the kernel is on an aggressive
>> update schedule anyway.
>>
>
> Kubernetes might be removed from the list soon if we can containerize it,
> there's extra work to do so but that might simplify the atomic builds a bit.
>
>    -Mike
>
>> My suggestion, at least, is to try this approach with an eye towards
>> building the thing you talk about in this next paragraph for post-F23,
>> if it turns out to be necessary.
>>
>> > Basically this again. If anything I think if Fedora Atomic wants to be
>> > it's own product then it should exist slightly differently such that
>> > it has it's own set of koji tags, inherits from $current, but
>> > subplants the components it needs. This would however be a massive
>> > shift because as it turns out that is something that's never been done
>> > before and I think this would require a whole new Bodhi stream since
>> > it's effectively a new product//release. If it's not Rawhide but it's
>> > not Fedora $current, then It's basically it's own Distro that's just
>> > based on Fedora and if it were to exist within the Fedora umbrella
>> > then I think there need to be policy changes and that's going to be a
>> > giant pile of other work to sort out how that's all going to fit.
>>
>> --
>> Matthew Miller
>> <mattdm at fedoraproject.org>
>> Fedora Project Leader
>> _______________________________________________
>> cloud mailing list
>> cloud at lists.fedoraproject.org
>> https://admin.fedoraproject.org/mailman/listinfo/cloud
>> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>>
> _______________________________________________
> cloud mailing list
> cloud at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct