Atomic status

[Michael Scherer]

Le vendredi 18 juillet 2014 à 09:40 -0400, Colin Walters a écrit :
> Hi,
> 
> I feel like things aren't moving forward quickly enough with Atomic,
> both on the Docker base image side and the rpm-ostree host side.  A lot
> of that is to be expected - we're trying to introduce *two* new ways to
> deliver software beyond RPMs at the same time.  It's quite nontrivial.
> 
> There's also:
> 1) We're trying to do new projects, but we haven't added significant
> human resources
> 2) This is also the first Fedora.next release
> 3) We don't know how popular the whole Atomic/Docker model will be until
> we do a stable release really, but it's hard to get current
> Infrastructure people to spend time on it without knowing how valuable
> it is...
> 
> Let's look at blocking issues for Atomic in Fedora:
> 
> - Having rpm-ostree be run as part of nightly compose
> - Content mirroring
>   * Server with SSL, some level of redundancy
>   * Open question around how often/when composes are done, and how much
>   history is retained
> - GPG signing
> - Method of having Koji/ImageFactory pull mirrored OSTree repository
> content to compose ISOs
> - MirrorManager integration (no code written on ostree side yet)
> 
> To the best of my knowledge, code exists for all of the above except the
> last, and the last bit is hard to write without having a deployment of
> MirrorManager using it, which depends
> 
> I see two solution paths:
> 
> - Double down and fix the above, continue trying to sync Atomic with
> Fedora
>   * Have more time from current Infra people on above
>   * Have me/other members of my team join infra
>   * Mix of both?
> - Have Atomic be a Remix, partially in infra, partially outside
>   - Have custom GPG key
>   - Continue doing composes on the atomic01.qa.fedoraproject.org machine
>   - Mirror to something like Amazon CloudFront (would require external
>   sponsorship from Red Hat)

One of the issue we will have is that Fedora infrastructure is used in
production for Fedora, so i think we will not have as much agility if we
start to target it right from the start. 

We also have the issue that there is currently a proposal for a Centos
based atomic image
( http://lists.centos.org/pipermail/centos-devel/2014-June/011233.html
), and I think making thing work for Fedora and Centos at the same time
may add some contraints to us.

So i would be in favor of having atomic on a separate infrastructure for
now, and later try to push as much as possible on Fedora (and Centos)
once the dust settle regarding Atomic infra.

So i would propose:
- have rpm-ostree compose be done outside of koji for now. thanks to
fedmsg, this should be hard. This will permit to get a idea of the
ressources used for this, and will not put more burden on the current
host just for atomic. In fact, we could even imagine have a different
schedule for compose, based on the load. 

- have the gpg key being separate from the one of Fedora, and signing
done somewhere else. This would free us of the contraint we have
regarding access of Fedora key for now. It also permit us to test key
replacement more freely, something we will need to do in the future.

- use a set of mirrors outside Fedora ( i would use 3 or 4 VM properly
placed on the globe, under our control rather than using a CDN ). It
permit to know the requirement in term of size after a few months as
well as bandwidth in a more precise way since it will not mixed with
others stuff. in turn, this permit to data to do evaluate the sizing
when we move to Fedora and Centos infra. I also think that this is the
only way to have a shared ssl cert for the http server, as that seems to
be a requirement ( ie, we cannot do that with a regular set of mirrors
operated by volunteer, so that's something we will have to fix somehow
). Having discussed off-list about this, Stephen spoke also of the GPL
requirement regarding source, so that's something to keep in mind.


- the same go for imagefactory. it can be run on a separate host, and
maybe with a separate schedule, or a separate schedule ( ie, we could
make update during the release freeze, etc )

Then once everything is ok, we can start to move things back to Centos
and Fedora, one service after the others.

Doing this also break the loop, ie, we can make sure the project is
successful using Fedora package and later integrate it to Fedora rather
than just being a remix/spin/derivative.

Of course, we will discuss in details during meeting, but I prefer to
provides the details of the proposal to go forward before.

So if people agree with the idea of going separate for now, and then
come back with data and tested infrastructure, I can try to provide a
few hosting ressources for the compose ( nothing fancy for now, but we
can always try to find more ressources, i have some idea regarding who
could sponsor stuff ), and help as part of $dayjob for setting part of
the infrastructure. I tend to use ansible, so the work done for
deploying the part of the infra can then be reused for Fedora.


-- 
Michael Scherer