Atomic status

[Colin Walters]

Hi,

I feel like things aren't moving forward quickly enough with Atomic,
both on the Docker base image side and the rpm-ostree host side.  A lot
of that is to be expected - we're trying to introduce *two* new ways to
deliver software beyond RPMs at the same time.  It's quite nontrivial.

There's also:
1) We're trying to do new projects, but we haven't added significant
human resources
2) This is also the first Fedora.next release
3) We don't know how popular the whole Atomic/Docker model will be until
we do a stable release really, but it's hard to get current
Infrastructure people to spend time on it without knowing how valuable
it is...

Let's look at blocking issues for Atomic in Fedora:

- Having rpm-ostree be run as part of nightly compose
- Content mirroring
  * Server with SSL, some level of redundancy
  * Open question around how often/when composes are done, and how much
  history is retained
- GPG signing
- Method of having Koji/ImageFactory pull mirrored OSTree repository
content to compose ISOs
- MirrorManager integration (no code written on ostree side yet)

To the best of my knowledge, code exists for all of the above except the
last, and the last bit is hard to write without having a deployment of
MirrorManager using it, which depends

I see two solution paths:

- Double down and fix the above, continue trying to sync Atomic with
Fedora
  * Have more time from current Infra people on above
  * Have me/other members of my team join infra
  * Mix of both?
- Have Atomic be a Remix, partially in infra, partially outside
  - Have custom GPG key
  - Continue doing composes on the atomic01.qa.fedoraproject.org machine
  - Mirror to something like Amazon CloudFront (would require external
  sponsorship from Red Hat)

Would more regular meetings help?

[Colin Walters]

At a high level, it feels to me like what we really want for Atomic is a
"staging" release.  We need something where interested parties can try
it, provide feedback, and we can iterate.

But we just don't have a way to do this right now.  There's COPR for RPM
content, which works well, but we don't (to my knowledge) have a way to
create a fork of Koji itself to try patches there and distribute that
content.  Or do we?  Would the alternative architecture approach work
here?

Hmm, maybe I should just be doing this on the existing
atomic01.qa.fedoraproject.org box - just start running more
infrastructure there.  But that doesn't solve the mirroring problem
though.

[Matthew Miller]

On Fri, Jul 18, 2014 at 10:37:42AM -0400, Colin Walters wrote:
>  At a high level, it feels to me like what we really want for Atomic
> is a
>  "staging" release.  We need something where interested parties can try
>  it, provide feedback, and we can iterate. 
Once we have the basic stuff discussed in the other thread, we will have F21
and rawhide nightlies. We could make an alternative spin kickstarts in
either of those branches to test with.

>  But we just don't have a way to do this right now.  There's
> COPR for RPM
>  content, which works well, but we don't (to my knowledge) have a way to
>  create a fork of Koji itself to try patches there and distribute that
>  content.  Or do we?  Would the alternative architecture approach work
>  here? 
I'm missing why a Koji fork would be needed (except for staging improvements
to the future tree compose plugin or whatever)... can you elaborate?


-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Stephen John Smoogen]

On 18 July 2014 07:40, Colin Walters <walters(a)verbum.org&gt; wrote:

>  Hi,
>
>  I feel like things aren't moving forward quickly enough with Atomic,
>  both on the Docker base image side and the rpm-ostree host side.  A lot
>  of that is to be expected - we're trying to introduce *two* new ways to
>  deliver software beyond RPMs at the same time.  It's quite nontrivial.
>
>  There's also:
>  1) We're trying to do new projects, but we haven't added significant
>  human resources
>  2) This is also the first Fedora.next release
>  3) We don't know how popular the whole Atomic/Docker model will be until
>  we do a stable release really, but it's hard to get current
>  Infrastructure people to spend time on it without knowing how valuable
>  it is...
>
>  
I don't want to go into 'if X is more valuable than Y' as even if
Atomic/Docker is the most valuable item we would still have 1 & 2 taking up
all our time.

1) is even worse because we are actually down several full time people
because Seth Vidal did the work of multiple people (from development,
systems architecture, and administration). There are also large pushes of
existing projects to grow. Matt Domsch who did 99.9% of the mirrormanager
work did this on his volunteer time which has had to be directed elsewhere.
This means that the code has to be updated and looked at by new people
which is taking time.

2) effectively adds 2 releases onto our one set which means that various
tools have to be rebuilt. Then add in several secondary architectures being
added.


Let's look at blocking issues for Atomic in Fedora:
>  - Having rpm-ostree be run as part of nightly compose
>  - Content mirroring
>    * Server with SSL, some level of redundancy
>    * Open question around how often/when composes are done, and how much
>    history is retained
>  - GPG signing
>  - Method of having Koji/ImageFactory pull mirrored OSTree repository
>  content to compose ISOs
>  - MirrorManager integration (no code written on ostree side yet)
>
>  
Most of the above are also in grumpy areas of infrastructure where adding a
person to fix it means they will need to learn a lot of other things before
it works well or doesn't snag up something else. It would be good to add a
person, but don't expect an instant win but more instant pain in doing so.

There is also a part where infrastructure and releng are separate units in
some ways and not separate in others. Some of your blockers are on one and
some are on another and some in between. I can really answer on the content
mirroring side as I believe we can figure out a way to do that. Other parts
are on koji developers and other developers who work on signing.


-- 
Stephen J Smoogen.

[Colin Walters]

Hi Stephen,



Thanks for the response.



On Fri, Jul 18, 2014, at 10:43 AM, Stephen John Smoogen wrote:


Most of the above are also in grumpy areas of infrastructure
where adding a person to fix it means they will need to learn a
lot of other things before it works well or doesn't snag up
something else. It would be good to add a person, but don't
expect an instant win but more instant pain in doing so.



Right.



There is also a part where infrastructure and releng are
separate units in some ways and not separate in others. Some of
your blockers are on one and some are on another and some in
between. I can really answer on the content mirroring side as I
believe we can figure out a way to do that. Other parts are on
koji developers and other developers who work on signing.



Yeah.  The more I think about this, maybe Project Atomic should
operate on its own, deriving from Fedora, but with separate
infrastructure.



Let's look at an example of another project:

[1]http://www.ovirt.org/Home



They have their own ISO page
<[2]http://resources.ovirt.org/pub/ovirt-3.4/> (and it's
slick!), their own GPG keys, their own release schedule, their
own ovirt-release RPMs, their own installation instructions,
their own mirroring list.  And presumably their own system
administrators.



And ultimately their own branding.  Maybe that's the right
thing for Atomic too?



I can see a lot of advantages to that path; disadvantages as
well of course.  Does anyone else have opinions on this?  Are
we trying to do too much in Fedora?  Should it remain a base
set of RPMs, with differently branded products deriving from it
externally?

References

1. http://www.ovirt.org/Home
2. http://resources.ovirt.org/pub/ovirt-3.4/

[Colin Walters]

On Fri, Jul 18, 2014, at 11:07 AM, Colin Walters wrote:



Let's look at an example of another project:

[1]http://www.ovirt.org/Home



Other references:



[2]http://openstack.redhat.com



It has its own yum repositories on Fedorapeople, though it
appears to also have RPMs regularly going into Fedora?  They,
like oVirt, also handle an EPEL version.



RDO is also oriented around scripted deployments.



Gluster:

[3]http://download.gluster.org/pub/gluster/glusterfs/LATEST/Fed
ora/What_About_Fedora.TXT



Looks like they're integrated into Fedora now.  But that makes
sense as Gluster is add-on software - it's not a new
installation/delivery vehicle itself.

References

1. http://www.ovirt.org/Home
2. http://openstack.redhat.com/Repositories
3. http://download.gluster.org/pub/gluster/glusterfs/LATEST/Fedora/What_Abou...

[Joe Brockmeier]

On 07/18/2014 10:07 AM, Colin Walters wrote:
>  Hi Stephen,
>   
>  Thanks for the response.
>   
>  On Fri, Jul 18, 2014, at 10:43 AM, Stephen John Smoogen wrote:
> >  
> > Most of the above are also in grumpy areas of infrastructure where
> > adding a person to fix it means they will need to learn a lot of other
> > things before it works well or doesn't snag up something else. It
> > would be good to add a person, but don't expect an instant win but
> > more instant pain in doing so. 
>   
>  Right.
>   
> > There is also a part where infrastructure and releng are separate
> > units in some ways and not separate in others. Some of your blockers
> > are on one and some are on another and some in between. I can really
> > answer on the content mirroring side as I believe we can figure out a
> > way to do that. Other parts are on koji developers and other
> > developers who work on signing. 
>   
>  Yeah.  The more I think about this, maybe Project Atomic should operate
>  on its own, deriving from Fedora, but with separate infrastructure. 
Yes and no. I think we should probably work as a Remix for the F21
timeframe while still trying to get as much in for the Cloud Product as
possible - and then try to be well-integrated with Fedora infra within
the F22 timeframe.

I know the infrastructure + releng folks are always overtasked, and I
don't want to overload them. At the same time, it'd be good to do this
as an official Fedora piece in the long term.

>  Let's look at an example of another project:
>  http://www.ovirt.org/Home
>   
>  They have their own ISO page <http://resources.ovirt.org/pub/ovirt-3.4/>
>  <http://resources.ovirt.org/pub/ovirt-3.4/> (and it's slick!), their own
>  GPG keys, their own release schedule, their own ovirt-release RPMs,
>  their own installation instructions, their own mirroring list.  And
>  presumably their own system administrators.
>   
>  And ultimately their own branding.  Maybe that's the right thing for
>  Atomic too?
>   
>  I can see a lot of advantages to that path; disadvantages as well of
>  course.  Does anyone else have opinions on this?  Are we trying to do
>  too much in Fedora?  Should it remain a base set of RPMs, with
>  differently branded products deriving from it externally? 
Can we take a middle path and simply treat the F21 cycle as a ramp-up?

Best,

jzb
-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

[Colin Walters]

On Fri, Jul 18, 2014, at 11:30 AM, Joe Brockmeier wrote:
>  Yes and no. I think we should probably work as a Remix for the F21
>  timeframe while still trying to get as much in for the Cloud Product as
>  possible - and then try to be well-integrated with Fedora infra within
>  the F22 timeframe. 
But the thing I'm really increasingly questioning is whether that
Fedora-branded endgame is right.

If it is right, why are RDO and oVirt separate?

[Joe Brockmeier]

On 07/18/2014 10:35 AM, Colin Walters wrote:
>  But the thing I'm really increasingly questioning is whether
> that
>  Fedora-branded endgame is right. 
I think so? In part, I would hope Fedora Atomic would also bring more
folks in to use Fedora *in* containers running on Atomic.

Also - the plan originally for Project Atomic was explicitly not to
develop a distribution ourselves, but help develop the tools/patterns,
etc. for Fedora, CentOS, RHEL, and potentially others to create Atomic
hosts. Long-term, would a Remix be too close to Atomic creating its own
distribution?

>  If it is right, why are RDO and oVirt separate? 
I'm probably not the right person to answer that. Are those models more
desirable than Atomic being part of the Cloud product?

Best,

jzb
-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

[Kevin Fenzi]

On Fri, 18 Jul 2014 08:43:30 -0600
Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:

>  On 18 July 2014 07:40, Colin Walters <walters(a)verbum.org&gt;
> wrote:
>  
>  > Hi,
>  >
>  > I feel like things aren't moving forward quickly enough with Atomic,
>  > both on the Docker base image side and the rpm-ostree host side.  A
>  > lot of that is to be expected - we're trying to introduce *two* new
>  > ways to deliver software beyond RPMs at the same time.  It's quite
>  > nontrivial. 
Sorry it's going slower than you would like. ;( 

...snip all the stuff I agree with from smooge... 

>  Let's look at blocking issues for Atomic in Fedora:
>  >
>  > - Having rpm-ostree be run as part of nightly compose 
I think the plan here was to get a koji plugin?

Mathieu Bridon wrote a koji plugin for us to do mashes in. 
We also know of another more generic plugin that lets you do 'commands
in a chroot' type of things, but it's not clear on it's licensing. 

Perhaps we can see if Mathieu can rework the koji plugin as a 'run
commands' type of thing and we can use it for atomic and also for
rawhide/branched?

I'm not sure there's a way to do this in the mean time, I guess we
could spin up another compose host have it it just run from cron... 

>  > - Content mirroring
>  >   * Server with SSL, some level of redundancy
>  >   * Open question around how often/when composes are done, and how
>  > much history is retained 
We can setup something with dl.fedoraproject.org now... and adjust as
we go?

>  > - GPG signing
>  > - Method of having Koji/ImageFactory pull mirrored OSTree repository
>  > content to compose ISOs
>  > - MirrorManager integration (no code written on ostree side yet)
>  >
>  Most of the above are also in grumpy areas of infrastructure where
>  adding a person to fix it means they will need to learn a lot of
>  other things before it works well or doesn't snag up something else.
>  It would be good to add a person, but don't expect an instant win but
>  more instant pain in doing so. 
Yeah. Of course adding people to these areas could also be a long term
win, as then we have more people who know them. But it's not a magic
fix. 

kevin

[Colin Walters]

On Fri, Jul 18, 2014, at 01:01 PM, Kevin Fenzi wrote:
>  I think the plan here was to get a koji plugin? 
I think Dennis was thinking it would be run as part of the compose
script, like pungi?  Although we didn't take minutes which was a
mistake, that was my understanding.
 
>  I'm not sure there's a way to do this in the mean time, I
> guess we
>  could spin up another compose host have it it just run from cron...  
That works, but it leads into the next question: Where does it write and
how is that data managed?  Is that data accessible to Koji?

The current flow design to create say a cloud qcow2 is:

Source -> koji -> RPMs -> createrepo -> rpm-ostree -> ostreerepo ->
koji
-> imagefactory -> qcow2

Note the fact that we bounce in and out of koji.  If we had koji running
rpm-ostree composes, it'd be involved 3 times.

The current flow for a boot iso is:

Source -> koji -> RPMs -> createrepo -> rpm-ostree -> ostreerepo ->
lorax
 
>  We can setup something with dl.fedoraproject.org now... and adjust
> as
>  we go? 
Ok, a multihomed box with SSL, but not globally mirrored (is that
correct)?  That would certainly be an improvement!

[Stephen John Smoogen]

On 18 July 2014 11:14, Colin Walters <walters(a)verbum.org&gt; wrote:

>  > We can setup something with dl.fedoraproject.org now... and adjust as
>  > we go?
>
>  Ok, a multihomed box with SSL, but not globally mirrored (is that
>  correct)?  That would certainly be an improvement!
>  ______
>  
Well it will be mirrored by the dozen or so systems that mirror
alt.fedorarproject.org It isn't mirrored as much as the main release tree.

If it looks like it needs more resources we can ask more mirrors to do so.
How does ostree grow over time in disk usage? Is it still the same git
repository you talked about in your blogs last year? [And if so does this
mean that the equivalent of /.git would grow over time?]

-- 
Stephen J Smoogen.

[Matthew Miller]

On Fri, Jul 18, 2014 at 09:40:59AM -0400, Colin Walters wrote:
>  Let's look at blocking issues for Atomic in Fedora:
>  - Having rpm-ostree be run as part of nightly compose 
As I understand it, Dennis is working on this. Dennis, anyone -- let me know
where particular help is needed. David Gay (oddshocks) is also available to
help (pretty much full time for a little while once fedimg is in production
in the next few days, as I understand it).

>  - Content mirroring
>    * Server with SSL, some level of redundancy
>    * Open question around how often/when composes are done, and how much
>    history is retained 
> From when we last talked, I thought you were going to bring up this
> last one 
as a policy question on the Fedora Cloud list? But also from our
discussion,
it doesn't look like keeping even full history would be hugely significant
-- 1.6GB over the lifetime, _worst case_. I also don't think that we need to
have this answered urgently... it's something we need to know by October,
right?

>  - GPG signing 
Can you elaborate here? If I remember right we had discussed this as a
release blocker but not necessarily an immediate blocker (for example, not
all RPMs in F21 branch are currently signed.)


>  - Method of having Koji/ImageFactory pull mirrored OSTree repository
>  content to compose ISOs 
I think this is just making the above mirror available to ImageFactory
during install.


>  - MirrorManager integration (no code written on ostree side yet)
>  To the best of my knowledge, code exists for all of the above except the
>  last, and the last bit is hard to write without having a deployment of
>  MirrorManager using it, which depends 
Again, let me know if it would help to have someone focused on this and I'll
see if I can find the right help.


>  I see two solution paths:
>  - Double down and fix the above, continue trying to sync Atomic with
>  Fedora 
Based on our conversations last week and earlier this week, I don't think
it's really so dire that it comes to "double down", but this is the
direction I'd like to continue in. It's important to Fedora Cloud, and it's
important to F21 messaging overall.

>  Would more regular meetings help? 
If it would, let's do it. :)

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Colin Walters]

On Fri, Jul 18, 2014, at 04:48 PM, Matthew Miller wrote:
>  
>  From when we last talked, I thought you were going to bring up this last
>  one
>  as a policy question on the Fedora Cloud list? But also from our
>  discussion,
>  it doesn't look like keeping even full history would be hugely
>  significant
>  -- 1.6GB over the lifetime, _worst case_. 
It's hard to give good numbers without a crystal ball for what packages
are going to change unfortunately.  But I would feel comfortable with
saying just a few gigabytes.

>  I also don't think that we need
>  to
>  have this answered urgently... it's something we need to know by October,
>  right? 
Right.

>  Can you elaborate here? If I remember right we had discussed this as
> a
>  release blocker but not necessarily an immediate blocker (for example,
>  not
>  all RPMs in F21 branch are currently signed.) 
Having the OSTree commits be signed will allow offline verification and
fetching over untrusted HTTP.

I think this is blocked on a decision about whether Fedora's GPG signing
could do detached signatures.

I talked with Mark Cox from Red Hat SRT, and we successfully prototyped
detatched signatures, and Trevor Jay validated the code.  It would be
unfortunate to implement an entirely new codepath for inline signatures,
particularly because this would break the ability to sign *after*
committing without changing the content.

(Think "promote existing testing build to stable)

But it's not out of the question, it just needs a conversation with the
relevant people.
 
>  I think this is just making the above mirror available to
> ImageFactory
>  during install. 
Yeah, additionally we need some equivalent of fedora-repo-rawhide.ks in
the spin-kickstarts which knows how to retrieve it.  Now that current
one is based on MirrorManager, which ends up blocking on that thread,
until we get content sync'd to something like dl.fedoraproject.org.
 
>  Again, let me know if it would help to have someone focused on this
> and
>  I'll
>  see if I can find the right help. 
This one is on my plate to figure out.

>  Based on our conversations last week and earlier this week, I
> don't think
>  it's really so dire that it comes to "double down", but this is the
>  direction I'd like to continue in. It's important to Fedora Cloud, and
>  it's
>  important to F21 messaging overall. 
Ok, thanks!

[Paul W. Frields]

On Fri, Jul 18, 2014 at 04:48:50PM -0400, Matthew Miller wrote:
>  On Fri, Jul 18, 2014 at 09:40:59AM -0400, Colin Walters wrote:
>  > Let's look at blocking issues for Atomic in Fedora:
>  > - Having rpm-ostree be run as part of nightly compose
>  
>  As I understand it, Dennis is working on this. Dennis, anyone -- let me know
>  where particular help is needed. David Gay (oddshocks) is also available to
>  help (pretty much full time for a little while once fedimg is in production
>  in the next few days, as I understand it). 
That's correct, and David, Dennis, Matt, and I talked about that as
well.  Before I left for vacation I believe David, Colin, Dennis, and
some others met to discuss about how to approach the ostree work, but
no tasks came out of it for David yet (at least then).

>  > - Content mirroring
>  >   * Server with SSL, some level of redundancy
>  >   * Open question around how often/when composes are done, and how much
>  >   history is retained
>  
>  From when we last talked, I thought you were going to bring up this last one
>  as a policy question on the Fedora Cloud list? But also from our discussion,
>  it doesn't look like keeping even full history would be hugely significant
>  -- 1.6GB over the lifetime, _worst case_. I also don't think that we need to
>  have this answered urgently... it's something we need to know by October,
>  right?
>  
>  > - GPG signing
>  
>  Can you elaborate here? If I remember right we had discussed this as a
>  release blocker but not necessarily an immediate blocker (for example, not
>  all RPMs in F21 branch are currently signed.)
>  
>  > - Method of having Koji/ImageFactory pull mirrored OSTree repository
>  > content to compose ISOs
>  
>  I think this is just making the above mirror available to ImageFactory
>  during install.
>  
>  
>  > - MirrorManager integration (no code written on ostree side yet)
>  > To the best of my knowledge, code exists for all of the above except the
>  > last, and the last bit is hard to write without having a deployment of
>  > MirrorManager using it, which depends
>  
>  Again, let me know if it would help to have someone focused on this and I'll
>  see if I can find the right help.
>  
>  > I see two solution paths:
>  > - Double down and fix the above, continue trying to sync Atomic with
>  > Fedora
>  
>  Based on our conversations last week and earlier this week, I don't think
>  it's really so dire that it comes to "double down", but this is the
>  direction I'd like to continue in. It's important to Fedora Cloud, and it's
>  important to F21 messaging overall.
>  
>  > Would more regular meetings help?
>  
>  If it would, let's do it. :) 
+1.  Given the speed and priority of Atomic, I think I'd benefit from
a regular checkpoint to see where things are blocked and making sure
specific tasks get assigned to someone to do, especially where it hits
my team.

I feel like we have a sort of split -- some tasks for which we have
resources for (like coding that David can help with), but not sure of
direction; and some tasks for which we have a better idea of direction
(the infra related stuff, to some extent), but not sure about
resources.  The resource problem we can probably solve, and we have
alternatives other than doubling down, or overloading everyone already
doing lots of stuff.  But we need to be able to quantify exactly
what's not getting done and why first, so I'd love to hash that out.

Is Fedocal what we'd use to set something up for early next week?

-- 
Paul W. Frields                                http://paul.frields.org/
  gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233  5906 ACDB C937 BD11 3717
  http://redhat.com/   -  -  -  -   http://pfrields.fedorapeople.org/
    The open source story continues to grow: http://opensource.com

[Joe Brockmeier]

On 07/18/2014 04:10 PM, Paul W. Frields wrote:
>  Is Fedocal what we'd use to set something up for early next week?
I set up a Doodle pool for people to see what times would work:

http://doodle.com/84sb3sda57ychmkm

Hopefully we can get this in Monday or Tuesday.

Best,

jzb
-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

[Stephen John Smoogen]

On 18 July 2014 16:49, Joe Brockmeier <jzb(a)redhat.com&gt; wrote:

>  On 07/18/2014 04:10 PM, Paul W. Frields wrote:
>  > Is Fedocal what we'd use to set something up for early next week?
>
>  I set up a Doodle pool for people to see what times would work:
>
>  http://doodle.com/84sb3sda57ychmkm
>
>  Hopefully we can get this in Monday or Tuesday.
>
>  
Kevin and I will be in PHX doing a buildout of hardware from Tuesday ->
Friday. Monday is about our only day we can do this.


-- 
Stephen J Smoogen.

[Kevin Fenzi]

On Fri, 18 Jul 2014 17:08:29 -0600
Stephen John Smoogen <smooge(a)gmail.com&gt; wrote:

>  On 18 July 2014 16:49, Joe Brockmeier <jzb(a)redhat.com&gt; wrote:
>  
>  > On 07/18/2014 04:10 PM, Paul W. Frields wrote:
>  > > Is Fedocal what we'd use to set something up for early next week?
>  >
>  > I set up a Doodle pool for people to see what times would work:
>  >
>  > http://doodle.com/84sb3sda57ychmkm
>  >
>  > Hopefully we can get this in Monday or Tuesday.
>  >
>  >
>  Kevin and I will be in PHX doing a buildout of hardware from Tuesday
>  -> Friday. Monday is about our only day we can do this. 
Could I make a plea for irc meeting too? ;) 

Or are there specific things that higher bandwith via phone would be
good for?

kevin

[Joe Brockmeier]

On 07/18/2014 06:16 PM, Kevin Fenzi wrote:
>  Could I make a plea for irc meeting too? ;) 
>  
>  Or are there specific things that higher bandwith via phone would be
>  good for? 
As long as we have a good agenda, I think everyone here is good to go
for IRC. (Please correct me if I'm wrong!)

Best,

jzb
-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

Time Set: 11 a.m. Eastern (was Re: Atomic status)

[Joe Brockmeier]

On 07/18/2014 05:49 PM, Joe Brockmeier wrote:
>  On 07/18/2014 04:10 PM, Paul W. Frields wrote:
> > Is Fedocal what we'd use to set something up for early next week? 
Looks like the best time is 11 a.m. Eastern. I'm sending a Zimbra invite
to the list + folks who are RHT internal. All are welcome to attend.
Let's meet in the #atomic channel on Freenode?

Best,

jzb
-- 
Joe Brockmeier | Principal Cloud & Storage Analyst
jzb(a)redhat.com | http://community.redhat.com/
Twitter: @jzb  | http://dissociatedpress.net/

[Michael Scherer]

Le vendredi 18 juillet 2014 à 09:40 -0400, Colin Walters a écrit :
>  Hi,
>  
>  I feel like things aren't moving forward quickly enough with Atomic,
>  both on the Docker base image side and the rpm-ostree host side.  A lot
>  of that is to be expected - we're trying to introduce *two* new ways to
>  deliver software beyond RPMs at the same time.  It's quite nontrivial.
>  
>  There's also:
>  1) We're trying to do new projects, but we haven't added significant
>  human resources
>  2) This is also the first Fedora.next release
>  3) We don't know how popular the whole Atomic/Docker model will be until
>  we do a stable release really, but it's hard to get current
>  Infrastructure people to spend time on it without knowing how valuable
>  it is...
>  
>  Let's look at blocking issues for Atomic in Fedora:
>  
>  - Having rpm-ostree be run as part of nightly compose
>  - Content mirroring
>    * Server with SSL, some level of redundancy
>    * Open question around how often/when composes are done, and how much
>    history is retained
>  - GPG signing
>  - Method of having Koji/ImageFactory pull mirrored OSTree repository
>  content to compose ISOs
>  - MirrorManager integration (no code written on ostree side yet)
>  
>  To the best of my knowledge, code exists for all of the above except the
>  last, and the last bit is hard to write without having a deployment of
>  MirrorManager using it, which depends
>  
>  I see two solution paths:
>  
>  - Double down and fix the above, continue trying to sync Atomic with
>  Fedora
>    * Have more time from current Infra people on above
>    * Have me/other members of my team join infra
>    * Mix of both?
>  - Have Atomic be a Remix, partially in infra, partially outside
>    - Have custom GPG key
>    - Continue doing composes on the atomic01.qa.fedoraproject.org machine
>    - Mirror to something like Amazon CloudFront (would require external
>    sponsorship from Red Hat) 
One of the issue we will have is that Fedora infrastructure is used in
production for Fedora, so i think we will not have as much agility if we
start to target it right from the start. 

We also have the issue that there is currently a proposal for a Centos
based atomic image
( http://lists.centos.org/pipermail/centos-devel/2014-June/011233.html
), and I think making thing work for Fedora and Centos at the same time
may add some contraints to us.

So i would be in favor of having atomic on a separate infrastructure for
now, and later try to push as much as possible on Fedora (and Centos)
once the dust settle regarding Atomic infra.

So i would propose:
- have rpm-ostree compose be done outside of koji for now. thanks to
fedmsg, this should be hard. This will permit to get a idea of the
ressources used for this, and will not put more burden on the current
host just for atomic. In fact, we could even imagine have a different
schedule for compose, based on the load. 

- have the gpg key being separate from the one of Fedora, and signing
done somewhere else. This would free us of the contraint we have
regarding access of Fedora key for now. It also permit us to test key
replacement more freely, something we will need to do in the future.

- use a set of mirrors outside Fedora ( i would use 3 or 4 VM properly
placed on the globe, under our control rather than using a CDN ). It
permit to know the requirement in term of size after a few months as
well as bandwidth in a more precise way since it will not mixed with
others stuff. in turn, this permit to data to do evaluate the sizing
when we move to Fedora and Centos infra. I also think that this is the
only way to have a shared ssl cert for the http server, as that seems to
be a requirement ( ie, we cannot do that with a regular set of mirrors
operated by volunteer, so that's something we will have to fix somehow
). Having discussed off-list about this, Stephen spoke also of the GPL
requirement regarding source, so that's something to keep in mind.


- the same go for imagefactory. it can be run on a separate host, and
maybe with a separate schedule, or a separate schedule ( ie, we could
make update during the release freeze, etc )

Then once everything is ok, we can start to move things back to Centos
and Fedora, one service after the others.

Doing this also break the loop, ie, we can make sure the project is
successful using Fedora package and later integrate it to Fedora rather
than just being a remix/spin/derivative.

Of course, we will discuss in details during meeting, but I prefer to
provides the details of the proposal to go forward before.

So if people agree with the idea of going separate for now, and then
come back with data and tested infrastructure, I can try to provide a
few hosting ressources for the compose ( nothing fancy for now, but we
can always try to find more ressources, i have some idea regarding who
could sponsor stuff ), and help as part of $dayjob for setting part of
the infrastructure. I tend to use ansible, so the work done for
deploying the part of the infra can then be reused for Fedora.


-- 
Michael Scherer

[Michael Scherer]

On Sat, Jul 19, 2014 at 03:32:14AM +0200, Michael Scherer wrote:

>  So i would propose:
>  - have rpm-ostree compose be done outside of koji for now. thanks to
>  fedmsg, this should be hard. 
( of course, everybody will have infered that i mean "shouldn't be hard",
and that 3 in the morning is not the best time for me to send a email )

-- 
Michael Scherer

[Colin Walters]

>  One of the issue we will have is that Fedora infrastructure is used
> in
>  production for Fedora, so i think we will not have as much agility if we
>  start to target it right from the start.  
Right.  But on the other hand if we do it, then it is actually
integrated in a way that's potentially usable by other downstreams.

I've certainly oscillated on this myself, sometimes not in a stable
fashion ;)  Your below plan is potentially quite useful though if we end
up doing anything more experimental in the future.  For example, one
thing I am thinking about in the background is having a Docker registry,
plus code to do automated rebuilds *from RPMs* (and potentially other
bits) in a reliable fashion.  There's also having experimental
builds/integration on the host side.

But it seems like now that there's progress, let's keep trying to move
in Fedora?