Proposal for Reorienting Two-Week Atomic (Ticket #6313)

[Adam Miller]

Hello all,
    This is a proposal for implementing a solution for releng ticket #6313[0]

Currently the compose (pungi) produces an Atomic ostree, installable ISO, and
various cloud images nightly. Those artifacts are then tested via AutoCloud.
I'm proposing that we add a second "new" repo, every two weeks we will sync
the latest commit from the current repo/ref that has passed testing in
AutoCloud then using the corresponding compose, reproduce the ISO and various
cloud images using the exact same content but pointing to the new remote ref
(which we will call fedora-atomic/24/x86_64/stable/docker-host) that is
only updated every two-weeks. This would be added to the current two-week
atomic release script instead of impacting pungi or other current processes.

We then will add documentation for users who want to switch between release
streams via different repos/refs, either nightly updates.

The main concerns are:
- We would need to manage the versioning of the ref that gets synced over
  ourselves but I think we can just version it as the compose id for
  consistency.
- Need to figure out how to extract the ostree repo/ref commit from AutoCloud
  to ensure we're getting the right thing. This should be predictable though
  since the artifacts are built from an ostree created in the compose.

[0] - https://fedorahosted.org/rel-eng/ticket/6313

[Matthew Miller]

On Thu, Aug 18, 2016 at 11:35:17AM -0500, Adam Miller wrote:
>  Currently the compose (pungi) produces an Atomic ostree, installable
> ISO, and
>  various cloud images nightly. Those artifacts are then tested via AutoCloud.
>  I'm proposing that we add a second "new" repo, every two weeks we will
> sync
>  the latest commit from the current repo/ref that has passed testing in 
Presumably everything in the new repo could be hardlinks, so there
wouldn't be much by way of space increase?

Do we want to also have a mechanism for adding off-cycle critical
security updates? 

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Jason Brooks]

On Thu, Aug 18, 2016 at 10:10 AM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Aug 18, 2016 at 11:35:17AM -0500, Adam Miller wrote:
> > Currently the compose (pungi) produces an Atomic ostree, installable ISO, and
> > various cloud images nightly. Those artifacts are then tested via AutoCloud.
> > I'm proposing that we add a second "new" repo, every two weeks we will
> sync
> > the latest commit from the current repo/ref that has passed testing in
>
>  Presumably everything in the new repo could be hardlinks, so there
>  wouldn't be much by way of space increase?
>
>  Do we want to also have a mechanism for adding off-cycle critical
>  security updates? 
Someone on the two week stream could manually add a critical update
with `ostree admin unlock` or they could rebase to the continuous
stream (assuming the two-week is the default) to get the update. Would
handling this with docs and manual intervention be sufficient?

>
> --
> Matthew Miller
> <mattdm(a)fedoraproject.org&gt;
> Fedora Project Leader
> _______________________________________________
> rel-eng mailing list
> rel-eng(a)lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/[e-mail]

[Matthew Miller]

On Thu, Aug 18, 2016 at 10:21:48AM -0700, Jason Brooks wrote:
>  Someone on the two week stream could manually add a critical update
>  with `ostree admin unlock` or they could rebase to the continuous
>  stream (assuming the two-week is the default) to get the update. Would
>  handling this with docs and manual intervention be sufficient? 
I think so, yeah. Hopefully it's a rare occurrence.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Adam Miller]

On Thu, Aug 18, 2016 at 12:10 PM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Aug 18, 2016 at 11:35:17AM -0500, Adam Miller wrote:
> > Currently the compose (pungi) produces an Atomic ostree, installable ISO, and
> > various cloud images nightly. Those artifacts are then tested via AutoCloud.
> > I'm proposing that we add a second "new" repo, every two weeks we will
> sync
> > the latest commit from the current repo/ref that has passed testing in
>
>  Presumably everything in the new repo could be hardlinks, so there
>  wouldn't be much by way of space increase?
>  
It could be, yes.

>  Do we want to also have a mechanism for adding off-cycle critical
>  security updates?
>  
We would just run the release script by hand off-schedule. The script
itself doesn't really care when it's ran, it just releases the
artifact.

-AdamM

> --
> Matthew Miller
> <mattdm(a)fedoraproject.org&gt;
> Fedora Project Leader

[Jason Brooks]

On Thu, Aug 18, 2016 at 9:35 AM, Adam Miller
<maxamillion(a)fedoraproject.org&gt; wrote:
>  Hello all,
>      This is a proposal for implementing a solution for releng ticket #6313[0]
>
>  Currently the compose (pungi) produces an Atomic ostree, installable ISO, and
>  various cloud images nightly. Those artifacts are then tested via AutoCloud.
>  I'm proposing that we add a second "new" repo, every two weeks we will
> sync
>  the latest commit from the current repo/ref that has passed testing in
>  AutoCloud then using the corresponding compose, reproduce the ISO and various
>  cloud images using the exact same content but pointing to the new remote ref
>  (which we will call fedora-atomic/24/x86_64/stable/docker-host) that is
>  only updated every two-weeks. This would be added to the current two-week
>  atomic release script instead of impacting pungi or other current processes. 
This sounds good to me. Would this two-week repo become the default
repo on the images, and you'd rebase to the continuous one if that's
what you wanted, or would it be the other way around?

>
> We then will add documentation for users who want to switch between release
> streams via different repos/refs, either nightly updates.
>
> The main concerns are:
> - We would need to manage the versioning of the ref that gets synced over
>   ourselves but I think we can just version it as the compose id for
>   consistency.
> - Need to figure out how to extract the ostree repo/ref commit from AutoCloud
>   to ensure we're getting the right thing. This should be predictable though
>   since the artifacts are built from an ostree created in the compose.
>
> [0] - https://fedorahosted.org/rel-eng/ticket/6313
> _______________________________________________
> rel-eng mailing list
> rel-eng(a)lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/[e-mail]

[Adam Miller]

On Thu, Aug 18, 2016 at 12:19 PM, Jason Brooks <jbrooks(a)redhat.com&gt; wrote:
>  On Thu, Aug 18, 2016 at 9:35 AM, Adam Miller
>  <maxamillion(a)fedoraproject.org&gt; wrote:
> > Hello all,
> >     This is a proposal for implementing a solution for releng ticket #6313[0]
> >
> > Currently the compose (pungi) produces an Atomic ostree, installable ISO, and
> > various cloud images nightly. Those artifacts are then tested via AutoCloud.
> > I'm proposing that we add a second "new" repo, every two weeks we will
> sync
> > the latest commit from the current repo/ref that has passed testing in
> > AutoCloud then using the corresponding compose, reproduce the ISO and various
> > cloud images using the exact same content but pointing to the new remote ref
> > (which we will call fedora-atomic/24/x86_64/stable/docker-host) that is
> > only updated every two-weeks. This would be added to the current two-week
> > atomic release script instead of impacting pungi or other current processes.
>
>  This sounds good to me. Would this two-week repo become the default
>  repo on the images, and you'd rebase to the continuous one if that's
>  what you wanted, or would it be the other way around?
>  
New Two-Week repo would be default and we'd document how to switch to
the continuously updated repo that comes out of bodhi for users who
would like that option. The current compose repos would then become
both the thing that is tested in AutoCloud as an unit and the origin
of content that lands in the Two-Week repo.

-AdamM

>>
>> We then will add documentation for users who want to switch between release
>> streams via different repos/refs, either nightly updates.
>>
>> The main concerns are:
>> - We would need to manage the versioning of the ref that gets synced over
>>   ourselves but I think we can just version it as the compose id for
>>   consistency.
>> - Need to figure out how to extract the ostree repo/ref commit from AutoCloud
>>   to ensure we're getting the right thing. This should be predictable though
>>   since the artifacts are built from an ostree created in the compose.
>>
>> [0] - https://fedorahosted.org/rel-eng/ticket/6313
>> _______________________________________________
>> rel-eng mailing list
>> rel-eng(a)lists.fedoraproject.org
>> https://lists.fedoraproject.org/admin/lists/[e-mail]

[Matthew Miller]

On Thu, Aug 18, 2016 at 01:44:55PM -0500, Adam Miller wrote:
>  New Two-Week repo would be default and we'd document how to
> switch to
>  the continuously updated repo that comes out of bodhi for users who
>  would like that option. The current compose repos would then become
>  both the thing that is tested in AutoCloud as an unit and the origin
>  of content that lands in the Two-Week repo. 
Probably because of timestamps these won't be bit-for-bit identical,
right? I think we need to either guarantee that they are (which seems
like a tall order, unless I am being dense), or else run the two-week
artifacts through another round of automated testing.

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Adam Miller]

On Thu, Aug 18, 2016 at 1:56 PM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Aug 18, 2016 at 01:44:55PM -0500, Adam Miller wrote:
> > New Two-Week repo would be default and we'd document how to switch to
> > the continuously updated repo that comes out of bodhi for users who
> > would like that option. The current compose repos would then become
> > both the thing that is tested in AutoCloud as an unit and the origin
> > of content that lands in the Two-Week repo.
>
>  Probably because of timestamps these won't be bit-for-bit identical,
>  right? I think we need to either guarantee that they are (which seems
>  like a tall order, unless I am being dense), or else run the two-week
>  artifacts through another round of automated testing. 
I'm don't understand the question.

-AdamM

>
> --
> Matthew Miller
> <mattdm(a)fedoraproject.org&gt;
> Fedora Project Leader

[Matthew Miller]

On Thu, Aug 18, 2016 at 04:10:05PM -0500, Adam Miller wrote:
>  > Probably because of timestamps these won't be bit-for-bit
> identical,
>  > right? I think we need to either guarantee that they are (which seems
>  > like a tall order, unless I am being dense), or else run the two-week
>  > artifacts through another round of automated testing.
>  I'm don't understand the question. 
The proposal calls for ISOs and various cloud images to be created
twice, right? From the same content, but still new artifacts. And it
mentions the _first_ set going through AutoCloud but not the second.
Correct me if I'm being dumb, please. :)

-- 
Matthew Miller
<mattdm(a)fedoraproject.org&gt;
Fedora Project Leader

[Adam Miller]

On Thu, Aug 18, 2016 at 4:16 PM, Matthew Miller
<mattdm(a)fedoraproject.org&gt; wrote:
>  On Thu, Aug 18, 2016 at 04:10:05PM -0500, Adam Miller wrote:
> > > Probably because of timestamps these won't be bit-for-bit identical,
> > > right? I think we need to either guarantee that they are (which seems
> > > like a tall order, unless I am being dense), or else run the two-week
> > > artifacts through another round of automated testing.
> > I'm don't understand the question.
>
>  The proposal calls for ISOs and various cloud images to be created
>  twice, right? From the same content, but still new artifacts. And it
>  mentions the _first_ set going through AutoCloud but not the second.
>  Correct me if I'm being dumb, please. :) 
It does, you are correct and it's likely something we need to keep in
mind as well as weigh the benefits of re-running the tests. However,
if we use the same tools to build the same artifacts out of the same
compose content and it doesn't work the same as the original creation
of those artifacts then we have bigger issues than it not passing
AutoCloud tests. (Though it could be a good metric for verification)

-AdamM

>
> --
> Matthew Miller
> <mattdm(a)fedoraproject.org&gt;
> Fedora Project Leader

[Dennis Gilmore]

On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
>  Hello all,
>      This is a proposal for implementing a solution for releng ticket
>  #6313[0]
>  
>  Currently the compose (pungi) produces an Atomic ostree, installable ISO,
>  and various cloud images nightly. Those artifacts are then tested via
>  AutoCloud. I'm proposing that we add a second "new" repo, every two weeks
>  we will sync the latest commit from the current repo/ref that has passed
>  testing in AutoCloud then using the corresponding compose, reproduce the
>  ISO and various cloud images using the exact same content but pointing to
>  the new remote ref (which we will call
>  fedora-atomic/24/x86_64/stable/docker-host) that is only updated every
>  two-weeks. This would be added to the current two-week atomic release
>  script instead of impacting pungi or other current processes. 
The compose currently
does not produce a ostree repo so it owuld have to.  we 
could achive what you propose today though by tagging the commit used in the 
repo with that tag.

Dennis

>  We then will add documentation for users who want to switch between
> release
>  streams via different repos/refs, either nightly updates.
>  
>  The main concerns are:
>  - We would need to manage the versioning of the ref that gets synced over
>    ourselves but I think we can just version it as the compose id for
>    consistency.
>  - Need to figure out how to extract the ostree repo/ref commit from
>  AutoCloud to ensure we're getting the right thing. This should be
>  predictable though since the artifacts are built from an ostree created in
>  the compose.
>  
>  [0] - https://fedorahosted.org/rel-eng/ticket/6313
>  _______________________________________________
>  rel-eng mailing list
>  rel-eng(a)lists.fedoraproject.org
>  https://lists.fedoraproject.org/admin/lists/[e-mail]

[Adam Miller]

On Fri, Aug 19, 2016 at 9:54 AM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
>  On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
> > Hello all,
> >     This is a proposal for implementing a solution for releng ticket
> > #6313[0]
> >
> > Currently the compose (pungi) produces an Atomic ostree, installable ISO,
> > and various cloud images nightly. Those artifacts are then tested via
> > AutoCloud. I'm proposing that we add a second "new" repo, every two
> weeks
> > we will sync the latest commit from the current repo/ref that has passed
> > testing in AutoCloud then using the corresponding compose, reproduce the
> > ISO and various cloud images using the exact same content but pointing to
> > the new remote ref (which we will call
> > fedora-atomic/24/x86_64/stable/docker-host) that is only updated every
> > two-weeks. This would be added to the current two-week atomic release
> > script instead of impacting pungi or other current processes.
>  The compose currently does not produce a ostree repo so it owuld have to.  we
>  could achive what you propose today though by tagging the commit used in the
>  repo with that tag. 
Ah ok, I didn't realize the compose wasn't producing an ostree. Is the
ostree that's used in the Atomic ISO and cloud/VM images produced
during the compose not from the compose content?

-AdamM

>
> Dennis
>
>> We then will add documentation for users who want to switch between release
>> streams via different repos/refs, either nightly updates.
>>
>> The main concerns are:
>> - We would need to manage the versioning of the ref that gets synced over
>>   ourselves but I think we can just version it as the compose id for
>>   consistency.
>> - Need to figure out how to extract the ostree repo/ref commit from
>> AutoCloud to ensure we're getting the right thing. This should be
>> predictable though since the artifacts are built from an ostree created in
>> the compose.
>>
>> [0] - https://fedorahosted.org/rel-eng/ticket/6313
>> _______________________________________________
>> rel-eng mailing list
>> rel-eng(a)lists.fedoraproject.org
>> https://lists.fedoraproject.org/admin/lists/[e-mail]
>
> _______________________________________________
> rel-eng mailing list
> rel-eng(a)lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/[e-mail]

[Dennis Gilmore]

On Friday, August 19, 2016 3:31:31 PM CDT Adam Miller wrote:
>  On Fri, Aug 19, 2016 at 9:54 AM, Dennis Gilmore
> <dennis(a)ausil.us&gt; wrote:
>  > On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
>  >> Hello all,
>  >> 
>  >>     This is a proposal for implementing a solution for releng ticket
>  >> 
>  >> #6313[0]
>  >> 
>  >> Currently the compose (pungi) produces an Atomic ostree, installable ISO,
>  >> and various cloud images nightly. Those artifacts are then tested via
>  >> AutoCloud. I'm proposing that we add a second "new" repo, every
> two weeks
>  >> we will sync the latest commit from the current repo/ref that has passed
>  >> testing in AutoCloud then using the corresponding compose, reproduce the
>  >> ISO and various cloud images using the exact same content but pointing to
>  >> the new remote ref (which we will call
>  >> fedora-atomic/24/x86_64/stable/docker-host) that is only updated every
>  >> two-weeks. This would be added to the current two-week atomic release
>  >> script instead of impacting pungi or other current processes.
>  > 
>  > The compose currently does not produce a ostree repo so it owuld have to. 
>  > we could achive what you propose today though by tagging the commit used
>  > in the repo with that tag.
>  
>  Ah ok, I didn't realize the compose wasn't producing an ostree. Is the
>  ostree that's used in the Atomic ISO and cloud/VM images produced
>  during the compose not from the compose content? 
The ostree used is produced by
bodhi as part of the updates push cycle. We 
just grab and put in whatever is there.

Dennis
>  -AdamM
>  
>  > Dennis
>  > 
>  >> We then will add documentation for users who want to switch between
>  >> release
>  >> streams via different repos/refs, either nightly updates.
>  >> 
>  >> The main concerns are:
>  >> - We would need to manage the versioning of the ref that gets synced over
>  >> 
>  >>   ourselves but I think we can just version it as the compose id for
>  >>   consistency.
>  >> 
>  >> - Need to figure out how to extract the ostree repo/ref commit from
>  >> AutoCloud to ensure we're getting the right thing. This should be
>  >> predictable though since the artifacts are built from an ostree created
>  >> in
>  >> the compose.
>  >> 
>  >> [0] - https://fedorahosted.org/rel-eng/ticket/6313
>  >> _______________________________________________
>  >> rel-eng mailing list
>  >> rel-eng(a)lists.fedoraproject.org
>  >> https://lists.fedoraproject.org/admin/lists/[e-mail].o
>  >> rg
>  > 
>  > _______________________________________________
>  > rel-eng mailing list
>  > rel-eng(a)lists.fedoraproject.org
>  > https://lists.fedoraproject.org/admin/lists/[e-mail]
>  > g
>  
>  _______________________________________________
>  rel-eng mailing list
>  rel-eng(a)lists.fedoraproject.org
>  https://lists.fedoraproject.org/admin/lists/[e-mail]

[Adam Miller]

On Sun, Aug 21, 2016 at 4:14 PM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
>  On Friday, August 19, 2016 3:31:31 PM CDT Adam Miller wrote:
> > On Fri, Aug 19, 2016 at 9:54 AM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
> > > On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
> > >> Hello all,
> > >>
> > >>     This is a proposal for implementing a solution for releng ticket
> > >>
> > >> #6313[0]
> > >>
> > >> Currently the compose (pungi) produces an Atomic ostree, installable ISO,
> > >> and various cloud images nightly. Those artifacts are then tested via
> > >> AutoCloud. I'm proposing that we add a second "new" repo,
> every two weeks
> > >> we will sync the latest commit from the current repo/ref that has passed
> > >> testing in AutoCloud then using the corresponding compose, reproduce the
> > >> ISO and various cloud images using the exact same content but pointing to
> > >> the new remote ref (which we will call
> > >> fedora-atomic/24/x86_64/stable/docker-host) that is only updated every
> > >> two-weeks. This would be added to the current two-week atomic release
> > >> script instead of impacting pungi or other current processes.
> > >
> > > The compose currently does not produce a ostree repo so it owuld have to.
> > > we could achive what you propose today though by tagging the commit used
> > > in the repo with that tag.
> >
> > Ah ok, I didn't realize the compose wasn't producing an ostree. Is the
> > ostree that's used in the Atomic ISO and cloud/VM images produced
> > during the compose not from the compose content?
>  The ostree used is produced by bodhi as part of the updates push cycle. We
>  just grab and put in whatever is there. 
Doesn't that defeat part of the point of the compose? I was under the
impression that the compose was supposed to be able to reproduce all
it's artifacts from the content within the compose itself, is that
incorrect?

-AdamM

>
> Dennis
>> -AdamM
>>
>> > Dennis
>> >
>> >> We then will add documentation for users who want to switch between
>> >> release
>> >> streams via different repos/refs, either nightly updates.
>> >>
>> >> The main concerns are:
>> >> - We would need to manage the versioning of the ref that gets synced
over
>> >>
>> >>   ourselves but I think we can just version it as the compose id for
>> >>   consistency.
>> >>
>> >> - Need to figure out how to extract the ostree repo/ref commit from
>> >> AutoCloud to ensure we're getting the right thing. This should be
>> >> predictable though since the artifacts are built from an ostree created
>> >> in
>> >> the compose.
>> >>
>> >> [0] - https://fedorahosted.org/rel-eng/ticket/6313
>> >> _______________________________________________
>> >> rel-eng mailing list
>> >> rel-eng(a)lists.fedoraproject.org
>> >>
https://lists.fedoraproject.org/admin/lists/[e-mail].o
>> >> rg
>> >
>> > _______________________________________________
>> > rel-eng mailing list
>> > rel-eng(a)lists.fedoraproject.org
>> > https://lists.fedoraproject.org/admin/lists/[e-mail]
>> > g
>>
>> _______________________________________________
>> rel-eng mailing list
>> rel-eng(a)lists.fedoraproject.org
>> https://lists.fedoraproject.org/admin/lists/[e-mail]
>
>
> _______________________________________________
> rel-eng mailing list
> rel-eng(a)lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/[e-mail]
>

[Dennis Gilmore]

On Tuesday, August 23, 2016 9:49:06 AM CDT Adam Miller wrote:
>  On Sun, Aug 21, 2016 at 4:14 PM, Dennis Gilmore
> <dennis(a)ausil.us&gt; wrote:
>  > On Friday, August 19, 2016 3:31:31 PM CDT Adam Miller wrote:
>  >> On Fri, Aug 19, 2016 at 9:54 AM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
>  >> > On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
>  >> >> Hello all,
>  >> >> 
>  >> >>     This is a proposal for implementing a solution for releng ticket
>  >> >> 
>  >> >> #6313[0]
>  >> >> 
>  >> >> Currently the compose (pungi) produces an Atomic ostree, installable
>  >> >> ISO,
>  >> >> and various cloud images nightly. Those artifacts are then tested via
>  >> >> AutoCloud. I'm proposing that we add a second "new" repo,
> every two
>  >> >> weeks
>  >> >> we will sync the latest commit from the current repo/ref that has
>  >> >> passed
>  >> >> testing in AutoCloud then using the corresponding compose, reproduce
>  >> >> the
>  >> >> ISO and various cloud images using the exact same content but pointing
>  >> >> to
>  >> >> the new remote ref (which we will call
>  >> >> fedora-atomic/24/x86_64/stable/docker-host) that is only updated every
>  >> >> two-weeks. This would be added to the current two-week atomic release
>  >> >> script instead of impacting pungi or other current processes.
>  >> > 
>  >> > The compose currently does not produce a ostree repo so it owuld have
>  >> > to.
>  >> > we could achive what you propose today though by tagging the commit
>  >> > used
>  >> > in the repo with that tag.
>  >> 
>  >> Ah ok, I didn't realize the compose wasn't producing an ostree. Is the
>  >> ostree that's used in the Atomic ISO and cloud/VM images produced
>  >> during the compose not from the compose content?
>  > 
>  > The ostree used is produced by bodhi as part of the updates push cycle. We
>  > just grab and put in whatever is there.
>  
>  Doesn't that defeat part of the point of the compose? I was under the
>  impression that the compose was supposed to be able to reproduce all
>  it's artifacts from the content within the compose itself, is that
>  incorrect? 
the compose is supposed to be able to reproduce the artifacts of the compose, 
the ostree repo is not an artifact of the compose, it is an input to it, the 
same as the GA fedora rpm repo and updates rpm repos are. we do not make those 
either. because we do not make all the things for the compsoe, because that is 
what was requetsed it breaks some of the guarantees we have with the nightly 
branched/rawhide and milestone composes. That is what was demanded of us. It 
is what we have delivered as a result.

Dennis

[Adam Miller]

On Tue, Aug 23, 2016 at 10:41 AM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
>  On Tuesday, August 23, 2016 9:49:06 AM CDT Adam Miller wrote:
> > On Sun, Aug 21, 2016 at 4:14 PM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
> > > On Friday, August 19, 2016 3:31:31 PM CDT Adam Miller wrote:
> > >> On Fri, Aug 19, 2016 at 9:54 AM, Dennis Gilmore <dennis(a)ausil.us&gt;
> wrote:
> > >> > On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
> > >> >> Hello all,
> > >> >>
> > >> >>     This is a proposal for implementing a solution for releng
> ticket
> > >> >>
> > >> >> #6313[0]
> > >> >>
> > >> >> Currently the compose (pungi) produces an Atomic ostree,
> installable
> > >> >> ISO,
> > >> >> and various cloud images nightly. Those artifacts are then tested
> via
> > >> >> AutoCloud. I'm proposing that we add a second "new"
> repo, every two
> > >> >> weeks
> > >> >> we will sync the latest commit from the current repo/ref that has
> > >> >> passed
> > >> >> testing in AutoCloud then using the corresponding compose,
> reproduce
> > >> >> the
> > >> >> ISO and various cloud images using the exact same content but
> pointing
> > >> >> to
> > >> >> the new remote ref (which we will call
> > >> >> fedora-atomic/24/x86_64/stable/docker-host) that is only updated
> every
> > >> >> two-weeks. This would be added to the current two-week atomic
> release
> > >> >> script instead of impacting pungi or other current processes.
> > >> >
> > >> > The compose currently does not produce a ostree repo so it owuld have
> > >> > to.
> > >> > we could achive what you propose today though by tagging the commit
> > >> > used
> > >> > in the repo with that tag.
> > >>
> > >> Ah ok, I didn't realize the compose wasn't producing an ostree. Is
> the
> > >> ostree that's used in the Atomic ISO and cloud/VM images produced
> > >> during the compose not from the compose content?
> > >
> > > The ostree used is produced by bodhi as part of the updates push cycle. We
> > > just grab and put in whatever is there.
> >
> > Doesn't that defeat part of the point of the compose? I was under the
> > impression that the compose was supposed to be able to reproduce all
> > it's artifacts from the content within the compose itself, is that
> > incorrect?
>
>  the compose is supposed to be able to reproduce the artifacts of the compose,
>  the ostree repo is not an artifact of the compose, it is an input to it, the
>  same as the GA fedora rpm repo and updates rpm repos are. we do not make those
>  either. because we do not make all the things for the compsoe, because that is
>  what was requetsed it breaks some of the guarantees we have with the nightly
>  branched/rawhide and milestone composes. That is what was demanded of us. It
>  is what we have delivered as a result. 
Ah ok, I was mistaken about that then.

I'll need to figure out how to get the exact OSTree commit then and I
can just version it with the compose id anyways since it will still
correlate all said and done.

Otherwise, does this sound like an acceptable approach to everyone?

-AdamM

>
> Dennis
> _______________________________________________
> rel-eng mailing list
> rel-eng(a)lists.fedoraproject.org
> https://lists.fedoraproject.org/admin/lists/[e-mail]
>

[Adam Miller]

On Wed, Aug 24, 2016 at 5:08 PM, Adam Miller
<maxamillion(a)fedoraproject.org&gt; wrote:
>  On Tue, Aug 23, 2016 at 10:41 AM, Dennis Gilmore
> <dennis(a)ausil.us&gt; wrote:
> > On Tuesday, August 23, 2016 9:49:06 AM CDT Adam Miller wrote:
> >> On Sun, Aug 21, 2016 at 4:14 PM, Dennis Gilmore <dennis(a)ausil.us&gt; wrote:
> >> > On Friday, August 19, 2016 3:31:31 PM CDT Adam Miller wrote:
> >> >> On Fri, Aug 19, 2016 at 9:54 AM, Dennis Gilmore <dennis(a)ausil.us&gt;
> wrote:
> >> >> > On Thursday, August 18, 2016 11:35:17 AM CDT Adam Miller wrote:
> >> >> >> Hello all,
> >> >> >>
> >> >> >>     This is a proposal for implementing a solution for releng
> ticket
> >> >> >>
> >> >> >> #6313[0]
> >> >> >>
> >> >> >> Currently the compose (pungi) produces an Atomic ostree,
> installable
> >> >> >> ISO,
> >> >> >> and various cloud images nightly. Those artifacts are then
> tested via
> >> >> >> AutoCloud. I'm proposing that we add a second
> "new" repo, every two
> >> >> >> weeks
> >> >> >> we will sync the latest commit from the current repo/ref that
> has
> >> >> >> passed
> >> >> >> testing in AutoCloud then using the corresponding compose,
> reproduce
> >> >> >> the
> >> >> >> ISO and various cloud images using the exact same content but
> pointing
> >> >> >> to
> >> >> >> the new remote ref (which we will call
> >> >> >> fedora-atomic/24/x86_64/stable/docker-host) that is only
> updated every
> >> >> >> two-weeks. This would be added to the current two-week atomic
> release
> >> >> >> script instead of impacting pungi or other current processes.
> >> >> >
> >> >> > The compose currently does not produce a ostree repo so it owuld
> have
> >> >> > to.
> >> >> > we could achive what you propose today though by tagging the
> commit
> >> >> > used
> >> >> > in the repo with that tag.
> >> >>
> >> >> Ah ok, I didn't realize the compose wasn't producing an ostree.
> Is the
> >> >> ostree that's used in the Atomic ISO and cloud/VM images produced
> >> >> during the compose not from the compose content?
> >> >
> >> > The ostree used is produced by bodhi as part of the updates push cycle. We
> >> > just grab and put in whatever is there.
> >>
> >> Doesn't that defeat part of the point of the compose? I was under the
> >> impression that the compose was supposed to be able to reproduce all
> >> it's artifacts from the content within the compose itself, is that
> >> incorrect?
> >
> > the compose is supposed to be able to reproduce the artifacts of the compose,
> > the ostree repo is not an artifact of the compose, it is an input to it, the
> > same as the GA fedora rpm repo and updates rpm repos are. we do not make those
> > either. because we do not make all the things for the compsoe, because that is
> > what was requetsed it breaks some of the guarantees we have with the nightly
> > branched/rawhide and milestone composes. That is what was demanded of us. It
> > is what we have delivered as a result.
>
>  Ah ok, I was mistaken about that then.
>
>  I'll need to figure out how to get the exact OSTree commit then and I
>  can just version it with the compose id anyways since it will still
>  correlate all said and done.
>
>  Otherwise, does this sound like an acceptable approach to everyone? 
Bump.

Don't want this thread to die without at least one or two +/-1's so I
know if I should move forward with this or go back to the drawing
board.

Thanks,
-AdamM

>
> -AdamM
>
>>
>> Dennis
>> _______________________________________________
>> rel-eng mailing list
>> rel-eng(a)lists.fedoraproject.org
>> https://lists.fedoraproject.org/admin/lists/[e-mail]
>>

[Adam Williamson]

On Thu, 2016-08-18 at 11:35 -0500, Adam Miller wrote:

>  Currently the compose (pungi) produces an Atomic ostree, installable
> ISO, and
>  various cloud images nightly. Those artifacts are then tested via AutoCloud. 
In fact the ISO is tested by openQA. The other deliverables are tested
by autocloud.