v2: moving around some ostree repos

[Dusty Mabe]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I got some suggestions on the original proposal [1] so here is a v2.

We are moving to a unified ostree repo model where we compose/ship
content for different releases in a single repo.

- From this we'll basically have one "compose" repo where all the different
branches and rawhide write into it and we'll have a unified repo where the 
content will be published to.

When we moved bodhi over to start using pungi we configured it to use a
unified compose repo located at https://kojipkgs.fedoraproject.org/compose/updates/atomic/.
The plan is to take this existing repo and mv it to /compose/atomic/repo and
use it for *all* composes.

The steps are:

1. archive /compose/atomic to /compose/atomic_archive_$date
2. move the /compose/updates/atomic/ unified repo to /compose/atomic/
3. update all configs to compose into /compose/atomic and publish to /atomic/repo/
4. sync ostree content from /atomic/workstation/ into /atomic/repo/

Summary:
- - rawhide now composes in /compose/atomic/
- - branched now composes in /compose/atomic/
- - bodhi now composes in /compose/atomic

- - FAH & FAW rawhide now publishes to /atomic/repo/
- - FAH & FAW branched now publishes to /atomic/repo/
- - FAH & FAW released updates now publishes to /atomic/repo/
- - FAH 27 will continue to publish to /atomic/27/, 28 will move to /atomic/repo/

How do we handle the transition?

[FAH rawhide clients]
- - currently pull from /atomic/rawhide/
- - can redirect *or* tell clients to update configs

[FAW rawhide clients]
- - currently pull from /compose/ostree/rawhide/
- - we need to have them update anyway as they should not be pulling from compose location

[FAH 27 clients]
- - currently pull from https://kojipkgs.fedoraproject.org/atomic/27/
- - continue to pull from /atomic/27, will move to unified repo when they rebase to 28

[FAW 27 clients]
- - currently pull from https://dl.fedoraproject.org/ostree/27/ which is redirect for /atomic/workstation
- - can change existing redirect to point to /atomic/repo

If no one objects patrick and I will perform these changes soon.

Thanks,
Dusty

[1] https://lists.fedoraproject.org/archives/list/[e-mail]...
> -----BEGIN PGP SIGNATURE-----
>
> iQIzBAEBCAAdFiEEPb6zG5c6sV89tRYPMwLb1zlS5nEFAlqYtWEACgkQMwLb1zlS
> 5nFhmg//Q2K/HeDGwFQo44lk31tMOuDr9xvCNiHUsU88xd8BII9TbPbTGnYvgo1o
> 5F3E/T8SQ7AH/ZAQpym9ctmrC3R5DtPlJPRbNI0+EkEDviDMMAbAjwiUgafb+KhX
> hJURXvP/SVjuasRFplhIgMrgD/ycTQPMJ4vdV2vUq9W7PTt6zmKtCNZrgkFu/CcF
> m01B8uzqoJzYy9ojCjaQIZ1MYk8j5tIuuO+IBsNsbeTXgamQ10nRwo0lC9s+/UGp
> pNewEagaAqJ/nmsb41Z2bx/DW5MkNRgr+KNuSZf1MPQDm9RDWzSFpC8VFchXHdDt
> iS6fTQZTZtNx+aWzEqZlhbq8TCKsyED/xJjgn91iXvkmFdELf2PL87UKKBUx30L8
> JAFnU9n/XvAHMF2eCxQ/VvPNivLuapwcSFem+6YE++VP60hDcCf8DKW4ck94zYhp
> 99Xapz+UHOLuf5TZ/2CRFZWuJoeQz3xnRgKQVyd5uWWoZwa91n5KXH4o2G1QBmIa
> w3qVrsnZH1hxxiWZjCrBo0o6pPAjzzbujWzS6Q7RVBp5U0s0XNELg11H2v+0sbY/
> dsV1Gh0WuRR4CoRuozxjtA2R3PLgJN+nut/fbZmIKyyxF/qp45m2qV0JGYPKyK2N
> RcK2ZIvE9VBeteRvibMpGdxf2oBxUAF7oF5vOBnndl4FJOk9OaA=
> =HGgG
> -----END PGP SIGNATURE-----

[Dennis Gilmore]

El jue, 01-03-2018 a las 21:22 -0500, Dusty Mabe escribió:
> I got some suggestions on the original proposal [1] so here is a v2.
>
> We are moving to a unified ostree repo model where we compose/ship
> content for different releases in a single repo.
>
> From this we'll basically have one "compose" repo where all the
> different
> branches and rawhide write into it and we'll have a unified repo
> where the 
> content will be published to.
>
> When we moved bodhi over to start using pungi we configured it to use
> a
> unified compose repo located at https://kojipkgs.fedoraproject.org/co
> mpose/updates/atomic/.
> The plan is to take this existing repo and mv it to
> /compose/atomic/repo and
> use it for *all* composes.
>
> The steps are:
> archive /compose/atomic to /compose/atomic_archive_$date
> move the /compose/updates/atomic/ unified repo to /compose/atomic/
> update all configs to compose into /compose/atomic and publish to
>
> /atomic/repo/
> 4. sync ostree content from /atomic/workstation/ into /atomic/repo/
>
> Summary:
>
> rawhide now composes in /compose/atomic/
>
> branched now composes in /compose/atomic/
>
> bodhi now composes in /compose/atomic
>
> FAH & FAW rawhide now publishes to /atomic/repo/
>
> FAH & FAW branched now publishes to /atomic/repo/
>
> FAH & FAW released updates now publishes to /atomic/repo/
>
> FAH 27 will continue to publish to /atomic/27/, 28 will move to
>
> /atomic/repo/
>
> How do we handle the transition?
>
> [FAH rawhide clients]
> currently pull from /atomic/rawhide/
> can redirect *or* tell clients to update configs
>
> [FAW rawhide clients]
> currently pull from /compose/ostree/rawhide/
> we need to have them update anyway as they should not be pulling
>
> from compose location
>
> [FAH 27 clients]
> currently pull from https://kojipkgs.fedoraproject.org/atomic/27/
> continue to pull from /atomic/27, will move to unified repo when
>
> they rebase to 28
>
> [FAW 27 clients]
> currently pull from https://dl.fedoraproject.org/ostree/27/ which
>
> is redirect for /atomic/workstation
> can change existing redirect to point to /atomic/repo
>
> If no one objects patrick and I will perform these changes soon.

I am good with this,  we can put in redirects for a few months for the
old urls. updated client builds should update the urls.

Dennis

[Dusty Mabe]

On 03/01/2018 09:49 PM, Dennis Gilmore wrote:
> El jue, 01-03-2018 a las 21:22 -0500, Dusty Mabe escribió:
> > I got some suggestions on the original proposal [1] so here is a v2.
> >
> > We are moving to a unified ostree repo model where we compose/ship
> > content for different releases in a single repo.
> >
> > From this we'll basically have one "compose" repo where all the
> > different
> > branches and rawhide write into it and we'll have a unified repo
> > where the 
> > content will be published to.
> >
> > When we moved bodhi over to start using pungi we configured it to use
> > a
> > unified compose repo located at https://kojipkgs.fedoraproject.org/co
> > mpose/updates/atomic/.
> > The plan is to take this existing repo and mv it to
> > /compose/atomic/repo and
> > use it for *all* composes.
> >
> > The steps are:
> > archive /compose/atomic to /compose/atomic_archive_$date
> > move the /compose/updates/atomic/ unified repo to /compose/atomic/
> > update all configs to compose into /compose/atomic and publish to
> >
> > /atomic/repo/
> > 4. sync ostree content from /atomic/workstation/ into /atomic/repo/
> >
> > Summary:
> >
> > rawhide now composes in /compose/atomic/
> >
> > branched now composes in /compose/atomic/
> >
> > bodhi now composes in /compose/atomic
> >
> > FAH & FAW rawhide now publishes to /atomic/repo/
> >
> > FAH & FAW branched now publishes to /atomic/repo/
> >
> > FAH & FAW released updates now publishes to /atomic/repo/
> >
> > FAH 27 will continue to publish to /atomic/27/, 28 will move to
> >
> > /atomic/repo/
> >
> > How do we handle the transition?
> >
> > [FAH rawhide clients]
> > currently pull from /atomic/rawhide/
> > can redirect *or* tell clients to update configs
> >
> > [FAW rawhide clients]
> > currently pull from /compose/ostree/rawhide/
> > we need to have them update anyway as they should not be pulling
> >
> > from compose location
> >
> > [FAH 27 clients]
> > currently pull from https://kojipkgs.fedoraproject.org/atomic/27/
> > continue to pull from /atomic/27, will move to unified repo when
> >
> > they rebase to 28
> >
> > [FAW 27 clients]
> > currently pull from https://dl.fedoraproject.org/ostree/27/ which
> >
> > is redirect for /atomic/workstation
> > can change existing redirect to point to /atomic/repo
> >
> > If no one objects patrick and I will perform these changes soon.
>
> I am good with this,  we can put in redirects for a few months for the
> old urls. updated client builds should update the urls.

yeah, the client builds will definitely be updated to point to the new URLs.
As far as redirects go, I almost prefer the 'rip the band-aid off' approach
if the client *is* going to have to update one day. The reason being that
the change is fresher in everyone's minds and there should be a recent 
relevant change announcement to point to.

Dusty

[Dusty Mabe]

On 03/01/2018 09:22 PM, Dusty Mabe wrote:
> I got some suggestions on the original proposal [1] so here is a v2.
>
> We are moving to a unified ostree repo model where we compose/ship
> content for different releases in a single repo.
>
> From this we'll basically have one "compose" repo where all the different
> branches and rawhide write into it and we'll have a unified repo where the
> content will be published to.
>
> When we moved bodhi over to start using pungi we configured it to use a
> unified compose repo located at https://kojipkgs.fedoraproject.org/compose/updates/atomic/.
> The plan is to take this existing repo and mv it to /compose/atomic/repo and
> use it for *all* composes.
>
> The steps are:
> archive /compose/atomic to /compose/atomic_archive_$date
> move the /compose/updates/atomic/ unified repo to /compose/atomic/
> update all configs to compose into /compose/atomic and publish to /atomic/repo/
> sync ostree content from /atomic/workstation/ into /atomic/repo/
>
> Summary:
>
> rawhide now composes in /compose/atomic/
>
> branched now composes in /compose/atomic/
>
> bodhi now composes in /compose/atomic
>
> FAH & FAW rawhide now publishes to /atomic/repo/
>
> FAH & FAW branched now publishes to /atomic/repo/
>
> FAH & FAW released updates now publishes to /atomic/repo/
>
> FAH 27 will continue to publish to /atomic/27/, 28 will move to /atomic/repo/
>
> How do we handle the transition?
>
> [FAH rawhide clients]
> currently pull from /atomic/rawhide/
> can redirect *or* tell clients to update configs
>
> [FAW rawhide clients]
> currently pull from /compose/ostree/rawhide/
> we need to have them update anyway as they should not be pulling from compose location
>
> [FAH 27 clients]
> currently pull from https://kojipkgs.fedoraproject.org/atomic/27/
> continue to pull from /atomic/27, will move to unified repo when they rebase to 28
>
> [FAW 27 clients]
> currently pull from https://dl.fedoraproject.org/ostree/27/ which is redirect for /atomic/workstation
> can change existing redirect to point to /atomic/repo
>
> If no one objects patrick and I will perform these changes soon.

After talking with patrick there is one update to the proposal:

- For symmetry we'll make the compose repo be /compose/atomic/repo 
  similar to the proposed /atomic/repo/ for the prod published repo.

I'm going to start opening PRs for these changes to our configs today. In each PR I'll ask
that only Patrick merge the PRs as we'll need to coordinate the changes with the actual move
of the repos.

Dusty

[Dusty Mabe]

On 03/03/2018 10:50 AM, Dusty Mabe wrote:
> On 03/01/2018 09:22 PM, Dusty Mabe wrote:
> > I got some suggestions on the original proposal [1] so here is a v2.
> >
> > We are moving to a unified ostree repo model where we compose/ship
> > content for different releases in a single repo.
> >
> > From this we'll basically have one "compose" repo where all the different
> > branches and rawhide write into it and we'll have a unified repo where the
> > content will be published to.
> >
> > When we moved bodhi over to start using pungi we configured it to use a
> > unified compose repo located at https://kojipkgs.fedoraproject.org/compose/updates/atomic/.
> > The plan is to take this existing repo and mv it to /compose/atomic/repo and
> > use it for *all* composes.
> >
> > The steps are:
> > archive /compose/atomic to /compose/atomic_archive_$date
> > move the /compose/updates/atomic/ unified repo to /compose/atomic/
> > update all configs to compose into /compose/atomic and publish to /atomic/repo/
> > sync ostree content from /atomic/workstation/ into /atomic/repo/
> >
> > Summary:
> >
> > rawhide now composes in /compose/atomic/
> >
> > branched now composes in /compose/atomic/
> >
> > bodhi now composes in /compose/atomic
> >
> > FAH & FAW rawhide now publishes to /atomic/repo/
> >
> > FAH & FAW branched now publishes to /atomic/repo/
> >
> > FAH & FAW released updates now publishes to /atomic/repo/
> >
> > FAH 27 will continue to publish to /atomic/27/, 28 will move to /atomic/repo/
> >
> > How do we handle the transition?
> >
> > [FAH rawhide clients]
> > currently pull from /atomic/rawhide/
> > can redirect *or* tell clients to update configs
> >
> > [FAW rawhide clients]
> > currently pull from /compose/ostree/rawhide/
> > we need to have them update anyway as they should not be pulling from compose location
> >
> > [FAH 27 clients]
> > currently pull from https://kojipkgs.fedoraproject.org/atomic/27/
> > continue to pull from /atomic/27, will move to unified repo when they rebase to 28
> >
> > [FAW 27 clients]
> > currently pull from https://dl.fedoraproject.org/ostree/27/ which is redirect for /atomic/workstation
> > can change existing redirect to point to /atomic/repo
> >
> > If no one objects patrick and I will perform these changes soon.
>
> After talking with patrick there is one update to the proposal:
> For symmetry we'll make the compose repo be /compose/atomic/repo 
> similar to the proposed /atomic/repo/ for the prod published repo.
>
> I'm going to start opening PRs for these changes to our configs today. In each PR I'll ask
> that only Patrick merge the PRs as we'll need to coordinate the changes with the actual move
> of the repos.

This work is now complete. Please notify me/patrick of any breakages that are found.

Dusty

[Colin Walters]

Thanks a ton for working on this!  It's awesome to have happen.
It'll really help with the "rebase" major version updates story.

[Dusty Mabe]

On 03/05/2018 04:22 PM, Colin Walters wrote:
> Thanks a ton for working on this!  It's awesome to have happen.
> It'll really help with the "rebase" major version updates story.

Thanks! :)
I appreciate it a lot.