#6366: missing metadata about nightly rawhide/24 composes

[Fedora Release Engineering]

#6366: missing metadata about nightly rawhide/24 composes
-----------------------------+------------------------
 Reporter:  bpeck            |       Owner:  rel-eng@…
     Type:  defect           |      Status:  new
Milestone:  Fedora 24 Alpha  |   Component:  other
 Keywords:                   |  Blocked By:
 Blocking:                   |
-----------------------------+------------------------
 Currently there is no metadata about a compose in the rsync share.  This
 makes it very difficult to know what is available to rsync/import.

 rsync rsync://dl.fedoraproject.org/fedora-linux-development/rawhide/

 ----------------------------
 Fedora Master Mirror Servers
 ----------------------------
 Modules for Fedora Core and Extras have been removed, as this content is
 no longer updated. See the instructions below for how to mirror current
 content.

 See http://fedoraproject.org/wiki/Infrastructure/Mirroring for
 instructions.
 ----------------------------

 drwxrwxr-x          4,096 2016/03/09 13:26:55 .
 drwxrwxr-x          4,096 2016/03/08 11:08:04 Atomic
 drwxrwxr-x          4,096 2016/03/08 07:33:16 CloudImages
 drwxrwxr-x          4,096 2016/03/08 07:33:17 Docker
 drwxrwxr-x          4,096 2016/03/08 07:33:20 Everything
 drwxrwxr-x          4,096 2016/03/08 07:36:56 Server
 drwxrwxr-x          4,096 2016/03/08 07:37:03 Spins
 drwxrwxr-x          4,096 2016/03/08 07:37:04 Workstation


 In the past we used to have a .composeinfo file which not only told us
 which trees were available in a compose but whether the compose completed
 successfully.

 Both of these attributes are important to keep testing.  We don't want to
 overwrite a working release with a broken one.

 It appears that the .composeinfo file format is going away and being
 replaced with metadata/composeinfo.json which is fine.  We can update our
 code to work with this but according to dgilmore its not as easy as simply
 copying this file over since the compose gets carved up.

 It sounds like the process that does the carving should produce a new
 composeinfo.json which describes the carved up version.

 Until this ticket is resolved we will not be able to mirror Fedora inside
 Red Hat for testing.

-- 
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6366>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project

[Fedora Release Engineering]

#6366: missing metadata about nightly rawhide/24 composes
------------------------------+-----------------------
  Reporter:  bpeck            |      Owner:  rel-eng@…
      Type:  defect           |     Status:  new
 Milestone:  Fedora 24 Alpha  |  Component:  other
Resolution:                   |   Keywords:
Blocked By:                   |   Blocking:
------------------------------+-----------------------

Comment (by ausil):

 I am honestly not sure the best way to resolve this.  A big part of the
 problem is taht we split the compose up after it is done.

 Lets look at a nigtly f24 compose, The compose is done in

 https://kojipkgs.fedoraproject.org/compose/branched/Fedora-24-20160318.n....

 We have a bunch of directories that have the various parts of the compose
 {{{
 Atomic  Cloud  CloudImages  Docker  Everything  Labs  metadata  Server
 Spins  Workstation
 }}}
 the metadata directory has a composeinfo.json file in it that describes
 nicely all the content of the compose

 however, when we put the compose on the mirrors, we go and carve it up.
 {{{
 Atomic  CloudImages  Docker  Everything  Server  Spins  Workstation
 }}}
 all go into the main public location
  http://dl.fedoraproject.org/pub/fedora/linux/development/24/

 we then put the remianing deliverables over in alt
 {{{
 Cloud  Labs
 }}}
  http://dl.fedoraproject.org/pub/alt/development/24/

 throwing metadata on the floor as it now is incorrect.

 there is .treeinfo files all through the compose but you need to tell
 beaker where to look for them.

 we have a ticket https://pagure.io/pungi/issue/98 open to enable us to
 change things up to better support our use cases.

 AFAIK the only use case for .composeinfo is beaker. dmach would be the
 person that made the change from .composeinfo to composeinfo.json

 another way to solve this is pungi gives a message when it is done
 composing, it is follow up with a message later that says the rsync is
 done. If you use the message from pungi you could fetch the
 composeinfo.json and prep things for when the rsync is done. knowing how
 we carve things up.  It is not as ideal as it could be as we would need to
 let you know if we change how we carve things up, but I do not anticipate
 changing that before we get pungi being able to better support our use
 cases.


 Secondary arches are a whole different problem. Since we are merging three
 composes into one.

-- 
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6366#comment:1>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project