hierarchical comps groups proposal

[Jens-Ulrik Petersen]

Hi,

I would like to suggest the idea of adding support for
hierarchical comps groups to Fedora.

The idea is make yum groups in comps more modular,
ie groups could require other groups not just packages;
at this time I don't think it would require any GUI changes.
Currently comps is quite linear (flat) and we have
to do silly things like adding a group to add a package to
an environment group,  and duplicating certain sets of packages
across certain groups. Hierarchical groups would allow more flexible
package group choices, avoid package repetition in comps, and
also allow people to install what they need more precisely.

eg Someone might only want to install @gnome-core-desktop
@mate-core-desktop, @asian-fonts, etc.  I am sure there are
many more possible examples.

With the Fedora.next initiative now seems a good time to do this.
It should probably be made a system-wide change proposal
(I only noticed the deadline this week!)
but I wanted to post here first for discussion since
I think it needs changes to yum, dnf, and maybe other tools.

I would be happy to help (at least) prepare a Change proposal for
this, specially if other people (stakeholders) are willing
to help make this happen.

Jens

[Ales Kozumplik]

On 03/05/2014 09:40 AM, Jens Petersen wrote:
>  Hi,
>
>  I would like to suggest the idea of adding support for
>  hierarchical comps groups to Fedora.
>
>  The idea is make yum groups in comps more modular,
>  ie groups could require other groups not just packages;
>  at this time I don't think it would require any GUI changes.
>  Currently comps is quite linear (flat) and we have
>  to do silly things like adding a group to add a package to
>  an environment group,  and duplicating certain sets of packages
>  across certain groups. Hierarchical groups would allow more flexible
>  package group choices, avoid package repetition in comps, and
>  also allow people to install what they need more precisely.
>
>  eg Someone might only want to install @gnome-core-desktop
>  @mate-core-desktop, @asian-fonts, etc.  I am sure there are
>  many more possible examples.
>  
Yep, I'm for it. One thing: the proposal should explicitly mention how 
hierarchical groups make environment groups obsolete, and then make them 
obsolete, so both the implementators and the users have one less 
obscurely defined concept to deal with.

>  With the Fedora.next initiative now seems a good time to do this.
>  It should probably be made a system-wide change proposal
>  (I only noticed the deadline this week!)
>  but I wanted to post here first for discussion since
>  I think it needs changes to yum, dnf, and maybe other tools.
>
>  I would be happy to help (at least) prepare a Change proposal for
>  this, specially if other people (stakeholders) are willing
>  to help make this happen. 
We can contribute the changes on DNF side once libcomps supports that.

Ales

[Miloslav Trmač]

2014-03-05 9:40 GMT+01:00 Jens Petersen <petersen(a)redhat.com&gt;:

>  I would like to suggest the idea of adding support for
>  hierarchical comps groups to Fedora.
>  
(I'm not going to contribute actual work on this anyway, but) do we
actually need that complexity?


>  The idea is make yum groups in comps more modular,
>  ie groups could require other groups not just packages;
>  at this time I don't think it would require any GUI changes.
>  
So the proposal is not to show them as really hierarchical, and not to add
any structurally new user-visible features[1] just to avoid repetition in
the comps file?

Would it be an option to use an XML entity for the duplicated parts, or a
similar preprocessing step during generation of comps.xml, to create the
current group format instead?  That would have the advantage of not having
to adjust yum/dnf/anaconda to support the new feature.
    Mirek

[1] Having more groups available is certainly a feature, but not
"structurally new", i.e. it keeps the existing usage model.

[Jens-Ulrik Petersen]

Hi,

>  2014-03-05 9:40 GMT+01:00 Jens Petersen < petersen(a)redhat.com >
> :
>  > I would like to suggest the idea of adding support for
>  > hierarchical comps groups to Fedora. 
>  (I'm not going to contribute actual work on this anyway, but) do
> we actually
>  need that complexity? 
I am not sure how complex it is. As Ales pointed out
it might allow us to remove environment groups for example
so it might actually simplify comps, as well as eliminating some repetitions.
Note that more modular comps would also be helpful for people
writing kickstart files for example.

>  > The idea is make yum groups in comps more modular,
>  > ie groups could require other groups not just packages;
>  > at this time I don't think it would require any GUI changes. 
>  So the proposal is not to show them as really hierarchical, and not
> to add
>  any structurally new user-visible features[1] just to avoid repetition in
>  the comps file? 
To me current comps is quite messy because everything has to be "flat"
which limits its flexibility in various ways.
But right, *initially* I reckon hierarchical groups need not *require*
any GUI changes to anaconda, etc.  In the first instance it would be
about providing greater structure and more fine-grained groups in comps.
Later these could also be used for more advanced selection of groups
in the installer and package GUI.

>  Would it be an option to use an XML entity for the duplicated parts,
> or a
>  similar preprocessing step during generation of comps.xml, to create the
>  current group format instead? That would have the advantage of not having to
>  adjust yum/dnf/anaconda to support the new feature. 
Perhaps, but the primary goal/motivation is not just to eliminate
the duplication of common package sets but to provide a richer, more modular,
flexible hierarchy of comps groups. Perhaps an explicit example would help.

Jens

[Bruno Wolff III]

> > So the proposal is not to show them as really hierarchical, and not to add
> > any structurally new user-visible features[1] just to avoid repetition in
> > the comps file?
>
> To me current comps is quite messy because everything has to be "flat"
> which limits its flexibility in various ways.
> But right, *initially* I reckon hierarchical groups need not *require*
> any GUI changes to anaconda, etc.  In the first instance it would be
> about providing greater structure and more fine-grained groups in comps.
> Later these could also be used for more advanced selection of groups
> in the installer and package GUI. 
I don't think you want to use the word hierarchical to describe this. It's 
really more like an include macro.

[Bill Nottingham]

Jens Petersen (petersen(a)redhat.com) said: 
>  > (I'm not going to contribute actual work on this anyway,
> but) do we actually
>  > need that complexity?
>  
>  I am not sure how complex it is. As Ales pointed out
>  it might allow us to remove environment groups for example
>  so it might actually simplify comps, as well as eliminating some repetitions.
>  Note that more modular comps would also be helpful for people
>  writing kickstart files for example.
>  
>  > > The idea is make yum groups in comps more modular,
>  > > ie groups could require other groups not just packages;
>  > > at this time I don't think it would require any GUI changes.
>  
>  > So the proposal is not to show them as really hierarchical, and not to add
>  > any structurally new user-visible features[1] just to avoid repetition in
>  > the comps file?
>  
>  To me current comps is quite messy because everything has to be "flat"
>  which limits its flexibility in various ways.
>  But right, *initially* I reckon hierarchical groups need not *require*
>  any GUI changes to anaconda, etc.  In the first instance it would be
>  about providing greater structure and more fine-grained groups in comps.
>  Later these could also be used for more advanced selection of groups
>  in the installer and package GUI. 
My concern would be how deeply woven the current format is into our tools -
it's yum, PK, DNF, anaconda, and all the tools built on top of it. It's
exposed in kickstart. It's expected to be handled by older versions on
upgrade, or even by mock when constructing arbitrary build roots.

So care would need to be taken to figure out a good way to land this all at
once, in a way that's backwards compatible enough to not break other things.

Bill

[Jens-Ulrik Petersen]

>  My concern would be how deeply woven the current format is into our
> tools -
>  it's yum, PK, DNF, anaconda, and all the tools built on top of it. It's
>  exposed in kickstart. It's expected to be handled by older versions on
>  upgrade, or even by mock when constructing arbitrary build roots.
>  
>  So care would need to be taken to figure out a good way to land this all at
>  once, in a way that's backwards compatible enough to not break other things. 
True, so maybe Miloslav's suggestion of using XML entities to implement this
might indeed the safest and most pragmatic at this time.
Presumably that would only need changes to the generation of comps.xml
files themselves.

Jens

ps (In the long term I would not mind seeing more drastic changes though,
like maybe even changing to YAML format or something similar.)

[Matthew Miller]

On Wed, Mar 05, 2014 at 03:40:34AM -0500, Jens Petersen wrote:
>  I would like to suggest the idea of adding support for
>  hierarchical comps groups to Fedora. 
Worth noting that it _used_ to have that functionality and it was dropped.

>  With the Fedora.next initiative now seems a good time to do this.
>  It should probably be made a system-wide change proposal
>  (I only noticed the deadline this week!) 
>  I would be happy to help (at least) prepare a Change proposal for
>  this, specially if other people (stakeholders) are willing
>  to help make this happen. 
Primary stakeholders would be:

 - yum/dnf
 - anaconda (possibly insulated from this by yum? Not sure.)
 - product/spin maintainers
 - comps file maintainers
 - installation docs writers
 - users with existing kickstart files

Others?
 
-- 
Matthew Miller    --   Fedora Project    --    <mattdm(a)fedoraproject.org&gt;