Java naming scheme

[Fernando Nasser]

Hi,

I will try to summarize the findings so far.

1) We seem to agree that we want to make the following transformation on 
the release fields :

3jpp -> 3.1 in Fedora

This will satisfy all our upgrade paths, as:

4.1 > 4jpp > 3.1 > 3jpp

so one can always have the latest fixes.


2) Some would be agreeable to leaving the 'jpp' in there as a temporary 
measure, as what we really want is some Groups (or Categories) 
functionality that is not yet available.

So, temporarely, 3jpp -> 3jpp.1


3)  What do we need to get rid of the 'jpp'

a) Query for the set of Java packages

The rpm -qg (or rpm -q --group) command currently does not accept patterns

If we could do 'rpm -qg "Java*"' we could add "Java/" to all
"Group:" 
tags of all Java packages and that would work.

I am trying to create a patch for that (which would be RPM patch #37 in 
Fedora) but I am encountering some difficulties as it is the first time 
I am looking at the rpm source code and I have been working with Java 
rather than C in the last years.

P.S.: We would need a similar query functionality when Categories 
replace Groups.

b) yum has already some group functionality (groupinstall, groupupdate, 
groupremove, groupinfo) that is based on an XML file that is kept in the 
repository.  But the option --exclude only acts on file names, we need a 
--groupexclude.

We already have a "Java" group, with only 2 packages on it:

# yum groupinfo Java
Loading "installonlyn" plugin
Setting up Group Process
Setting up repositories

Group: Java
 Description: Support for running programs written in the Java 
programming language.
 Mandatory Packages:
   libgcj
   java-1.4.2-gcj-compat

We could just make sure all Java packages are in the Java group to make 
use of the yum group functionality.
We just need the --groupexclude.

I will try and create a patch to add a '--groupexclude' as soon as I 
finish with the rpm one.


P.S.: If we could get the fixes for 3a and 3b we could get rid of the 
'jpp' immediately.  Anyone that could help me with those?


Fernando

[seth vidal]

On Wed, 2007-01-17 at 16:23 -0500, Fernando Nasser wrote:
>  Hi,
>  b) yum has already some group functionality (groupinstall, groupupdate, 
>  groupremove, groupinfo) that is based on an XML file that is kept in the 
>  repository.  But the option --exclude only acts on file names, we need a 
>  --groupexclude.
>  
>  We already have a "Java" group, with only 2 packages on it:
>  
>  # yum groupinfo Java
>  Loading "installonlyn" plugin
>  Setting up Group Process
>  Setting up repositories
>  
>  Group: Java
>   Description: Support for running programs written in the Java 
>  programming language.
>   Mandatory Packages:
>     libgcj
>     java-1.4.2-gcj-compat
>  
>  We could just make sure all Java packages are in the Java group to make 
>  use of the yum group functionality.
>  We just need the --groupexclude.
>  
>  I will try and create a patch to add a '--groupexclude' as soon as I 
>  finish with the rpm one. 
why do you need a 'groupexclude' option? I guess I don't understand the
use case for it, yet.

-sv

[Fernando Nasser]

seth vidal wrote:
>  
>  why do you need a 'groupexclude' option? I guess I don't understand the
>  use case for it, yet.
>   
For instance, to upgrade one's Fedora system without interfering with 
the Java stack there.

[David Lutterkort]

On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
>  seth vidal wrote:
>  > 
>  > why do you need a 'groupexclude' option? I guess I don't understand the
>  > use case for it, yet.
>  > 
>  
>  For instance, to upgrade one's Fedora system without interfering with 
>  the Java stack there. 
What happens to your Java stack at that point ? Will it just stay
frozen ?

David

[Fernando Nasser]

David Lutterkort wrote:
>  On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
> > seth vidal wrote:
> >> why do you need a 'groupexclude' option? I guess I don't understand
> the
> >> use case for it, yet.
> >>
> > For instance, to upgrade one's Fedora system without interfering with 
> > the Java stack there.
>  
>  What happens to your Java stack at that point ? Will it just stay
>  frozen ?
>   
That is the idea, update your Fedora system except for the Java 
packages, which you can update selectively, from a different repo, from 
local RPMs etc.

[seth vidal]

On Thu, 2007-01-18 at 17:59 -0500, Fernando Nasser wrote:
>  David Lutterkort wrote:
>  > On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
>  >> seth vidal wrote:
>  >>> why do you need a 'groupexclude' option? I guess I don't
> understand the
>  >>> use case for it, yet.
>  >>>
>  >> For instance, to upgrade one's Fedora system without interfering with 
>  >> the Java stack there.
>  > 
>  > What happens to your Java stack at that point ? Will it just stay
>  > frozen ?
>  > 
>  
>  That is the idea, update your Fedora system except for the Java 
>  packages, which you can update selectively, from a different repo, from 
>  local RPMs etc.
>   
so you want --groupexclude to exclude from the update all packages in
that group? Not exclude the group from the list of available groups?

-sv

[Tom Callaway]

On Thu, 2007-01-18 at 20:05 -0500, seth vidal wrote:
>  On Thu, 2007-01-18 at 17:59 -0500, Fernando Nasser wrote:
>  > David Lutterkort wrote:
>  > > On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
>  > >> seth vidal wrote:
>  > >>> why do you need a 'groupexclude' option? I guess I don't
> understand the
>  > >>> use case for it, yet.
>  > >>>
>  > >> For instance, to upgrade one's Fedora system without interfering with 
>  > >> the Java stack there.
>  > > 
>  > > What happens to your Java stack at that point ? Will it just stay
>  > > frozen ?
>  > > 
>  > 
>  > That is the idea, update your Fedora system except for the Java 
>  > packages, which you can update selectively, from a different repo, from 
>  > local RPMs etc.
>  > 
>  
>  so you want --groupexclude to exclude from the update all packages in
>  that group? Not exclude the group from the list of available groups? 
Yep. I'm pretty sure that's what he's looking to do.

~spot

[Fernando Nasser]

Tom 'spot' Callaway wrote:
>  On Thu, 2007-01-18 at 20:05 -0500, seth vidal wrote:
> > On Thu, 2007-01-18 at 17:59 -0500, Fernando Nasser wrote:
> >> David Lutterkort wrote:
> >>> On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
> >>>> seth vidal wrote:
> >>>>> why do you need a 'groupexclude' option? I guess I don't
> understand the
> >>>>> use case for it, yet.
> >>>>>
> >>>> For instance, to upgrade one's Fedora system without interfering with
>
> >>>> the Java stack there.
> >>> What happens to your Java stack at that point ? Will it just stay
> >>> frozen ?
> >>>
> >> That is the idea, update your Fedora system except for the Java 
> >> packages, which you can update selectively, from a different repo, from 
> >> local RPMs etc.
> >>
> > so you want --groupexclude to exclude from the update all packages in
> > that group? Not exclude the group from the list of available groups?
>  
>  Yep. I'm pretty sure that's what he's looking to do.
>   
Correct.

[David Lutterkort]

On Thu, 2007-01-18 at 20:05 -0500, seth vidal wrote:
>  On Thu, 2007-01-18 at 17:59 -0500, Fernando Nasser wrote:
>  > David Lutterkort wrote:
>  > > On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
>  > >> seth vidal wrote:
>  > >>> why do you need a 'groupexclude' option? I guess I don't
> understand the
>  > >>> use case for it, yet.
>  > >>>
>  > >> For instance, to upgrade one's Fedora system without interfering with 
>  > >> the Java stack there.
>  > > 
>  > > What happens to your Java stack at that point ? Will it just stay
>  > > frozen ?
>  > > 
>  > 
>  > That is the idea, update your Fedora system except for the Java 
>  > packages, which you can update selectively, from a different repo, from 
>  > local RPMs etc.
>  > 
>  
>  so you want --groupexclude to exclude from the update all packages in
>  that group? Not exclude the group from the list of available groups? 
This is starting to be OT for this list, but wouldn't it be better if
users could just add groups into the existing mechanisms with a separate
notation, similar to kickstart ? I.e., instead of a separate
--groupexclude, allow --exclude=@java ?

David

[seth vidal]

On Mon, 2007-01-22 at 20:02 +0000, David Lutterkort wrote:
>  On Thu, 2007-01-18 at 20:05 -0500, seth vidal wrote:
>  > On Thu, 2007-01-18 at 17:59 -0500, Fernando Nasser wrote:
>  > > David Lutterkort wrote:
>  > > > On Thu, 2007-01-18 at 09:11 -0500, Fernando Nasser wrote:
>  > > >> seth vidal wrote:
>  > > >>> why do you need a 'groupexclude' option? I guess I
> don't understand the
>  > > >>> use case for it, yet.
>  > > >>>
>  > > >> For instance, to upgrade one's Fedora system without interfering
> with 
>  > > >> the Java stack there.
>  > > > 
>  > > > What happens to your Java stack at that point ? Will it just stay
>  > > > frozen ?
>  > > > 
>  > > 
>  > > That is the idea, update your Fedora system except for the Java 
>  > > packages, which you can update selectively, from a different repo, from 
>  > > local RPMs etc.
>  > > 
>  > 
>  > so you want --groupexclude to exclude from the update all packages in
>  > that group? Not exclude the group from the list of available groups?
>  
>  This is starting to be OT for this list, but wouldn't it be better if
>  users could just add groups into the existing mechanisms with a separate
>  notation, similar to kickstart ? I.e., instead of a separate
>  --groupexclude, allow --exclude=@java ?
>   
I don't terribly like special character delimiters like that.

-sv

[Toshio Kuratomi]

On Wed, 2007-01-17 at 16:23 -0500, Fernando Nasser wrote:
>  1) We seem to agree that we want to make the following transformation
> on 
>  the release fields :
>  
>  3jpp -> 3.1 in Fedora
>  
>  This will satisfy all our upgrade paths, as:
>  
>  4.1 > 4jpp > 3.1 > 3jpp
>  
>  so one can always have the latest fixes.
>   
Spot brought the 3jpp.fc6.1 format to the packaging group which was
discussed and agreed as a possible temporary format.  Unfortunately,
3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
removed.  So we'll probably have to have another (hopefully short)
discussion about using 3jpp.1.fc6 & 3.1.fc6.

>  
>  2) Some would be agreeable to leaving the 'jpp' in there as a temporary 
>  measure, as what we really want is some Groups (or Categories) 
>  functionality that is not yet available.
>  
>  So, temporarely, 3jpp -> 3jpp.1
>   
Yep.

>  
>  3)  What do we need to get rid of the 'jpp'
>  
>  a) Query for the set of Java packages
>  
>  The rpm -qg (or rpm -q --group) command currently does not accept patterns
>  
>  If we could do 'rpm -qg "Java*"' we could add "Java/" to all
> "Group:" 
>  tags of all Java packages and that would work.
>  
>  I am trying to create a patch for that (which would be RPM patch #37 in 
>  Fedora) but I am encountering some difficulties as it is the first time 
>  I am looking at the rpm source code and I have been working with Java 
>  rather than C in the last years.
>  
>  P.S.: We would need a similar query functionality when Categories 
>  replace Groups.
>   
Although I can't speak for the likelihood of this getting into rpm, it's
usefulness is going to be near nil.  The distribution is moving away
from storing group information in the rpm spec file.  These reasons have
been articulated by Jeremy Katz and others on fedora-devel many times in
the past.

There are some pointers to the last discussion here:
http://www.fedoraproject.org/wiki/Extras/SIGs/Comps/PackageGroupEnhancements

The page has the justification for moving away from the group tag at the
top of the page although I think the discussion for that happened in
previous discussions (so the threads that are linked will assume that
everyone already knows the shortcomings of the Group tag.) 

Given this, I'd be very hesitant to make group functionality in rpm be
one of the criteria to get rid of the jpp naming.

>  b) yum has already some group functionality (groupinstall,
> groupupdate, 
>  groupremove, groupinfo) that is based on an XML file that is kept in the 
>  repository.  But the option --exclude only acts on file names, we need a 
>  --groupexclude.
>  
>  We already have a "Java" group, with only 2 packages on it:
>  
>  # yum groupinfo Java
>  Loading "installonlyn" plugin
>  Setting up Group Process
>  Setting up repositories
>  
>  Group: Java
>   Description: Support for running programs written in the Java 
>  programming language.
>   Mandatory Packages:
>     libgcj
>     java-1.4.2-gcj-compat
>   
If I understood spot's summary of your requirements, you want to
categorize all the packages you are importing from jpackage.  I would
point out that the java group would be a poor place to implement that as
any java packages which did not come through jpackage could also
legitimately be placed in there.

>  We could just make sure all Java packages are in the Java group to
> make 
>  use of the yum group functionality.
>  We just need the --groupexclude.
>  
>  I will try and create a patch to add a '--groupexclude' as soon as I 
>  finish with the rpm one.
>   
One controversy with using the yum group functionality for this is that
it uses comps.xml to achieve its ends.  Jeremy Katz is against using
comps.xml to make categories that shouldn't be visible to the installer.
skvidal has shown how we could have multiple comps.xml files that yum
reads (whereas the installer only reads from one) which addresses a few
of Jeremy's concerns.  When we go ahead with this we'll have to work out
whether we're going to store the information in a comps file or teach
yum about a new grouping file format.

-Toshio

[Fernando Nasser]

Hi Toshio,

Toshio Kuratomi wrote:
>  On Wed, 2007-01-17 at 16:23 -0500, Fernando Nasser wrote:
> > 1) We seem to agree that we want to make the following transformation on 
> > the release fields :
> >
> > 3jpp -> 3.1 in Fedora
> >
> > This will satisfy all our upgrade paths, as:
> >
> > 4.1 > 4jpp > 3.1 > 3jpp
> >
> > so one can always have the latest fixes.
> >
>  Spot brought the 3jpp.fc6.1 format to the packaging group which was
>  discussed and agreed as a possible temporary format.  Unfortunately,
>  3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
>  removed.  So we'll probably have to have another (hopefully short)
>  discussion about using 3jpp.1.fc6 & 3.1.fc6.
>   
Why the %{_dist} is necessary here?  Can't we just add the number?
I thought we would need the %{_dist} only if we needed to have the same 
RPM built in two different distro releases and they were release 
specific (depend on some shared library or something).


> > 2) Some would be agreeable to leaving the 'jpp' in there
> as a temporary 
> > measure, as what we really want is some Groups (or Categories) 
> > functionality that is not yet available.
> >
> > So, temporarely, 3jpp -> 3jpp.1
> >
>  Yep.
>  
> > 3)  What do we need to get rid of the 'jpp'
> >
> > a) Query for the set of Java packages
> >
> > The rpm -qg (or rpm -q --group) command currently does not accept patterns
> >
> > If we could do 'rpm -qg "Java*"' we could add "Java/" to
> all "Group:" 
> > tags of all Java packages and that would work.
> >
> > I am trying to create a patch for that (which would be RPM patch #37 in 
> > Fedora) but I am encountering some difficulties as it is the first time 
> > I am looking at the rpm source code and I have been working with Java 
> > rather than C in the last years.
> >
> > P.S.: We would need a similar query functionality when Categories 
> > replace Groups.
> >
>  Although I can't speak for the likelihood of this getting into rpm, it's
>  usefulness is going to be near nil.  The distribution is moving away
>  from storing group information in the rpm spec file.  These reasons have
>  been articulated by Jeremy Katz and others on fedora-devel many times in
>  the past.
>  
>  There are some pointers to the last discussion here:
>  http://www.fedoraproject.org/wiki/Extras/SIGs/Comps/PackageGroupEnhancements
>  
>  The page has the justification for moving away from the group tag at the
>  top of the page although I think the discussion for that happened in
>  previous discussions (so the threads that are linked will assume that
>  everyone already knows the shortcomings of the Group tag.) 
>  
>  Given this, I'd be very hesitant to make group functionality in rpm be
>  one of the criteria to get rid of the jpp naming.
>   
You missed the "P.S."

I am creating a patch for Groups because Categories are not there yet, 
so I can't do anything for it ATM.  Once Groups are abandoned in favor 
of Categories, the functionality available for Groups must be adapted to 
work with Categories instead.

In any case, I am not trying to add any new switch or functionality, 
just fix an existing one that has an annoying shortcoming ('rpm -q 
--group' does not take a regexp/glob).


> > b) yum has already some group functionality (groupinstall,
> groupupdate, 
> > groupremove, groupinfo) that is based on an XML file that is kept in the 
> > repository.  But the option --exclude only acts on file names, we need a 
> > --groupexclude.
> >
> > We already have a "Java" group, with only 2 packages on it:
> >
> > # yum groupinfo Java
> > Loading "installonlyn" plugin
> > Setting up Group Process
> > Setting up repositories
> >
> > Group: Java
> >  Description: Support for running programs written in the Java 
> > programming language.
> >  Mandatory Packages:
> >    libgcj
> >    java-1.4.2-gcj-compat
> >
>  If I understood spot's summary of your requirements, you want to
>  categorize all the packages you are importing from jpackage.  I would
>  point out that the java group would be a poor place to implement that as
>  any java packages which did not come through jpackage could also
>  legitimately be placed in there.
>   
We actually want all Java packages.  They are always extremely 
interrelated, with cascading effects among them, so they need to be 
treated as a whole.  Any Java package should be in there.

When categories become available, we can even be fancies and have the 
Java ones that are from Jpackage also in a specific category (I don't 
have a use case for that at the moment, but if necessary we could do it).


> > We could just make sure all Java packages are in the Java group
> to make 
> > use of the yum group functionality.
> > We just need the --groupexclude.
> >
> > I will try and create a patch to add a '--groupexclude' as soon as I 
> > finish with the rpm one.
> >
>  One controversy with using the yum group functionality for this is that
>  it uses comps.xml to achieve its ends.  Jeremy Katz is against using
>  comps.xml to make categories that shouldn't be visible to the installer.
>  skvidal has shown how we could have multiple comps.xml files that yum
>  reads (whereas the installer only reads from one) which addresses a few
>  of Jeremy's concerns.  When we go ahead with this we'll have to work out
>  whether we're going to store the information in a comps file or teach
>  yum about a new grouping file format.
>   
In any case, there will be always something for groupinstall, 
groupupdate, groupremove, groupinfo and also a --groupexclude to do the 
selection.


Regards,
Fernando

[Rex Dieter]

Fernando Nasser wrote:
>  Hi Toshio, 
> > Spot brought the 3jpp.fc6.1 format to the packaging group which
> was
> > discussed and agreed as a possible temporary format.  Unfortunately,
> > 3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
> > removed.  So we'll probably have to have another (hopefully short)
> > discussion about using 3jpp.1.fc6 & 3.1.fc6.
> >
>  
>  Why the %{_dist} is necessary here?  Can't we just add the number? 
Use of %{?dist} is/was/always-has-been optional. (:

-- Rex

[Jesse Keating]

On Thursday 18 January 2007 09:26, Fernando Nasser wrote:
>  > Spot brought the 3jpp.fc6.1 format to the packaging group which
> was
>  > discussed and agreed as a possible temporary format.  Unfortunately,
>  > 3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
>  > removed.  So we'll probably have to have another (hopefully short)
>  > discussion about using 3jpp.1.fc6 & 3.1.fc6.
>
>  Why the %{_dist} is necessary here?  Can't we just add the number?
>  I thought we would need the %{_dist} only if we needed to have the same
>  RPM built in two different distro releases and they were release
>  specific (depend on some shared library or something). 
Unfortunately it may be necessary for you folks, if we ever have to bump the 
package in an older release, without becoming NVR higher than the same 
package in a newer release.  If 3jpp is used in both FC6 and F7, and we have 
to rebuild in FC6 for whatever localized reason, we have to be able to do it 
in a way that doesn't promote it to be NVR higher than the 3jpp in F7.

-- 
Jesse Keating
Release Engineer: Fedora

[Tom Callaway]

On Thu, 2007-01-18 at 10:12 -0500, Jesse Keating wrote:
>  On Thursday 18 January 2007 09:26, Fernando Nasser wrote:
>  > > Spot brought the 3jpp.fc6.1 format to the packaging group which was
>  > > discussed and agreed as a possible temporary format.  Unfortunately,
>  > > 3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
>  > > removed.  So we'll probably have to have another (hopefully short)
>  > > discussion about using 3jpp.1.fc6 & 3.1.fc6.
>  >
>  > Why the %{_dist} is necessary here?  Can't we just add the number?
>  > I thought we would need the %{_dist} only if we needed to have the same
>  > RPM built in two different distro releases and they were release
>  > specific (depend on some shared library or something).
>  
>  Unfortunately it may be necessary for you folks, if we ever have to bump the 
>  package in an older release, without becoming NVR higher than the same 
>  package in a newer release.  If 3jpp is used in both FC6 and F7, and we have 
>  to rebuild in FC6 for whatever localized reason, we have to be able to do it 
>  in a way that doesn't promote it to be NVR higher than the 3jpp in F7. 
Well, it turns out I don't think this will be necessary:

I started writing this up late on Tuesday:

http://fedoraproject.org/wiki/PackagingDrafts/ExceptionJPackage

~spot

[Jesse Keating]

On Thursday 18 January 2007 10:18, Tom 'spot' Callaway wrote:
>  Well, it turns out I don't think this will be necessary:
>
>  I started writing this up late on Tuesday:
>
>  http://fedoraproject.org/wiki/PackagingDrafts/ExceptionJPackage 
I don't see how this handles multiple Fedora releases.

This works great if you only consider jpackage as one release, and Current 
Fedora as one release.  However we DO have to consider:

jpackage--
          |
           - Fedora Current
          |
           - Fedora (Current - 1)
          |
           - Fedora (Current - 2)

Obviously one can't use the same NVR on all three Fedora lines, so something 
has to be used to differentiate them, and allow for rebuilding something on 
Current - 1 that isn't built on Fedora Current.  (could be something as silly 
as a buildsystem bug that didn't produce all the packages.  Have to bump the 
nvr to build again)

-- 
Jesse Keating
Release Engineer: Fedora

[Tom Callaway]

On Thu, 2007-01-18 at 10:29 -0500, Jesse Keating wrote:
>  On Thursday 18 January 2007 10:18, Tom 'spot' Callaway
> wrote:
>  > Well, it turns out I don't think this will be necessary:
>  >
>  > I started writing this up late on Tuesday:
>  >
>  > http://fedoraproject.org/wiki/PackagingDrafts/ExceptionJPackage
>  
>  I don't see how this handles multiple Fedora releases.
>  
>  This works great if you only consider jpackage as one release, and Current 
>  Fedora as one release.  However we DO have to consider:
>  
>  jpackage--
>            |
>             - Fedora Current
>            |
>             - Fedora (Current - 1)
>            |
>             - Fedora (Current - 2)
>  
>  Obviously one can't use the same NVR on all three Fedora lines, so something 
>  has to be used to differentiate them, and allow for rebuilding something on 
>  Current - 1 that isn't built on Fedora Current.  (could be something as silly 
>  as a buildsystem bug that didn't produce all the packages.  Have to bump the 
>  nvr to build again) 
Hmm, that is a good point. Normally, we'd give the packager the choice
of using %{?dist} or bumping the release, but since we're trying to
ensure hierarchy, %{?dist} is the only real option.

Just appending %{?dist} to the end of the subrelease scheme solves that
issue, I'll update the document.

~spot

[Fernando Nasser]

Jesse Keating wrote:
>  On Thursday 18 January 2007 10:18, Tom 'spot' Callaway
> wrote:
> > Well, it turns out I don't think this will be necessary:
> >
> > I started writing this up late on Tuesday:
> >
> > http://fedoraproject.org/wiki/PackagingDrafts/ExceptionJPackage
>  
>  I don't see how this handles multiple Fedora releases.
>  
>  This works great if you only consider jpackage as one release, and Current 
>  Fedora as one release.  However we DO have to consider:
>  
>  jpackage--
>            |
>             - Fedora Current
>            |
>             - Fedora (Current - 1)
>            |
>             - Fedora (Current - 2)
>  
>  Obviously one can't use the same NVR on all three Fedora lines, so something 
>  has to be used to differentiate them, and allow for rebuilding something on 
>  Current - 1 that isn't built on Fedora Current.  (could be something as silly 
>  as a buildsystem bug that didn't produce all the packages.  Have to bump the 
>  nvr to build again)
>   
Perhaps this never happens because we always have some updates and we 
always do a mass rebuild with release bumping for any new Fedora release.

Ans as any fix for a Fedora -N is accompanied by it being applied to the 
subsequent Fedoras -(N -1) until current, that has never happened so far.

Anyway, if you guys want the %{_dist} at the end we will add them, I am 
just not really sure that this situation will ever happen.

[Jesse Keating]

On Thursday 18 January 2007 11:02, Fernando Nasser wrote:
>  Perhaps this never happens because we always have some updates and
> we
>  always do a mass rebuild with release bumping for any new Fedora release.
>
>  Ans as any fix for a Fedora -N is accompanied by it being applied to the
>  subsequent Fedoras -(N -1) until current, that has never happened so far.
>
>  Anyway, if you guys want the %{_dist} at the end we will add them, I am
>  just not really sure that this situation will ever happen. 
Why wouldn't FC6 users want the latest jpp releases of stuff when rawhide gets 
them?  Are they not suitable for FC6, ever?  We shouldn't actively prevent 
this from happening, especially now that we're opening the doors for more 
people to participate and take on some of the workload of import/building.

-- 
Jesse Keating
Release Engineer: Fedora

[Fernando Nasser]

Jesse Keating wrote:
>  On Thursday 18 January 2007 11:02, Fernando Nasser wrote:
> > Perhaps this never happens because we always have some updates and we
> > always do a mass rebuild with release bumping for any new Fedora release.
> >
> > Ans as any fix for a Fedora -N is accompanied by it being applied to the
> > subsequent Fedoras -(N -1) until current, that has never happened so far.
> >
> > Anyway, if you guys want the %{_dist} at the end we will add them, I am
> > just not really sure that this situation will ever happen.
>  
>  Why wouldn't FC6 users want the latest jpp releases of stuff when rawhide gets 
>  them?  Are they not suitable for FC6, ever?  We shouldn't actively prevent 
>  this from happening, especially now that we're opening the doors for more 
>  people to participate and take on some of the workload of import/building.
>   
I most certainly don't want to prevent that.  I was under the impression 
that Rawhide would always have a newer VR, which you are saying that 
needs the dist tag to be ensured.

I've asked the questions about the dist tag in my previous messages.  I 
believe I am not the only one with some questions about it, so the 
clarifications will be useful for others as well.

[Fernando Nasser]

Jesse Keating wrote:
>  On Thursday 18 January 2007 09:26, Fernando Nasser wrote:
> >> Spot brought the 3jpp.fc6.1 format to the packaging group which was
> >> discussed and agreed as a possible temporary format.  Unfortunately,
> >> 3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
> >> removed.  So we'll probably have to have another (hopefully short)
> >> discussion about using 3jpp.1.fc6 & 3.1.fc6.
> > Why the %{_dist} is necessary here?  Can't we just add the number?
> > I thought we would need the %{_dist} only if we needed to have the same
> > RPM built in two different distro releases and they were release
> > specific (depend on some shared library or something).
>  
>  Unfortunately it may be necessary for you folks, if we ever have to bump the 
>  package in an older release, without becoming NVR higher than the same 
>  package in a newer release.  If 3jpp is used in both FC6 and F7, and we have 
>  to rebuild in FC6 for whatever localized reason, we have to be able to do it 
>  in a way that doesn't promote it to be NVR higher than the 3jpp in F7.
>   
But isn't the norm that we do this only if such case ever happen?

Anyway, I believe the only case we would respin the FC6 one would be if 
there was a (security?) bug, so we will probably have to respin the FC7 
one as well, so it will end up being FC6 +1 anyway.  At least this is 
what has been happening so far.

Shouldn't we leave the %{_dist} out and add it only if absolutely 
necessary for a specific package?

[Jesse Keating]

On Thursday 18 January 2007 10:57, Fernando Nasser wrote:
>  But isn't the norm that we do this only if such case ever happen?
Without the dist there beforehand, it doesn't help to add it after, as foo.fc6 
will be newer than just foo, where foo could have been foo.fc7.

>  Anyway, I believe the only case we would respin the FC6 one would be
> if
>  there was a (security?) bug, so we will probably have to respin the FC7
>  one as well, so it will end up being FC6 +1 anyway.  At least this is
>  what has been happening so far. 
That isn't always the case.  A bug with the gcj version in FC6 could cause a 
rebuild, and that bug may not exist in the F7 version of gcj.  Or there could 
have been a buildsystem issue when building 3jpp for all branches, requiring 
you to rebuild on FC6 branch but not F7 branch.  These things do happen, we 
need to account for them instead of trying to work around them with epochs 
later.

>  Shouldn't we leave the %{_dist} out and add it only if
> absolutely
>  necessary for a specific package? 
Again, by the time you realize you need it, its too late, unless you rebuild 
on ALL branches just to add the dist tag, and then you defeat the purpose.  
ANY package that is using the same upstream version should probably use the 
dist tag to avoid non-fun games with adjusting release across branches.

-- 
Jesse Keating
Release Engineer: Fedora

[Fernando Nasser]

Hi Jesse,,

Please note that I don't have any problems with using the dist tag, I 
just want to make sure I understand the situations where it should be 
used so that I (and others who are followint the thread) know how to use 
it properly.


Jesse Keating wrote:
>  On Thursday 18 January 2007 10:57, Fernando Nasser wrote:
> > But isn't the norm that we do this only if such case ever happen?
>  
>  Without the dist there beforehand, it doesn't help to add it after, as foo.fc6 
>  will be newer than just foo, where foo could have been foo.fc7.
>  
> > Anyway, I believe the only case we would respin the FC6 one would be if
> > there was a (security?) bug, so we will probably have to respin the FC7
> > one as well, so it will end up being FC6 +1 anyway.  At least this is
> > what has been happening so far.
>  
>  That isn't always the case.  A bug with the gcj version in FC6 could cause a 
>  rebuild, and that bug may not exist in the F7 version of gcj.  Or there could 
>  have been a buildsystem issue when building 3jpp for all branches, requiring 
>  you to rebuild on FC6 branch but not F7 branch.  These things do happen, we 
>  need to account for them instead of trying to work around them with epochs 
>  later.
>   
But this can happen with any package right?  If any package is not 
upgraded between fc6 and FC7 for instance, it is subject to have to be 
respun for FC6 and that make it newer than the one in FC7.

Lets say ppp was not updated between FC6 and FC7. So we have:

FC6  ppp-2.4.4-1

FC7  ppp-2.4.4-2

And if ppp has to be updated in FC6 and not in FC7 we'd be in trouble, 
right?

As we cannot ensure this situation will never happen for any package, 
ALL packages would have to use dist.

Or is it that we only do that if there was no major version updates 
between FC6 and FC7?  As we never do major upgrades in older releases 
that would take care of it.


> > Shouldn't we leave the %{_dist} out and add it only if
> absolutely
> > necessary for a specific package?
>  
>  Again, by the time you realize you need it, its too late, unless you rebuild 
>  on ALL branches just to add the dist tag, and then you defeat the purpose.  
>  ANY package that is using the same upstream version should probably use the 
>  dist tag to avoid non-fun games with adjusting release across branches.
>   
I think what I need to understand now is why it affects more packages 
with an upstream version than the others.  You are probably thinking of 
a specific scenario, what is it?

[Jesse Keating]

On Thursday 18 January 2007 11:43, Fernando Nasser wrote:
>  > Again, by the time you realize you need it, its too late, unless
> you
>  > rebuild on ALL branches just to add the dist tag, and then you defeat the
>  > purpose. ANY package that is using the same upstream version should
>  > probably use the dist tag to avoid non-fun games with adjusting release
>  > across branches.
>
>  I think what I need to understand now is why it affects more packages
>  with an upstream version than the others.  You are probably thinking of
>  a specific scenario, what is it? 
Well, some people try to manage this themselves, by using a release of "6" for 
FC6 branch, release of "7" for F7 branch.  Then if they have to respin on 6, 
they go from 6 to 6.1 which is still older than 7.

grog upstream release 4.1.

grog-4.1-6  = FC6
grog-4.1-7  = F7

grog-4.1-6.1 = FC6 respin still older than F7

We still have to do this with disttag too, but then we don't have to have fake 
release numbers per se.

grog-4.1-1%{?dist}   Can be used across both branches, giving you:

grog-4.1-1.fc6 = FC6
grog-4.1-1.fc7 = F7

One can continually use the same exact spec file for both branches until there 
comes a time that you actually have to fork one of them, and you'd add a .1 
after the dist tag:

grog-4.1-1.fc6.1 = FC6 respun
grog-4.1-1.fc7 = F7, still NVR newer.

When using disttag, you can branch later.  When NOT using disttag, you have to 
branch immediately to ensure that NVRs are not the same and are in proper 
upgrade order.

-- 
Jesse Keating
Release Engineer: Fedora

[Toshio Kuratomi]

On Thu, 2007-01-18 at 09:26 -0500, Fernando Nasser wrote:
>  Hi Toshio,
>  
>  Toshio Kuratomi wrote:
>  > Spot brought the 3jpp.fc6.1 format to the packaging group which was
>  > discussed and agreed as a possible temporary format.  Unfortunately,
>  > 3jpp.fc6.1 doesn't work for interleaving with jpackage if jpp is
>  > removed.  So we'll probably have to have another (hopefully short)
>  > discussion about using 3jpp.1.fc6 & 3.1.fc6.
>  > 
>  
>  Why the %{_dist} is necessary here?  Can't we just add the number?
>  I thought we would need the %{_dist} only if we needed to have the same 
>  RPM built in two different distro releases and they were release 
>  specific (depend on some shared library or something).
>   
Jesse seems to be answering this one.

>  > 
>  >> 3)  What do we need to get rid of the 'jpp'
>  >>
>  >> a) Query for the set of Java packages
>  >>
>  >> The rpm -qg (or rpm -q --group) command currently does not accept patterns
>  >>
>  >> If we could do 'rpm -qg "Java*"' we could add
> "Java/" to all "Group:" 
>  >> tags of all Java packages and that would work.
>  >>
>  >> I am trying to create a patch for that (which would be RPM patch #37 in 
>  >> Fedora) but I am encountering some difficulties as it is the first time 
>  >> I am looking at the rpm source code and I have been working with Java 
>  >> rather than C in the last years.
>  >>
>  >> P.S.: We would need a similar query functionality when Categories 
>  >> replace Groups.
>  >> 
[snip]
>  > Given this, I'd be very hesitant to make group functionality
> in rpm be
>  > one of the criteria to get rid of the jpp naming.
>  > 
>  
>  You missed the "P.S."
>  
>  I am creating a patch for Groups because Categories are not there yet, 
>  so I can't do anything for it ATM.  Once Groups are abandoned in favor 
>  of Categories, the functionality available for Groups must be adapted to 
>  work with Categories instead.
>   
I read that but perhaps one of us is misinterpreting the other.  I'm
saying the following:
1) Keeping group and category functionality in rpm's spec file is going
to be deprecated for Fedora.
2) rpm doesn't have any business reading an external xml file from the
repository.

Therefore any Group functionality added to rpm now, should be
implemented outside of rpm when we get Categories.  As long as you're
okay with the functionality that you're currently implementing being
moved to another tool (for instance, yum or a program in yum-utils) then
I'm okay with this.

>  In any case, I am not trying to add any new switch or functionality,
>
>  just fix an existing one that has an annoying shortcoming ('rpm -q 
>  --group' does not take a regexp/glob).
>   
That's fine.  Like I say I'm not judging how likely it'll get into
rpm,
just that the functionality probably needs to live in another tool when
we move to categories.

>  
>  >> b) yum has already some group functionality (groupinstall, groupupdate, 
>  >> groupremove, groupinfo) that is based on an XML file that is kept in the 
>  >> repository.  But the option --exclude only acts on file names, we need a 
>  >> --groupexclude.
>  >>
>  >> We already have a "Java" group, with only 2 packages on it:
>  >>
>  >> # yum groupinfo Java
>  >> Loading "installonlyn" plugin
>  >> Setting up Group Process
>  >> Setting up repositories
>  >>
>  >> Group: Java
>  >>  Description: Support for running programs written in the Java 
>  >> programming language.
>  >>  Mandatory Packages:
>  >>    libgcj
>  >>    java-1.4.2-gcj-compat
>  >>
>  > If I understood spot's summary of your requirements, you want to
>  > categorize all the packages you are importing from jpackage.  I would
>  > point out that the java group would be a poor place to implement that as
>  > any java packages which did not come through jpackage could also
>  > legitimately be placed in there.
>  > 
>  
>  We actually want all Java packages.  They are always extremely 
>  interrelated, with cascading effects among them, so they need to be 
>  treated as a whole.  Any Java package should be in there.
>   
Okay.  My misunderstanding then.

[snip]
>  >>
>  > One controversy with using the yum group functionality for this is that
>  > it uses comps.xml to achieve its ends.  Jeremy Katz is against using
>  > comps.xml to make categories that shouldn't be visible to the installer.
>  > skvidal has shown how we could have multiple comps.xml files that yum
>  > reads (whereas the installer only reads from one) which addresses a few
>  > of Jeremy's concerns.  When we go ahead with this we'll have to work out
>  > whether we're going to store the information in a comps file or teach
>  > yum about a new grouping file format.
>  > 
>  
>  In any case, there will be always something for groupinstall, 
>  groupupdate, groupremove, groupinfo and also a --groupexclude to do the 
>  selection. 
Yep. Just letting you know if you're doing some hacking in this area to
be aware that the backend could change here.  And it's early in the
process so you can be a part of making this decision as well.

-Toshio

[Fernando Nasser]

Toshio Kuratomi wrote:
(...)
> > You missed the "P.S."
> >
> > I am creating a patch for Groups because Categories are not there yet, 
> > so I can't do anything for it ATM.  Once Groups are abandoned in favor 
> > of Categories, the functionality available for Groups must be adapted to 
> > work with Categories instead.
> >
> >     
>  I read that but perhaps one of us is misinterpreting the other.  I'm
>  saying the following:
>  1) Keeping group and category functionality in rpm's spec file is going
>  to be deprecated for Fedora.
>     
Oh, I thought the Group functionality was being deprecated but we would 
have a Category one to replace it (where packages could belong to many 
instead of a single one). 

So I did misunderstood that.  I even thought you wanted to store the 
Categories in the RPM database.   If that was the case it was just a 
different Query (on Categories instead of Groups, and I would not even 
need a glob pattern in that case).

>  2) rpm doesn't have any business reading an external xml file
> from the
>  repository.
>
>  Therefore any Group functionality added to rpm now, should be
>  implemented outside of rpm when we get Categories.  As long as you're
>  okay with the functionality that you're currently implementing being
>  moved to another tool (for instance, yum or a program in yum-utils) then
>  I'm okay with this.
>
>     
That is what I haven't understood.  It doesn't have to be in rpm at all,

anything that produces the list of packages in that Category can be used.


> > In any case, I am not trying to add any new switch or
> functionality, 
> > just fix an existing one that has an annoying shortcoming ('rpm -q 
> > --group' does not take a regexp/glob).
> >
> >     
>  That's fine.  Like I say I'm not judging how likely it'll get into rpm,
>  just that the functionality probably needs to live in another tool when
>  we move to categories.
>     
It is OK.   I was just trying to accelerate things but perhaps it would 
be wise to wait a little bit,

Are we removing the Groups table from the database?

>  (...)
> >> One controversy with using the yum group functionality for this is that
> >> it uses comps.xml to achieve its ends.  Jeremy Katz is against using
> >> comps.xml to make categories that shouldn't be visible to the installer.
> >> skvidal has shown how we could have multiple comps.xml files that yum
> >> reads (whereas the installer only reads from one) which addresses a few
> >> of Jeremy's concerns.  When we go ahead with this we'll have to work out
> >> whether we're going to store the information in a comps file or teach
> >> yum about a new grouping file format.
> >>
> >>       
> > In any case, there will be always something for groupinstall, 
> > groupupdate, groupremove, groupinfo and also a --groupexclude to do the 
> > selection.
> >     
>
>  Yep. Just letting you know if you're doing some hacking in this area to
>  be aware that the backend could change here.  And it's early in the
>  process so you can be a part of making this decision as well.
>
>     
Is that wiki page you've pointed us to the only place to watch ATM?

http://www.fedoraproject.org/wiki/Extras/SIGs/Comps/PackageGroupEnhancements


Regards,
Fernando

[Toshio Kuratomi]

On Thu, 2007-01-18 at 15:51 -0500, Fernando Nasser wrote:
>  Toshio Kuratomi wrote:
>  > Therefore any Group functionality added to rpm now, should be
>  > implemented outside of rpm when we get Categories.  As long as you're
>  > okay with the functionality that you're currently implementing being
>  > moved to another tool (for instance, yum or a program in yum-utils) then
>  > I'm okay with this.
>  >
>  >   
>  That is what I haven't understood.  It doesn't have to be in rpm at all, 
>  anything that produces the list of packages in that Category can be used.
>   
Cool.  No objections from me then.

>  
>  >> In any case, I am not trying to add any new switch or functionality, 
>  >> just fix an existing one that has an annoying shortcoming ('rpm -q 
>  >> --group' does not take a regexp/glob).
>  >>
>  >>     
>  > That's fine.  Like I say I'm not judging how likely it'll get into rpm,
>  > just that the functionality probably needs to live in another tool when
>  > we move to categories.
>  >   
>  It is OK.   I was just trying to accelerate things but perhaps it would 
>  be wise to wait a little bit,
>  
>  Are we removing the Groups table from the database?
>   
The way this is looking is that we'll be making the Group tag optional
when building rpms.  We'll port our tools to use the external Categories
file instead.  We'll encourage or help port third party tools (like apt
and smart) to to be able to use the external file as well.

The group table in the database will still be there but there won't be
any guarantees that it holds useful information.

>  > Yep. Just letting you know if you're doing some hacking in
> this area to
>  > be aware that the backend could change here.  And it's early in the
>  > process so you can be a part of making this decision as well.
>  >
>  >   
>  Is that wiki page you've pointed us to the only place to watch ATM?
>  
>  http://www.fedoraproject.org/wiki/Extras/SIGs/Comps/PackageGroupEnhancements 
Unfortunately it's the only place on the wiki that has information but
there's currently no place that's focusing the efforts.  The Packaging
Committee and FESCo have discussed dropping the Group tag and have
authorized rpm to allow packages without the Group tag, but no change to
the Guidelines has been made to allow approving packages without the
Group Tag.  We want the alternative categorization infrastructure to
exist first.

The alternative is likely to be enabled via the package db so some
effort is going in there.  However, the deployment that depsolvers see
needs to use a file in the repository rather than an SQL database.  This
is where using comps.xml or a separate format comes in.  This ties into
yum as yum will need to work with whatever file format is chosen.

-Toshio