Fedora Core 5 Wishlist

[Thomas Fitzsimmons]

Hi,

I tried to send this yesterday but there was a problem with the Fedora
list servers.

Tom

-------- Forwarded Message --------
>  From: Thomas Fitzsimmons <fitzsim(a)redhat.com&gt;
>  To: fedora-devel-java-list(a)redhat.com
>  Subject: Fedora Core 5 Wishlist
>  Date: Tue, 13 Sep 2005 21:16:31 -0400
>  
>  Hi,
>  
>  I've been thinking about my GCJ-related goals for Fedora Core 5, due out
>  February 2006.  Here's a categorized list:
>  
>  java-gcj-compat
>  ===============
>  
>  - GNU Crypto fix for Eclipse extssh support:
>  
>    https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=146782
>  
>    Casey Marshall has already committed a Diffie-Hellman JCE provider to
>  GNU Classpath so this is just a matter of testing.
>  
>  - import all JAWT fixes:
>  
>    All the necessary patches are already in GNU Classpath and libgcj so
>  this is just a matter of testing that JOGL works properly on our
>  java-1.4.2-gcj-compat RPM.
>  
>  - Jessie merged into GNU Classpath
>  
>  - GNU Crypto core algorithms merged into GNU Classpath
>  
>  - gjdoc merged into GNU Classpath and thus ...
>  
>  - an --enable-javadocs option to libgcj
>  
>  - gcjx becomes java-gcj-compat's javac and javah commands:
>  
>    Not sure if this one is doable in time; I don't mean full gcjx, just
>  the bytecode-compiler portion of it split off into a standalone javac
>  executable.
>  
>  - include Tritonus or some sound implementation
>  
>  AWT
>  ===
>  
>  - ImageMagick backend for ImageIO
>  
>  - Graphics/Imaging refactoring for GTK 2.8 and Cairo
>  
>  - MegaMek packages in Fedora Extras:
>  
>    http://megamek.sourceforge.net/
>  
>    This just involves a few AWT bug fixes now.
>  
>  - fix all 1.5 japi issues in the java.awt.* packages
>  
>  - fix all 1.5 japi issues in the javax.imageio.* packages
>  
>  Swing
>  =====
>  
>  - RHDB tools in Fedora Extras:
>  
>    http://sources.redhat.com/rhdb/administrator.html
>  
>    http://sources.redhat.com/rhdb/visualexplain.html
>  
>    http://sources.redhat.com/rhdb/controlcenter.html
>  
>    This will also require updating these tools to support PostgreSQL 8.0.
>  
>  - Limewire in Fedora Extras:
>  
>    http://www.limewire.org/
>  
>    It would be absolutely awesome to have this Swing app working on the
>  free stack but I'm not sure it's doable in the FC5 time frame.  Would
>  anyone like to undertake it? ;-)
>  
>  SWT
>  ===
>  
>  - Azureus in Fedora Extras:
>  
>    http://azureus.sourceforge.net/
>  
>    Again, not sure if this is doable before FC5 but this would be awesome
>  to have.
>  
>  gcjwebplugin
>  ============
>  
>  - implement and test security features in GNU Classpath to allow running
>  untrusted bytecode
>  
>    It's very unlikely that this one will be finished before FC5 but I'd
>  like to at least have started the implementation by then.
>  
>  I'm hoping this list will inspire some volunteers, especially for the
>  big apps and GNU Classpath's security framework.  Most of these items
>  are very doable and will greatly improve the free JPackage stack on
>  Fedora.
>  
>  Tom
>

[Ian Pilcher]

Something I think is missing....

I believe that there needs to be a clear strategy for the co-existence
of Fedora gcj packages and JPackage "pure Java" packages.  The current
situation is a mess:

  * If the JPackage repos are added to a FC4 yum configuration, yum will
    try to overwrite gcj packages with native packages, and vice versa.

  * There's no way for a user to know what the effects of this, beyond
    performance will be.

etc...

-- 
========================================================================
Ian Pilcher                                        i.pilcher(a)comcast.net
========================================================================

[Gary Benson]

Ian Pilcher wrote:
>  Something I think is missing....
>  
>  I believe that there needs to be a clear strategy for the co-existence
>  of Fedora gcj packages and JPackage "pure Java" packages.  The current
>  situation is a mess:
>  
>    * If the JPackage repos are added to a FC4 yum configuration, yum will
>      try to overwrite gcj packages with native packages, and vice versa.
>  
>    * There's no way for a user to know what the effects of this, beyond
>      performance will be.
>  
>  etc... 
Suggestions welcome :)

[Ian Pilcher]

Gary Benson wrote:
>  Suggestions welcome :) 
I'm not sure what the best solution is, but here are some alternatives
to consider:

  1.  Change the names of gcj packages, so that they don't match the
      JPackage versions.  For example, gcj-tomcat5 could "provide"
      tomcat5.  It should be pretty easy to conditionalize this in the
      SPEC files.

  2.  Move the Fedora gcj packages into a separate repository from the
      rest of the distribution.  This would at least allow people to
      create a "pure" JPackage system by simply disabling this repo.

That's all I can think of for now.  Perhaps others have ideas.

-- 
========================================================================
Ian Pilcher                                        i.pilcher(a)comcast.net
========================================================================

[Andrew Haley]

Ian Pilcher writes:
>  Gary Benson wrote:
>  > Suggestions welcome :)
>  
>  I'm not sure what the best solution is, but here are some alternatives
>  to consider:
>  
>    1.  Change the names of gcj packages, so that they don't match the
>        JPackage versions.  For example, gcj-tomcat5 could "provide"
>        tomcat5.  It should be pretty easy to conditionalize this in the
>        SPEC files.
>
>    2.  Move the Fedora gcj packages into a separate repository from the
>        rest of the distribution.  This would at least allow people to
>        create a "pure" JPackage system by simply disabling this repo. 
This seems like a really bad idea because it doesn't allow gcj
packages to provide dependencies for JPackages.

>  That's all I can think of for now.  Perhaps others have ideas.
Andrew.

[Gary Benson]

Ian Pilcher wrote:
>  Gary Benson wrote:
>  > Suggestions welcome :)
>  
>  I'm not sure what the best solution is, but here are some
>  alternatives to consider:
>  
>    1.  Change the names of gcj packages, so that they don't match the
>        JPackage versions.  For example, gcj-tomcat5 could "provide"
>        tomcat5.  It should be pretty easy to conditionalize this in
>        the SPEC files. 
You wouldn't believe how awful this is.  I've done it before, for
Stronghold.  It seems trivial, but it's actually a nightmare, and
the total package count in Stronghold was a tenth of what we're
dealing with now.

>    2.  Move the Fedora gcj packages into a separate repository from
> the
>        rest of the distribution.  This would at least allow people to
>        create a "pure" JPackage system by simply disabling this repo.
>  
>  That's all I can think of for now.  Perhaps others have ideas.
>  
>  -- 
>  ========================================================================
>  Ian Pilcher                                        i.pilcher(a)comcast.net
>  ========================================================================
>  
>  --
>  fedora-devel-java-list mailing list
>  fedora-devel-java-list(a)redhat.com
>  https://www.redhat.com/mailman/listinfo/fedora-devel-java-list

[Chris Hubick]

On Thu, 2005-09-15 at 14:56 +0100, Gary Benson wrote:
>  Ian Pilcher wrote:
>  > I believe that there needs to be a clear strategy for the co-existence
>  > of Fedora gcj packages and JPackage "pure Java" packages.  The current
>  > situation is a mess:
>
>  Suggestions welcome :) 
Distribute JPackage packages "as is".  For each JPP package shipped in
Fedora, create a corresponding "<jppname>-bin" package containing the
compiled library code.

RPM will know about the "-bin" dependencies, and those can depend on
each other if needed.  One would hope that the binary packages don't
need any specific patches, but if they do, feed them up to JPackage.
Maybe split JPP into not just "Free" and "Non-Free", but also
"Free and
works on Free compilers/runtimes".  Work to get upstream projects like
Apache to abstract code with Non-Free dependencies into separate Ant
targets, facilitating the creation of sub-packages for RPM's also
abstracting away Non-Free code.

Eventually work towards automating the binary builds from JPP SRPM's,
with the ultimate goal of having a Yum repository which parallels the
JPackage one, or is even included as part of JPP, containing "-bin"
RPM's for all packages.  Or you could create a whole Fedora specific
Java repository to replace using JPP, and it could suck in the JPP
srpm's, compile, and provide both noarch and arch results.

-- 
Chris Hubick
mailto:[e-mail]
http://www.hubick.com/

[Gary Benson]

Chris Hubick wrote:
>  On Thu, 2005-09-15 at 14:56 +0100, Gary Benson wrote:
>  > Ian Pilcher wrote:
>  > > I believe that there needs to be a clear strategy for the
>  > > co-existence of Fedora gcj packages and JPackage "pure Java"
>  > > packages.  The current situation is a mess:
>  >
>  > Suggestions welcome :)
>  
>  Distribute JPackage packages "as is". 
There's no way we can commit to shipping JPackage packages
unmodified.  But, even if we could...

>  For each JPP package shipped in Fedora, create a corresponding
>  "<jppname>-bin" package containing the compiled library code. 
There are several reasons not to do this but the showstopper is that
if someone does 'yum install jonas-examples' on an empty system they
will not get the native code.

Cheers,
Gary

[Thomas Fitzsimmons]

On Thu, 2005-09-15 at 14:56 +0100, Gary Benson wrote:
>  Ian Pilcher wrote:
>  > Something I think is missing....
>  > 
>  > I believe that there needs to be a clear strategy for the co-existence
>  > of Fedora gcj packages and JPackage "pure Java" packages.  The current
>  > situation is a mess:
>  > 
>  >   * If the JPackage repos are added to a FC4 yum configuration, yum will
>  >     try to overwrite gcj packages with native packages, and vice versa.
>  > 
>  >   * There's no way for a user to know what the effects of this, beyond
>  >     performance will be.
>  > 
>  > etc...
>  
>  Suggestions welcome :) 
All we need to do to make this work properly is make sure that the FC4
packages always have versions >= the corresponding JPackage versions.
Then packages that are in JPackage but not in Fedora will be provided by
JPackage and all other packages will come from Fedora.

Our repackaging of JPackages is meant to be 100% compatible with the
source JPackages.  If a given package is not compatible then that's a
bug in that package.  Otherwise Fedora users should always want the
Fedora versions of the packages.

In other words, if our Fedora package versions always match or exceed
their JPackage equivalents then there is no problem with JPackage and
Fedora packages co-existing without a clear boundary between the two
repositories.  I just don't see this as a problem at all.

Tom

[Vadim Nasardinov]

On Thursday 15 September 2005 12:52, Gary Benson wrote:
>  There are several reasons not to do this but the showstopper is that
>  if someone does 'yum install jonas-examples' on an empty system they
>  will not get the native code. 
You may extend the RPM spec file with an optional header like the
following:


    Name: jonas-examples

    ... yadda yadda yadda ...

    CanBeMassivelySpedUpBy: jonas-examples-compiled-by-gcj

With this in place, yum can now fetch jonas-examples-compiled-by-gcj
if is available.

This requires changes to both rpm and yum.  I have no idea how
feasible this would be, but I don't remember this option having been
discussed yet.

[Vadim Nasardinov]

On Thursday 15 September 2005 13:27, Thomas Fitzsimmons wrote:
>  In other words, if our Fedora package versions always match or
>  exceed their JPackage equivalents 
That's a pretty big "if", is it not?

[Vadim Nasardinov]

On Thursday 15 September 2005 12:24, Chris Hubick wrote:
>  Maybe split JPP into not just "Free" and
> "Non-Free", but also "Free
>  and works on Free compilers/runtimes". 
Although this would probably take more effort than anyone's is willing
to invest at this point, I do like the general sound of this idea.

[Chris Hubick]

On Thu, 2005-09-15 at 13:50 -0400, Vadim Nasardinov wrote:
>  On Thursday 15 September 2005 12:52, Gary Benson wrote:
>  > There are several reasons not to do this but the showstopper is that
>  > if someone does 'yum install jonas-examples' on an empty system they
>  > will not get the native code.
>  
>  You may extend the RPM spec file with an optional header like the
>  following:
>  
>  
>      Name: jonas-examples
>  
>      ... yadda yadda yadda ...
>  
>      CanBeMassivelySpedUpBy: jonas-examples-compiled-by-gcj
>  
>  With this in place, yum can now fetch jonas-examples-compiled-by-gcj
>  if is available. 
Or maybe some way to have one package state that it "masks" another
package?

Name: jonas-examples-compiled-by-gcj

# Anyone tries to install 'jonas-examples', they get this
# package instead.
Masks: jonas-examples 

# we need the real jonas-examples, because we are *adding*
# to it (binaries), without this line, this package would
# just *replace* the masked one.
Requires: jonas-examples

%install
install our additional binary stuff here

-- 
Chris Hubick
mailto:[e-mail]
http://www.hubick.com/

[David Walluck]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vadim Nasardinov wrote:

>  You may extend the RPM spec file with an optional header like the
>  following:  
I thought rpm has something like `Suggests' now, but this won't force a
package.

Anyway, I think having the only reason for binary java packages be that
new users can't figure out how to install them is a pretty bad reason,
since you could always force the install somehow (without modifying rpm)
by inserting code into yum that matches *jpp* and checks for a native
package to install on top of it.

- --
Sincerely,

David Walluck
<david(a)zarb.org&gt;
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDKgcNarJDwJ6gwowRAtPXAJ4twT3SaQVTrjXVceKX0NQHRYF3FACfZpWV
> 1huAOThmFsTzMy9jg5kBb2A=
> =X5Uu
> -----END PGP SIGNATURE-----

[Vadim Nasardinov]

On Thursday 15 September 2005 19:43, David Walluck wrote:
>  Vadim Nasardinov wrote:
>  > You may extend the RPM spec file with an optional header 
...
>  you could always force the install somehow (without modifying rpm)
>  by inserting code into yum that matches *jpp* and checks for a native
>  package to install on top of it. 
Even better.

[Gary Benson]

Thomas Fitzsimmons wrote:
>  Ian Pilcher wrote:
>  > I believe that there needs to be a clear strategy for the
>  > co-existence of Fedora gcj packages and JPackage "pure Java"
>  > packages.  The current situation is a mess:
>  
>  All we need to do to make this work properly is make sure that the
>  FC4 packages always have versions >= the corresponding JPackage
>  versions.  Then packages that are in JPackage but not in Fedora will
>  be provided by JPackage and all other packages will come from
>  Fedora.
>  
>  Our repackaging of JPackages is meant to be 100% compatible with the
>  source JPackages.  If a given package is not compatible then that's
>  a bug in that package.  Otherwise Fedora users should always want
>  the Fedora versions of the packages.
>  
>  In other words, if our Fedora package versions always match or
>  exceed their JPackage equivalents then there is no problem with
>  JPackage and Fedora packages co-existing without a clear boundary
>  between the two repositories.  I just don't see this as a problem
>  at all. 
Couldn't have said it better myself.  The only reason people are
starting to get strange mixes of packages is that Fedora has lagged
JPackage because there's not presently the manpower to keep them
level.

Cheers,
Gary

[David Walluck]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gary Benson wrote:
>  Couldn't have said it better myself.  The only reason people are
>  starting to get strange mixes of packages is that Fedora has lagged
>  JPackage because there's not presently the manpower to keep them
>  level. 
There's no guarantee of this and, in fact, it is quite likely that
jpackage versions will supercede FC4 versions (except there isn't much
jpackage manpower right now, either). That is, unless you were to bump
the epoch on every FC4 package as was done with eclipse (why the special
case here I am not sure).

On the other hand, people want more compatibility. And I want less
(ideally no) duplication. So I would again ask for some policy that
separates native libs from the jars. Or at least maybe someone is
willing to put some real thought into it rather than dismissing it
outright. By the way, I say this having fallen victim to the exact same
practices myself, where I have made patches and even full version
updates to several packages and just have had no time to push these
changes back to jpackage.org.

- --
Sincerely,

David Walluck
<david(a)zarb.org&gt;
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDKo+narJDwJ6gwowRAm/AAKCjz+hCPu04qQW1p9n/sajHOpnIKQCfSTCi
> sZyoWmiY0ga10MxKOSEI3l8=
> =LqHO
> -----END PGP SIGNATURE-----

[Gary Benson]

David Walluck wrote:
>  On the other hand, people want more compatibility. And I want less
>  (ideally no) duplication. So I would again ask for some policy that
>  separates native libs from the jars. Or at least maybe someone is
>  willing to put some real thought into it rather than dismissing it
>  outright. 
Before fedora-devel-java-list was set up we had precisely this
discussion internally, several times as it happens ;)  If I answer
you quickly it's not because I'm dismissing it but rather that I've
already thought long and hard about it beforehand.

I've been trying to dig out the discussion all morning, but I keep
getting sidetracked with email.  So, if you'll bear with me a few
moments longer...

Cheers,
Gary

[Andrew Haley]

David Walluck writes:
>  -----BEGIN PGP SIGNED MESSAGE-----
>  Hash: SHA1
>  
>  Gary Benson wrote:
>  > Couldn't have said it better myself.  The only reason people are
>  > starting to get strange mixes of packages is that Fedora has lagged
>  > JPackage because there's not presently the manpower to keep them
>  > level.
>  
>  There's no guarantee of this and, in fact, it is quite likely that
>  jpackage versions will supercede FC4 versions (except there isn't
>  much jpackage manpower right now, either). That is, unless you were
>  to bump the epoch on every FC4 package as was done with eclipse
>  (why the special case here I am not sure).
>  
>  On the other hand, people want more compatibility. And I want less
>  (ideally no) duplication. So I would again ask for some policy that
>  separates native libs from the jars. Or at least maybe someone is
>  willing to put some real thought into it rather than dismissing it
>  outright. 
No-one is doing that.  A great deal of real thought has been put into
this issue, and will continue to be.

Andrew.

[Gary Benson]

Gary Benson wrote:
>  David Walluck wrote:
>  > On the other hand, people want more compatibility. And I want less
>  > (ideally no) duplication. So I would again ask for some policy that
>  > separates native libs from the jars. Or at least maybe someone is
>  > willing to put some real thought into it rather than dismissing it
>  > outright.
>  
>  Before fedora-devel-java-list was set up we had precisely this
>  discussion internally, several times as it happens ;)  If I answer
>  you quickly it's not because I'm dismissing it but rather that I've
>  already thought long and hard about it beforehand.
>  
>  I've been trying to dig out the discussion all morning, but I keep
>  getting sidetracked with email.  So, if you'll bear with me a few
>  moments longer... 
Well, I'd forgotton just how much was said about it, but a brief
summary is that there were three methods proposed:

 1. Native code in the same rpms as the bytecode.
 2. Native code in separate rpms to the bytecode.
 3. Native code generated on user's machine.

Method 3, which hasn't been mentioned on this list, involved either
%post scriptlets or some kind of background job like slocate has.
It's main drawbacks were that native compilation is a hugely
computationally intensive process, particularly with aggressive
inter-class optimizations.  Many target machines simply wouldn't have
the RAM to do the job.  Even were this not the case, it can frequently
take hours, too long for a %post scriptlet, and using a background job
would mean that there'd be periods when your native code was not in
sync and your Java apps would run slowly.
 
Method 2, which is what you're proposing, has the following drawbacks
over Method 1:

 - There is no way at present to make rpm (yum, up2date, anaconda...)
   associate the native packages with the bytecode ones.  Users would
   have to manually install them in the first instance.

 - Installed native packages must Require their bytecode equivalents.
   Users could not upgrade a Fedora package with a JPackage one
   without manually uninstalling the native package first.

 - The native package would not have access to the sources, and would
   not be able to generate a useful debuginfo package.  This would
   render the packages undebuggable with gdb.

 - There's no obvious way to automate the generation of the native
   packages.  This causes the packagecount to more than double, and
   the non-atomic nature of it allows things to get out of sync and,
   for example, break rawhide.

But all this is somewhat tangential to the original point, which was
that "[if] the JPackage repos are added to a FC4 yum configuration,
yum will try to overwrite [Fedora] packages with [JPackage]
packages..."  Allowing the Fedora stack to be extended with JPackage
packages was and is a key requirement.  There's only one way I can see
that would prevent the former while still allowing the latter, and
that's for Fedora's packages to have the same EVR as the JPackage
packages.  Since that is something we cannot commit to, the problem
will remain regardless of where the native code is packaged.

Gary

[Andrew Overholt]

* David Walluck <david(a)zarb.org&gt; [2005-09-16 05:26]:
>  That is, unless you were to bump the epoch on every FC4 package as
> was
>  done with eclipse (why the special case here I am not sure). 
This was totally unrelated and was because we had decided to ship 3.0.2 in
FC4 at one point after having 3.1Mx in rawhide.  We had to bump the epoch
to go back.  Then we decided to go with 3.1 in the end.  Stupid?  Yes.

Andrew

[Thomas Fitzsimmons]

On Fri, 2005-09-16 at 12:24 +0100, Gary Benson wrote:
>  Gary Benson wrote: 
>  But all this is somewhat tangential to the original point, which was
>  that "[if] the JPackage repos are added to a FC4 yum configuration,
>  yum will try to overwrite [Fedora] packages with [JPackage]
>  packages..."  Allowing the Fedora stack to be extended with JPackage
>  packages was and is a key requirement.  There's only one way I can see
>  that would prevent the former while still allowing the latter, and
>  that's for Fedora's packages to have the same EVR as the JPackage
>  packages.  Since that is something we cannot commit to, the problem
>  will remain regardless of where the native code is packaged. 
I'm starting to think we shouldn't support Fedora Core/JPackage pull
arrangements, only Rawhide/JPackage arrangements.

Fedora Core and JPackage are on different release cycles; FC tends to
have "releases" whereas JPackages are continually updated individually.
I think each project's release strategy makes sense for it but the two
strategies are incompatible.

Without FC/JPackage setups where will people get packages that are in
JPackage but not in FC?  I'm wondering if a JPackage -> Fedora Extras
bridge is in order.  Right before a Fedora release we'd do a drop from
JPackage to Extras of all packages not in Core.  That way FE would be a
snapshot of JPackage at FC release time.  That would insulate FC users
from JPackage churn, while Rawhide users could continue using JPackage
directly.

Tom

raising Yum's IQ (was: Re: Re: Fedora Core 5 Wishlist)

[Vadim Nasardinov]

On Friday 16 September 2005 07:24, Gary Benson wrote:
>  a brief summary is that there were three methods proposed:
>  
>   1. Native code in the same rpms as the bytecode.
>   2. Native code in separate rpms to the bytecode.
>   3. Native code generated on user's machine.
>
>  Method 2, which is what you're proposing, has the following
>  drawbacks over Method 1:
>  
>   - There is no way at present to make rpm (yum, up2date,
>     anaconda...)  associate the native packages with the bytecode
>     ones.  Users would have to manually install them in the first
>     instance. 
Yum can be changed to have a way to associate the native packages with
the bytecode ones.

>   - Installed native packages must Require their bytecode
>     equivalents.  Users could not upgrade a Fedora package with a
>     JPackage one without manually uninstalling the native package
>     first. 
Yum can be changed to handle this.

>   - The native package would not have access to the sources, and
>     would not be able to generate a useful debuginfo package.  This
>     would render the packages undebuggable with gdb. 
Can you elaborate?  I thought native compilation worked by taking
.class files as input.  Where do the sources come in?  Secondly, why
can't you build native packages from the same SRPM that the binary
non-native RPMs were built from?  I don't believe this has been
discussed on this list in sufficient detail.

>   - There's no obvious way to automate the generation of the
> native
>     packages.  This causes the packagecount to more than double, and
>     the non-atomic nature of it allows things to get out of sync and,
>     for example, break rawhide. 
IIRC, David suggested repeatedly that JPackage would be more than
happy to accept .spec files where native compilation logic was present
conditionally and was off by default.  I seem to recall vaguely this
suggestion getting dismissed as impractical but don't remember any
detailed explanation of why this may not be feasible.

The bottom line is, although the current situation is the best
solution we could think of, it is not a good solution.  The fact of
the matter remains that Fedora and JPackage repositories don't mesh
very well at this time.  In view of this, why don't we explore the
option of adding explicit knowledge of the Java packaging situation to
Yum?  If this turns out to be a dead-end, fine.  At the very least, we
will have explained, in a public forum, why smarter Yum is not the
answer.

raising Yum's IQ (was: Re: Re: Fedora Core 5 Wishlist)

[Andrew Haley]

Vadim Nasardinov writes:
>  
>  >  - The native package would not have access to the sources, and
>  >    would not be able to generate a useful debuginfo package.  This
>  >    would render the packages undebuggable with gdb.
>  
>  Can you elaborate?  I thought native compilation worked by taking
>  .class files as input.  Where do the sources come in? 
You don't debug bytecode: you debug source.  Debuginfo contains
source.

Andrew.

raising Yum's IQ (was: Re: Re: Fedora Core 5 Wishlist)

[Vadim Nasardinov]

On Friday 16 September 2005 12:52, Andrew Haley wrote:
>  Vadim Nasardinov writes:
>  > On Friday 16 September 2005 07:24, Gary Benson wrote:
>  > > a brief summary:
>  > > 
>  > >  1. Native code in the same rpms as the bytecode.
>  > >  2. Native code in separate rpms to the bytecode.
>  > ...
>  > > Method 2, which is what you're proposing, has the following
>  > > drawbacks over Method 1:
>  > ...
>  > >  - The native package would not have access to the sources, and
>  > >    would not be able to generate a useful debuginfo package.
>  > >    This would render the packages undebuggable with gdb.
>  > 
>  > Can you elaborate?  I thought native compilation worked by taking
>  > .class files as input.  Where do the sources come in?
>  
>  You don't debug bytecode: you debug source.  Debuginfo contains
>  source. 
The part I didn't understand was how specifically Method 2 would
prevent you from generating appropriate debuginfo packages.  I think
the source of my confusion was that I misintepreted Gary's definition
of "Method 2".  I was thinking along the lines of David Walluck's
long-standing proposal (I hope I'm not putting words in his mouth) to
share exact same .spec files between Fedora and JPackage but have
native-compilation-related spec logic conditionalized.

I am a little fuzzy on why sharing exact same .spec files (modulo
minor distro-specific patches) is infeasible.

Even with Gary's definition of Method 2 (call it M2), it's not clear
to me why debuginfo packages are impossible to generate. (I think M2
amounts effectively to having two SRPMs for the same application).

raising Yum's IQ (was: Re: Re: Fedora Core 5 Wishlist)

[Andrew Overholt]

* Vadim Nasardinov <vadimn(a)redhat.com&gt; [2005-09-16 13:07]:
>  I am a little fuzzy on why sharing exact same .spec files (modulo
>  minor distro-specific patches) is infeasible. 
I have been out of the loop WRT Eclipse 3.1 packaging for a bit, but I
fully plan on sharing everything we possibly can between JPackage and FC
Eclipse .spec files.  I don't know how well this will work for other
packages, but I've already conditionalized most (if not all) of the
gcj-specific stuff in the Eclipse .spec file.  I never got the chance to
work with David and/or others to get our work into the JPackage .spec, but
I think David managed to extract most of the stuff that he could/wanted to.

Andrew "speaking only for myself" Overholt

raising Yum's IQ

[David Walluck]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Vadim Nasardinov wrote:
>  I am a little fuzzy on why sharing exact same .spec files (modulo
>  minor distro-specific patches) is infeasible. 
I think it was at my behest that gcj_support was added to some spec
files. Anyway, the good news is that I have actually gone and taken a
lot (if not all!) FC spec files and added `%if %{gcj_support}' in the
appropriate places. I just have been strapped for time so I haven't done
much of anything with these packages. Similarly, I really have very few
additions I would like to add to eclipse (wrt jpackage), I just haven't
had time to do it. So I don't blame Fedora here at all, but myself,
since I've actually already done this work and just can't seem to find
the time to do the last step.

Also, I obviously am not here to dictate Fedora policy, and we know that
without Fedora, the eclipse 3.1 work may never have gotten done. The
only thing I am in favor of, though, is a policy that might integrate
better cross-distro and/or avoids duplication of work (since I am sure
we all don't have the extra time).

In the end, I would prefer two separate spec files for each. It is
actually possible to do something with the rpm build scripts to perhaps
generate such a package 100% automatically (just as debuginfo packages
can be automatically generated now). The inability of rpm to have a
noarch subpackage of an arch package prevents what I'd call a near-ideal
solution. Already we have aot-compile-rpm in the rpm build scripts but
it's not being used. But I think this is a step in the right direction.
Perhaps a second rpmbuild process could be launched on the automatically
generated native spec file from the non-native build.

- --
Sincerely,

David Walluck
<david(a)zarb.org&gt;
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDKxrVarJDwJ6gwowRAiKYAJwNAtjiMeg4NHLRthx0aLvBSAxN5gCgnvZi
> 7ZrTPKKAjQIZSP8JkNm4yHI=
> =sxNz
> -----END PGP SIGNATURE-----

raising Yum's IQ

[Chris Hubick]

On Fri, 2005-09-16 at 15:19 -0400, David Walluck wrote:
>  Vadim Nasardinov wrote:
>  > I am a little fuzzy on why sharing exact same .spec files (modulo
>  > minor distro-specific patches) is infeasible.
>  
>  Anyway, the good news is that I have actually gone and taken a
>  lot (if not all!) FC spec files and added `%if %{gcj_support}' in the
>  appropriate places. 
When I read that, my first thought is: It would be nice if the macro's
used in JPP packages were generic/abstract enough that they didn't need
to mention GCJ at all, and that the definitions of the 'compilation'
macro could be modified to include doing .so builds in addition to the
class files on platforms with GCJ...

Then I start to realize that it would probably need to be wedged into
the Ant build file called by the macro.  Then I start to think it would
be nice if the Ant targets were generic enough to be able to do this
abstraction...

Then I realize to make this really work in an abstracted platform
independent fashion, we should all just use Maven for our projects, and
let the target platform sort out how to get that stuff distributed and
installed.

-- 
Chris Hubick
mailto:[e-mail]
http://www.hubick.com/

raising Yum's IQ

[David Walluck]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Chris Hubick wrote:
>  When I read that, my first thought is: It would be nice if the
> macro's
>  used in JPP packages were generic/abstract enough that they didn't need
>  to mention GCJ at all, and that the definitions of the 'compilation'
>  macro could be modified to include doing .so builds in addition to the
>  class files on platforms with GCJ... 
It mentions gcj because it's for gcj only. And gcj could be autodetected
by a macro too %(test -x %{_bindir}/gcj...). Remember that any
jpackage.org build server is also a platform with native binaries, and
probably has gcj installed. The existence of the gcj binary doesn't
imply someone necessarily wants native packages.

>  Then I start to realize that it would probably need to be wedged
> into
>  the Ant build file called by the macro.  Then I start to think it would
>  be nice if the Ant targets were generic enough to be able to do this
>  abstraction... 
Anthony Green has written gcjlib, but it is cumbersome to have to patch
every single build.xml, which is why a script that does the same thing
is generally preferred. But for upstream projects, it might make sense
for them to adopt gcjlib.

- --
Sincerely,

David Walluck
<david(a)zarb.org&gt;
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDKzNdarJDwJ6gwowRAs1TAJwLfSy85/D677AE5H1CUdLxHBeWugCffKBa
> acnPAK10RTTboaIjOXzwPKY=
> =EZp6
> -----END PGP SIGNATURE-----

Maven (was Re: raising Yum's IQ)

[Chris Hubick]

On Fri, 2005-09-16 at 17:04 -0400, David Walluck wrote:
>  Anthony Green has written gcjlib, but it is cumbersome to have to
> patch
>  every single build.xml, which is why a script that does the same thing
>  is generally preferred. But for upstream projects, it might make sense
>  for them to adopt gcjlib. 
It's cumbersome to have to patch build.xml files.

It's cumbersome to write and maintain spec files for hundreds of
applications, especially if there ended up having to be one 'generic'
JPP one, and one FC specific one.

Upstream Java developers shouldn't have to worry about RPM spec files,
deb files, GCJ libs, Windows installers, BeOS thingamajigs, etc.

When I moved my Java project from Windows using Sun's tools, to Fedora
and GCJ, my Ant build.xml file Just Worked.  That layer of insulation
meant I didnt' have to learn about GCJ command line options to compile
my code.  This type of abstraction needs to be taken to the next level.

Ideally, everyone just writes platform independent Java code, and fills
out some platform agnostic thing to describe their application, it's
version, dependencies, etc - all in a declarative fashion.  Everything
else from that point on through to end user deployment should be fully
automated by a mix of generic and platform specific tools.  I think
Maven's project.xml is a good start to supply exactly that.  There
should be tools/plugins/whatever, to turn a project.xml file into an RPM
suitable for Fedora, or a .deb for Debian, etc.  And if project.xml
doesn't supply all the information needed to make this work, maybe we
should help fix that.

Instead of JPP creating spec files, they could write a project.xml file
for everything.  Then as the mechanisms get worked out, you just upgrade
and maintain the tool chain.  Constantly futzing around tweaking code in
hundreds of hand written spec files seems silly to me.

-- 
Chris Hubick
mailto:[e-mail]
http://www.hubick.com/

raising Yum's IQ (was: Re: Re: Fedora Core 5 Wishlist)

[Gary Benson]

Vadim Nasardinov wrote:
>  The bottom line is, although the current situation is the best
>  solution we could think of, it is not a good solution.  The fact
>  of the matter remains that Fedora and JPackage repositories don't
>  mesh very well at this time. 
Placing the native code in separate rpms will not solve this because
the problem is not that Fedora rpms contain native code and JPackage
ones do not.  The problem is that Fedora and JPackage rpms share the
same namespace.  Ximian had problems with this five years ago, before
they were even called Ximian.  What is really needed is for some way
to tell yum to ignore packages in jpackage.repo that exist already in
fedora.repo.  I doubt it's that simple though.

Cheers,
Gary

raising Yum's IQ

[David Walluck]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gary Benson wrote:

>  ones do not.  The problem is that Fedora and JPackage rpms share the
>  same namespace.  Ximian had problems with this five years ago, before
>  they were even called Ximian.  What is really needed is for some way
>  to tell yum to ignore packages in jpackage.repo that exist already in
>  fedora.repo.  I doubt it's that simple though. 
But FC4 was said to be ``100% JPackage compatible''. Why couldn't
updates be provided through JPackage (I guess as a support issue it's
another story)? Also, if a particular package that *is not* in Fedora
requires an updated package (an older version of which *is* in Fedora),
then one loses the ability to upgrade this package from JPackage as well.

It would be nice to have one repository trump the other (in the general
case, but maybe not here). You'd have to specify either by name alone,
or by version. Also, you would have to be able to tell it to avoid
upgrades if it has to uninstall certain dependencies in order for the
upgrade to happen. As an example:

<package name>: [<'uninstall'|'keep'>] 'prefer'
<repo name> ['for'
<'name'|'version'>].

- --
Sincerely,

David Walluck
<david(a)zarb.org&gt;
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDLpFmarJDwJ6gwowRAmvsAKCgFoGUtHzVcqTXmyVRUu3cR4zFNQCgnEMm
> m9epODf8dX3jwrgX7KdunpE=
> =CUOJ
> -----END PGP SIGNATURE-----

raising Yum's IQ

[Gary Benson]

David Walluck wrote:
>  Gary Benson wrote:
>  > ones do not.  The problem is that Fedora and JPackage rpms share
>  > the same namespace.  Ximian had problems with this five years ago,
>  > before they were even called Ximian.  What is really needed is for
>  > some way to tell yum to ignore packages in jpackage.repo that
>  > exist already in fedora.repo.  I doubt it's that simple though.
>  
>  But FC4 was said to be ``100% JPackage compatible''. Why couldn't
>  updates be provided through JPackage? 
Some issues take time to fix properly.  The correct fix may not be
immediately obvious, or there may be several options that require
discussion in some upstream community.  Wherever possible I will work
around the issue locally so people aren't held up.  There's probably
15-20 such issues in rawhide at the moment.  Replacing Fedora packages
with JPackage ones will lose patches like that.

>  Also, if a particular package that *is not* in Fedora requires an
>  updated package (an older version of which *is* in Fedora), then one
>  loses the ability to upgrade this package from JPackage as well. 
Avoiding breakages without inhibiting things like this is difficult.

Cheers,
Gary

raising Yum's IQ

[Vadim Nasardinov]

On Monday 19 September 2005 05:14, Gary Benson wrote:
>  Placing the native code in separate rpms will not solve this because
>  the problem is not that Fedora rpms contain native code and JPackage
>  ones do not. 
Everyone agrees at this point that the problem is not solvable at the
RPM level.  The only hope is to move one level up and start looking at
what can be done at the Yum level.

>  What is really needed is for some way to tell yum to ignore packages
>  in jpackage.repo that exist already in fedora.repo.  I doubt it's
>  that simple though. 
Some of the possible choices are:

 (a) fedora.repo always trumps jpackage.repo;

 (b) jpackage.repo always trumps fedora.repo;

 (c) the user has a choice of specifying either (a) or (b) as their
     default policy.

     People who use java stuff in FC exclusively with GCJ won't want
     to lose the native bits due to updates coming from jpackage.repo.
     People who use java packages primarily under a proprietary JVM
     will be happy to pull upgrades from jpackage.repo even if it
     means losing the .so bits;

 (d) go wild and make the choice of repos configurable on a
     per-package basis.

Although I don't know how simple or difficult this is going to be, I
doubt it will be much worse than the current situation.  My initial
conservative preference would be (c) with fedora.repo trumping
jpackage.repo by default.

raising Yum's IQ

[Gary Benson]

Vadim Nasardinov wrote:
>  On Monday 19 September 2005 05:14, Gary Benson wrote:
>  > What is really needed is for some way to tell yum to ignore
>  > packages in jpackage.repo that exist already in fedora.repo.
>  > I doubt it's that simple though.
>  
>  Some of the possible choices are:
>  
>   (a) fedora.repo always trumps jpackage.repo;
>  
>   (b) jpackage.repo always trumps fedora.repo;
>  
>   (c) the user has a choice of specifying either (a) or (b) as their
>       default policy.
>  
>       People who use java stuff in FC exclusively with GCJ won't want
>       to lose the native bits due to updates coming from jpackage.repo.
>       People who use java packages primarily under a proprietary JVM
>       will be happy to pull upgrades from jpackage.repo even if it
>       means losing the .so bits;
>  
>   (d) go wild and make the choice of repos configurable on a
>       per-package basis.
>  
>  Although I don't know how simple or difficult this is going to be, I
>  doubt it will be much worse than the current situation.  My initial
>  conservative preference would be (c) with fedora.repo trumping
>  jpackage.repo by default. 
I like the sound of (c), though it's not just yum that needs to deal
with this: anaconda and possibly up2date need to handle it too.

Cheers,
Gary

raising Yum's IQ

[Vadim Nasardinov]

On Monday 19 September 2005 11:49, Gary Benson wrote:
>  I like the sound of (c), though it's not just yum that needs to
> deal
>  with this: anaconda and possibly up2date need to handle it too. 
I was hoping to keep my head in the sand about the anaconda/up2date
part of the story, until we get the yum situation straightened out
first.

raising Yum's IQ

[Mark Wilkinson]

Resurrecting an old thread as I'm catching up...

On 9/19/05, Vadim Nasardinov <vadimn(a)redhat.com&gt; wrote:
>  > What is really needed is for some way to tell yum to ignore packages
>  > in jpackage.repo that exist already in fedora.repo. I doubt it's
>  > that simple though.
>
>  (d) go wild and make the choice of repos configurable on a
>  per-package basis. 

My simple solution to this problem has been to add an exclude= line to my
jpackage.repo file listing the names of all the Java packages that shipped
with FC4:

[jpackage-generic]
name=JPackage (free), generic
mirrorlist=http://www.jpackage.org/jpackage_generic.txt
failovermethod=priority
gpgcheck=1
gpgkey=http://www.jpackage.org/jpackage.asc
enabled=1
exclude=ant, ant-antlr, ant-apache-bcel, ant-apache-log4j, ant-apache-oro,
ant-apache-regexp, ant-apache-resolver, ant-commons-logging, ...

and so on. The list of packages was built by grepping for [0-9]*jpp_[0-9]*fc
from the list of the packages that shipped with FC4, then stripping off the
version numbers.

When I first saw the problem mentioned on this list I was surprised that the
jpackage.org <http://jpackage.org> site didn't suggest something like this
to prevent JPackage packages from replacing the Fedora ones.

-Mark.