RFC: Proposal for a more agile "Fedora.next" (draft of my Flock talk)

[Toshio Kuratomi]

On Mon, Jul 22, 2013 at 05:54:27PM -0400, Peter MacKinnon wrote:
> 
> So far, so good...sort of. We can make the basic use case and tests work with
> the modified dependencies but in doing so we risk giving up parity with the
> Apache baseline (including the JRE) and potentially lose out to other so-called
> "dirty RPMs". Ideally, we wouldn't be forced into some of these adaptations and
> compromises if there were Fedora packaging alternatives that would give us (a
> SIG ring?) more control over the bundles needed by Hadoop as opposed to the
> ones mandated by the latest Fedora release. Make no mistake: patches are fed
> from the SIG to the Hadoop community to try to bump the versions there. But the
> upstream project can't and won't chase an ever-vanishing point in the distance.
> They view their lower dependencies much like a stable OS such as RHEL and
> change should be deliberated there.
> 
So -- would you be willing to go on a tangent with me for a short bit?
Looking at your issue, I have a few comments, some of which might even be
helpful :-)  I'm bringing this up because the way you describe Hadoop makes
it sound like people are going to want to build on top of it.  having it in
a Ring 2/3 non-rpm universe makes it more tempting for people to treat
Hadoop as an alternative choice which might mean that we get multiple copies
of hadoop there as people build on different hadoop foundations.  Figuring
out whether it's possible to build this in Fedora Commons might be better.

Observations-only:

* Bravo for your patching effort!  My experience in open source is that
  projects which relentlessly stick to old versions eventually fork and the
  fork which is supporting the newer dep stack.  Unfortunately, this switch
  of project direction is painful and can take a long while to come about.
  So even though your experience with patching up to more recent versions
  will be valuable when/if this actually happens, I understand that you will
  likely want to work on some other strategy first.

Ways to improve within the Fedora Core/Fedora Commons model:

* Although bundled libraries are not allowed in Fedora Core and Commons;
  both compat libraries and forks are allowed.  There are downsides to each:
  * Compat libraries may not have support from upstream.  Once upstream
    moves onto libfoo-2.0, they may no longer be interested in releasing
    bugfixes or even security fixes for libfoo-1.x.  This means that you, as
    the owner of the compat package would be responsible for those.  On the
    other hand, you'd be responsible for those even if they were bundled
    into the package you're really interested in.  It's just that you would
    mostly pretend that the Hadoop upstream was on top of those problems
    instead of having to deal with them yourself.  You could still rely
    heavily on the Hadoop upstream by syncing their fixes to the compat
    library in their bundled copy to your copy.
  * Forking increases the number of copies of virtually the same code, at
    least, until the forks diverge significantly.  Forking also means that
    someone needs to setup upstream infrastructure: an upstream SCM,
    a mailing list somewhere, an issue tracker (a lot of times, the ml and
    issue tracker start off as a piece of an existing project; only the SCM
    differs).  What does forking bring you, though?  The potential benefit
    is that you get a better upstream experience.  It's a place where people
    who need the same API version of libfoo can congregate and supply fixes
    that can be used in common with every other project that uses that code.
  * The upsides to both of these over bundled libraries are that you have
    better tracking of your dependency chain (Security issue discovered in
    libfoo that affects everything from 1.0.5 through 2.8.  When you have an
    external package, it's easier to see whether the code is affected and
    how many packages are affected than if it's bundled.  This can also be
    overcome using Virtual Provides to "tag" a package that's bundling
    a specific version.) and the possibility for more people to collaborate
    on this (We've often seen that multiple upstream packages are
    bundling upstream versions of a libfoo-1.x API.  When we pull these into
    a compat package, all the bugfix work can be pushed to the central
    package which cuts down on work for everyone.)
* Compat packages only help us if there's a way to parallel install library
  packages (and it sounds like for hadoop you might need parallel versions
  of the JRE as well).  Forking usually implies a new library name so that
  usually doesn't need parallel library support.  So the first
  implementation question might be:  Is parallel insallation and usage
  supported in your language?  if it isn't out of the box, is there a way we
  could create it (for instance, via a wrapper script that set CLASSPATH
  appropriately)?  For the JRE, is there a way to parallel iinstall it?  Is
  there also a way that the application being run can choose which JRE it
  needs to be run under?

-Toshio
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/devel/attachments/20130723/e2879947/attachment.sig>