[Fedora-packaging] Proposal to reduce anti-bundling requirements

[Adam Williamson]

On Thu, 2015-09-10 at 18:57 -0700, Adam Williamson wrote:

> I think I'm gonna go and do one of my deep dives into the history of
> these rules tonight. I have a definite memory that at one point the
> accepted wisdom was that only bundling of *already packaged* stuff
> was forbidden, but now i look i can find many recent counter-
> examples to this. So i want to look into it a little further...

Well, one of the first things I find is that I did this before...back
in June 2014 I posted this:

https://lists.fedoraproject.org/pipermail/devel/2014-June/199941.html

which got no responses. Pavel Alexeev followed up in August saying he
also would appreciate clarification:

https://lists.fedoraproject.org/pipermail/devel/2014-August/201705.html

and I replied again in January, again trying to get the issue some
attention:

https://lists.fedoraproject.org/pipermail/devel/2015-January/206142.html

but again there was no real clarification from anyone on FPC or FESCo
or anything like that. So, hell, I went and researched it myself.

Here's a kind of timeline of bundling policy/information page events:

before May 2008: "Duplication of system libraries" section (hereafter 'DOSL') added to Packaging:Guidelines
 * NOTE: Can find references to "duplication of system libraries" in review requests at least back to 2006
September 2008: "Bundling of multiple projects" section (hereafter 'BOMP') added to guidelines
 * FPC meeting: https://fedoraproject.org/wiki/Packaging:Minutes/20080826?rd=Packaging/Minutes/20080826
   This rule came from a discussion of font packages that bundled multiple font sources. It seems clearly
   to be the case that it was intended to cover 'downstream package using multiple unrelated source
   tarballs'; it was never intended to cover library bundling.
June 2009: https://fedoraproject.org/wiki/Packaging:No_Bundled_Libraries (hereafter 'NBL') drafted
October 2009: NBL is 'approved' (per page history; can't find approval ref)
October 2009: DOSL grows a link to NBL (clear that DOSL is the policy, NBL is 'more rationale')
January 2010: 'List' (at the time a single package) of approved exceptions added to NBL
October 2010: Policy for bundling exceptions added to NBL
October 2010: FPC ticket requesting clarification of 'unbundling' procedure: https://fedorahosted.org/fpc/ticket/19
October 2010: https://fedoraproject.org/wiki/Packaging:Treatment_Of_Bundled_Libraries (hereafter 'TOBL') drafted
December 2010: TOBL approved
December 2010: DOSL grows a link to TOBL (clear that TOBL's job is to explain how packagers should unbundle)
December 2010: Definition of 'library' added to DOSL; Javascript exception included
January 2012: BOMP grows a link to NBL: https://fedorahosted.org/fpc/ticket/131
 * NOTE: if we're working on the basis that BOMP is for 'multiple unrelated sources in a single package',
   this seems to be simply a mistake; the ticket reads as if those involved were acting on the belief
   that BOMP is the 'library bundling' policy. DOSL already *had* a link to NBL.
April 2012: 'Parallel stacks' explanation added to DOSL - https://fedorahosted.org/fpc/ticket/159

Unfortunately I can't find a lot of clear discussion of one of the
questions that interests me the most: the question of non-packaged
'libraries'. The best I can do is this. There's a passing line in an
FPC meeting in November 2011 - https://meetbot-raw.fedoraproject.org/te
ams/fpc/fpc.2010-11-17-16.07.log.html :

<tibbs> Personally I'd like to see things like a plan for resolving
the bundling, the attitude of the upstream of the bundled code and
input from the Fedora maintainer of the package which is being bundled
(if one exists).

no-one bats an eye at that 'if one exists', which at least strongly
implies that as far back as November 2010, FPC had a consensus that
things which weren't packaged could be considered to be 'bundled
libraries' for the purposes of DOSL. Then there is of course the
explicit line in TOBL:

"bundled libraries being defined as libraries which exist and are
mantained independently, whether or not they are packaged separately
for Fedora"

As near as I can tell, this line was present in TOBL from the very
first draft - https://fedoraproject.org/w/index.php?title=Packaging:Tre
atment_Of_Bundled_Libraries&oldid=201777 - and was not questioned or
discussed in any way in the various FPC and FESCo meetings around that
time. There's no express point where people look at it and say "yes,
that's OK" either, but at the very least it seems reasonable to assume
that the FPC folks of the time read the draft, saw that language, and
were OK with it.

So, my conclusions having dug through all this stuff:

1. The canonical definition of what constitutes 'library bundling' is
split between DOSL and the preamble of TOBL. It would be much clearer
if the bundled library definition from TOBL were moved into DOSL. It
would also help if TOBL linked back to DOSL.

2. The true purpose of TOBL should be 'explaining to packagers the
correct procedures for unbundling libraries'. All info on that topic
should go there, and nothing *else* should go there (see 1.)

3. The original purpose of NBL was to explain *why* the 'library
bundling' policy (i.e. DOSL) existed. It was not intended to be a
policy page. However, for a long time it has awkwardly served multiple
functions:

  i) explaining "why do we have DOSL?"
  ii) defining the policy on bundling exceptions
  iii) storing the list of granted exceptions

I suggest it would be much clearer if these three functions were
properly split up, and of course sensible links provided between all
relevant pages.

4. It seems fairly clear that BOMP was intended to mean, basically,
'don't take a bunch of tarballs from different places and stuff them
all into one package'. It was *not* intended to cover 'library
bundling' in any sense. I'd suggest that it should be clarified
somewhat - perhaps including an explicit internal link to DOSL - and
the link from BOMP to NBL should be *removed*, since it is not
appropriate there.

Does this make sense to folks? I'm willing to draft up the changes and
file an FPC ticket if so. I think any debate on what changes should be
made to the current policies would benefit from these changes to make
what the current policies actually *are* clearer, so I don't mind
doing it even if they all have to change again fairly soon.
-- 
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.ne