Proposal to reduce anti-bundling requirements

[Kevin Kofler]

I don't have the time to read the whole thread right now (and I'm not sure I 
ever will), but I think this is important, so here are my comments to the 
original proposal (to which I am totally opposed!):

Stephen Gallagher wrote:
> Right now, we have a policy that essentially forbids source code from
> being bundled into a package. In technical terms, this means
> essentially that the packaging policies mandate that any code that
> appears more than once in the repository must be turned into a shared
> library and dynamically linked into any package that requires it.

And that's a good thing.

> I would like to propose that the no-bundled-libraries policy be
> amended  as follows: "Any package that has an existing mechanism to
> link against a shared system library and functions correctly when
> doing so must link against that library and not bundle it internally.
> Any package whose upstream releases cannot link against a shared
> system library (or are incompatible with the version in Fedora) may
> bundle that library (without requiring a special exemption) but MUST
> add Provides: bundled(<libname>) = <version> in the spec file for each
> known bundled library.(This will allow us to track down the bundling
> when we need to). Package maintainers should continue attempt to
> engage upstream to support linking against shared system libraries
> wherever possible, due to the advantages it provides the package
> maintainer."

This will lead to package maintainers not even TRYING to unbundle libraries. 
But that is really what the packager is for. Instead, what we see right now 
is packagers just running to FPC for an exception, and if they don't get it, 
the package gets threatened of removal from Fedora if it is already in, or 
gets stuck under review forever otherwise. With your proposed policy, it 
would just bundle everything with no attempt whatsoever at fixing that. We 
need more proactive downstream unbundling. We have to realize that upstream 
will not care and that is is our job to fix their mess.

If the library is incompatible, there are generally 2 approaches to solving 
the problem: patch the application to not need the incompatible library 
additions, or patch the library to add them. Which one is more workable is a 
case by case decision. (In particular, we definitely need library 
maintainers in Fedora to be more flexible when it comes to adding non-
upstream patches to make applications work. "Upstream, upstream, upstream" 
does not always work, it should not be a dogma.) If the application simply 
needs a too new version of the library, then often this is a recent change 
in the application, so simply reverting the commits that need the new 
library is an effective solution. (In some cases, that also works for non-
upstream library changes: revert the application commit that started using 
them.)

> The reason for this proposal is relatively simple: we know the
> advantages to unbundling, particularly with security and resource-
> usage. However, the world's developer community largely *does not
> care*. We fought the good fight, we tried to bring people around to
> seeing our reasoning and we failed.

See above. Upstream will not care. Some of them never did. (This is not a 
new problem.) It is OUR job to unbundle it in downstream patches. If 
upstream accepts the patches, all the better. If they don't, we need to keep 
the downstream patches.

> The point of software is to provide a service to an end-user. Users
> don't run software because it has good packaging policies, they run
> software because it meets a need that they have. If they can't get
> that software from Fedora, they *will* get it from another source (or
> use a different OS that doesn't get in their way). I'll take a moment
> to remind people that two of Fedora's Four Foundations are "Features"
> and "First". We want Fedora to be the most feature-complete
> distribution available and we want to get there before anyone else
> does. I would say that holding to our no-bundling policy actively
> defeats our efforts on that score.

That really means our packagers are failing to do their job.

> == Advantages to bundling ==
>  * Guarantees that the application is running with the same set of
> code that upstream tested. (Fewer Fedora-specific bugs means less
> burden on the maintainer)

But multiplies the number of library versions in a given Fedora release by 
the number of applications! In addition to the obvious space and security 
issues, this is also a debugging/testing nightmare!

>  * Simplifies packaging of updates. (Fedora maintainer does not need
> to keep tabs on unbundling patches to keep in sync for new versions)

Not really. The maintainer then has to keep track of things such as new 
BuildRequires (and file list changes if there are more files than just one 
static library that's linked directly into the binary and not installed) for 
each and every bundled library.

Running into FTBFS bugs (which can sometimes happen on package updates, but 
most commonly, it happens when something else in the distribution gets 
updated) is also all the more likely the more libraries are bundled.

>  * Increases the available pool of software that can be packaged
> substantially (many modern languages such as Ruby and Go are
> realistically only functional with allowable bundling)

We need proper packaging guidelines that address that properly. The Go 
guidelines (distributing libraries in source form) are a joke! Go software 
should only be allowed with gcc-go shared libraries, and all libraries 
should then be packaged separately as shared libraries.

We need to ignore the language's own wannabe package management entirely and 
patch the build systems (hopefully centrally-installed ones – if it's 
autotools-style copypasta, then we need to patch each and every package, 
sadly) to work with system packages.

And for languages like Java where libraries get bundled as binary blobs, I 
think allowing that kind of bundling would be totally unacceptable and would 
go against our commitment to freedom.

>  * Did I mention the reduction in maintainer burden yet?

It's less burden initially, but fighting with updated build dependencies for 
the bundled libraries, FTBFS issues in the bundled libraries, etc. may well 
turn out to be MORE work. And unbundling delivers much better quality 
packages (disk space consumption, RAM consumption, timeliness of bug and 
security fixes), for which doing more work (if it is actually more work) is 
warranted.

        Kevin Kofler