Python libraries and backwards compat [was Re: What would it take to make Software Collections work in Fedora?]

[Mark McLoughlin]

On Mon, 2013-03-04 at 22:51 +0000, Mark McLoughlin wrote:

(following up with more thoughts from the distutils-sig thread)

> It started here:
> 
>   http://mail.python.org/pipermail/distutils-sig/2013-February/020030.html
> 
> and now we're talking about Software Collections here:
> 
>   http://mail.python.org/pipermail/distutils-sig/2013-March/020074.html
> 
> Two things I'm picking up from the thread:
> 
>   - A trend towards "semantic versioning" and, implicit in that, an 
>     acceptance of API breakages so long as the major number of a library
>     version is incremented
> 
>   - Supporting the parallel installation of incompatible versions of 
>     libraries isn't seen as an issue because you can "just use virtual 
>     environments" ... which amounts to Python Software Collections.

I think this parallel installs issue is the key thing we (Fedora,
OpenStack, etc.) need to help figure out.

There will be incompatible updates to libraries and, like with e.g. gtk2
and gtk3, I think we'll need to support apps using different versions of
the same library at the same time.

Right now, Software Collections is the only way I can see that OpenStack
can be packaged so we don't get screwed when an incompatible library
update comes along.

Nick Coghlan laid out some ideas for what could be done for Python which
I summarised as:

  http://mail.python.org/pipermail/distutils-sig/2013-March/020082.html

  - the system has multiple versions of somedep installed under /usr 
    somewhere

  - the latest version (2.1) is what you get if you naively just do 
    'import somedep'

  - most applications should instead somehow explicitly say they need
    ~>1.3, ->1.6 or ~->2.0 or whatever

  - distros would have multiple copies of the same library, but only 
    one copy for each incompatible stream rather than one copy for each 
    application

This is a much happier prospect than either Software Collections or no
support for parallel installs. It needs help to make it happen, though!

Cheers,
Mark.