Hi Pythonistas and Fedora packagers.
Recently I've realized we bundle too much wheels with our pythons +
virtualenv package. That is unfortunate:
* we don't build those. stricly seeking, it's just a zip with python
files in it, yet this is not permitted in Fedora
* we only sometimes list it as Provides: bundled(...) and when we do,
it is tedious
* we never list bundled deps in those bundled wheels (pip bundles a lot)
* we never adapt the license tag to include license of bundled wheels
(and bundled deps in those) - it would be even more tedious, as pip
License tag can be very complicated
* we duplicate those across packages
I went ahead and prepared a concept in:
This is one package that builds all the required wheels. It might be a
bit weird that it's only one package, but I think it can lower the
maintenance burden. Also, we won't update any wheel package, we only add
or remove them, so there is no "life cycle". Later we can decide that
there are simply too many thing sin one package and split it).
This package makes sure the license tag is right and all the virtual
bundled provides are in place.
Even as one package, I think it's a big improvement comparing to current
state of things.
Could you please review the decisions made in the spec? Namely:
* naming (main package, subpackages)
* virtual provides
* that the spec is generated by a script and how that script works
* the method of usage described in the package review request
I've also decided not to run tests, as for them to mean something, we
would need to run them against all relevant Python versions. Also, it
would complicate the package a lot.
The package is approved thanks to Robert-André Mauchin, yet I won't
request the repo until it's settled that we want this.
Also, once we start using this, maybe we can stop doing rewheel and just
use wheels from here in the python3 package as well?
I've just built python-pip-18.0-1.fc29.
It has a lot of breaking changes, so please file bugs if you have
problems. We do not plan to upgrade pip in stable Fedora releases.
Release notes: https://pip.pypa.io/en/stable/news/
See notes for 18 and 10, as we skipped 10.
Note that upstream changed version scheme, nothing existed between 10
Facebook just published an interesting bit of tech that uses squashfs
to create single-file executables that don't require a pre-installed
container engine to execute, but do place some more significant
demands on the FUSE subsystem, and also require a bootstrapping
executable to set up the filesystem mounts.
I recommend taking a look at their blog post, as xar seems like quite
an interesting capability to consider offering: whereas containers and
flatpaks give you both build time and runtime component isolation, xar
files aim to behave more like a statically linked ELF binary with no
supporting data files (no runtime isolation, but no direct
dependencies on the host filesystem either).
---------- Forwarded message ----------
From: Cooper Ry Lees <me(a)cooperlees.com>
Date: 14 July 2018 at 06:51
Subject: [Distutils]Introducing XAR - SquashFS based mountable
executables - Calling OS/Distro Maintainers
Today Facebook Open Sourced a competitor to PEX and other zip based
distribution methods for Python (and potentially other languages).
Basically it's claim to fame is start up time for large modules being
similar to regular on file system modules due to extracting on read
via SquashFS mounted executables. For more information read our blog
The bdist_xar 'wheel' like plugin is also now in PyPI:
Main reason I wanted to post here was I'd love to reach out to our OS
maintainers on list (e.g. Mr Stinner) and talk about getting the
components of XAR into the OS's package repos. The main components
that make sense are:
- xarexec_fuse: https://github.com/facebookincubator/xar/
- squashfuse: a newer version with squashfuse_ll (and optionally zstd
- squashfs-tools: a newer version with zstd support (made optional I guess)
Once we had this, it would make building XARs as easy as building
wheels, even more so when we can define OS level dependencies for it!
Please feel free to reach out to me directly with any questions etc. -
Also want to note, Sumanah suggested I floated this here, I apologize
if this is a misuse of the list. If so, please ignore. I was torn
wether or not this is appropriate.
Distutils-SIG mailing list -- distutils-sig(a)python.org
To unsubscribe send an email to distutils-sig-leave(a)python.org
Message archived at
Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia
The merge of the f29-python side tag into rawhide has started.
Expect Python 3.7 in rawhide soon.
If your package will have broken dependencies (on Python 3.6), rebuild it.
If you cannot rebuild it due to dependencies not being rebuilt, (search
and) open bugs.
Make sure to block PYTHON37  so we are aware of all the issues.
During the 3.7 boostrapping of the interstitial sequence of Python 3
packages, I have noticed it includes a lot of packages that are only
intended as Python 3 stdlib backports for older Pythons.
https://pypi.org/project/contextlib2/ is a backport of the standard
library’s contextlib module to earlier Python versions
https://pypi.org/project/linecache2/ is a backport of linecache to older
https://pypi.org/project/pathlib2/ The goal of pathlib2 is to provide a
backport of standard pathlib module which tracks the standard library
module, so all the newest features of the standard pathlib can be used
also on older Python versions.
https://pypi.org/project/traceback2/ is a backport of traceback to older
https://pypi.org/project/unittest2/ is a backport of the new features
added to the unittest testing framework in Python 2.7 and onwards.
https://pypi.org/project/simplejson/ is the externally maintained
development version of the json library included with Python 2.6 and
Python 3.0, but maintains backwards compatibility with Python 2.5.
https://pypi.org/project/funcsigs/ is a backport of the PEP 362 function
signature features from Python 3.3
https://pypi.org/project/mock/ is now part of the Python standard
library, available as unittest.mock in Python 3.3 onwards.
Now I see fairly no real reason to have python3-contextlib2,
python3-linecache2 etc. packaged and maintained in Fedora. The only
reason there is that other packages actually depend on them.
I think we should actively change the packages not to depend on those
and eventually get rid of them. This involves changing how upstreams are
using those. Thoughts?
Here is some dependency info:
$ whatrequires python3-contextlib2
$ whatrequires python3-linecache2
$ whatrequires python3-pathlib2
$ whatrequires python3-traceback2
$ whatrequires python3-unittest2
$ whatrequires python3-simplejson
$ whatrequires python3-funcsigs
$ whatrequires python3-mock
...320 source packages buildrequire this one!