python-devel October 2018

python-devel@lists.fedoraproject.org

10 participants
6 discussions

Python 3.7's Deterministic pycs
by Petr Viktorin 15 Jul '19

15 Jul '19

Hello! The first beta for Python 3.7 is out. It will hopefully get into Fedora soon as python37. After it comes out of beta, we'll upgrade python3 to it. The What's New list is at: https://docs.python.org/3.7/whatsnew/3.7.html One thing that's interesting for packagers is PEP 552: Deterministic pycs: https://www.python.org/dev/peps/pep-0552/ Let me summarize in my own words. A new opt-in mode for byte-compilation makes .pyc (bytecode cache) files depend only on the contents of the corresponding source file. If we use this, it will slow down imports, because the whole source file would need to be read and hashed in order to verify if a .pyc file is valid. (Currently, metadata like the modification time and file size is used.) To speed things up, there's an option, UNCHECKED_HASH, which skips cache validation entirely. Using this would mean that if you modify a .py source file installed by RPM, the changes wouldn't take effect (the .py contents would only be shown in tracebacks). Modifying installed files in production is extremely bad practice, of course, but it's quite useful for debugging on throw-away systems. If we adopt UNCHECKED_HASH, anyone doing it will have to remember to remove the corresponding .pyc file. Honestly, I'm not sure we want to use this in Fedora. Is anyone here into reproducible builds, to make a better argument for this? -- Petr Viktorin

5 5

Moving EPEL7 to python3.6
by Stephen John Smoogen 23 Oct '18

23 Oct '18

Hi, EPEL is a set of packages which work for CentOS and RHEL versions 6 and 7. In the version 7, we are currently using python34 and would like to move this to python36. In doing so, we need help in both our packaging rules and in updating a lot of packages to work for python36. First problem: Packaging rules. Because there could be updates of python versions from 3.4 to 3.6 or 3.8, we wanted to make clear what python was used for any particular library. This would make it so someone needing python-bottle did not end up with one packaged with python-3.6 installed on a python-3.4 system. So we wanted the names to be more specific than python3 and went with naming all the sub packages python34 or python36. However, this was decided a while ago and it may not be the best convention to use or one that the current python sig would like us to use. I would like to get a naming convention cleared up and documented so when we do a mass rebuild that the packages come out with either a python3-<foo> or python36-<foo> Second problem: When to do this update We had been looking to do this in October, but it may make more sense to do this in November after Fedora29 has shipped so that people can help focus on this versus anything F29 related. It also gives us some lead time to write blogs/magazine items. How does 2018-11-14 sound? Third problem: Updating and rebuilding packages to work with python36 Below are the list of packages I found which were making python34-<libraryname> packages currently in EPEL-7. In updating to python36, I would like to have a combined Virtual Fedora Activity Day where we work together on IRC. First we would get any scripts ready and then work with release engineering to change macros in epel-macros to point to the correct versions of python and any name changes. We would then do a mass release bump and rebuild all the packages against python3.6. As problems are found during that day we would make appropriate changes and fix. This might take 2 gos. autowrap-0.16.0-1.el7.src.rpm clustershell-1.8-1.el7.src.rpm debconf-1.5.69-1.el7.src.rpm espresso-4.0.0-1.el7.src.rpm fedfind-4.2.0-1.el7.src.rpm future-0.16.0-6.el7.src.rpm jpype-0.6.3-3.el7.src.rpm lammps-20180822-1.el7.src.rpm lensfun-0.3.2-13.el7.src.rpm lhapdf-6.2.1-1.el7.src.rpm libprelude-4.1.0-2.el7.src.rpm libpreludedb-4.1.0-1.el7.src.rpm lxc-1.0.11-1.el7.src.rpm netcdf4-python-1.2.7-3.el7.src.rpm nordugrid-arc-5.4.2-9.el7.src.rpm petsc4py-3.9.1-3.el7.src.rpm prelude-correlator-4.1.1-3.el7.src.rpm py4j-0.10.7-3.el7.src.rpm pycmd-1.2-4.el7.src.rpm pyflakes-1.3.0-2.el7.src.rpm pylint-1.6.5-4.el7.src.rpm pythia8-8.2.35-4.el7.src.rpm python-PyGithub-1.39-1.el7.src.rpm python-PyMySQL-0.8.1-1.el7.src.rpm python-aiosmtpd-1.0-2.el7.src.rpm python-apsw-3.7.17.r1-3.el7.src.rpm python-arrow-0.8.0-3.el7.src.rpm python-astroid-1.4.9-2.el7.src.rpm python-atpublic-0.5-1.el7.src.rpm python-attrs-17.4.0-3.el7.src.rpm python-backports_abc-0.5-1.el7.src.rpm python-bitarray-0.8.3-1.el7.src.rpm python-blessed-1.14.1-2.el7.src.rpm python-bottle-0.12.13-1.el7.src.rpm python-breathe-4.2.0-3.el7.src.rpm python-cached_property-1.3.0-7.el7.src.rpm python-chai-1.1.1-4.el7.src.rpm python-click-6.7-6.el7.src.rpm python-clyent-1.2.2-2.el7.src.rpm python-collada-0.4-15.el7.src.rpm python-colorclass-2.2.0-2.el7.src.rpm python-contextlib2-0.5.1-2.el7.src.rpm python-cookies-2.2.1-6.el7.src.rpm python-cov-core-1.15.0-8.el7.src.rpm python-crypto-2.6.1-13.el7.src.rpm python-cytoolz-0.7.5-1.el7.src.rpm python-ddt-1.1.3-1.el7.src.rpm python-defusedxml-0.5.0-1.el7.src.rpm python-distutils-extra-2.39-7.el7.src.rpm python-dockerpty-0.4.1-9.el7.src.rpm python-docopt-0.6.2-7.el7.src.rpm python-easyargs-0.9.4-2.el7.src.rpm python-easygui-0.96-19.el7.src.rpm python-ecdsa-0.13-4.el7.src.rpm python-execnet-1.2.0-5.el7.src.rpm python-falcon-1.4.1-1.el7.src.rpm python-flexmock-0.10.2-4.el7.src.rpm python-flufl-bounce-2.3-3.el7.src.rpm python-flufl-i18n-1.1.3-3.el7.src.rpm python-flufl-lock-2.4.1-3.el7.src.rpm python-flufl-testing-0.4-1.el7.src.rpm python-freezegun-0.1.19-1.el7.src.rpm python-gammu-2.11-2.el7.src.rpm python-hexdump-3.4-0.2.20160818hg66325cb5fed8.el7.src.rpm python-hypothesis-3.12.0-1.el7.src.rpm python-idstools-0.6.3-1.el7.src.rpm python-ipython_genutils-0.1.0-7.el7.src.rpm python-iso8601-0.1.11-7.el7.src.rpm python-isort-4.2.5-8.el7.src.rpm python-ivi-0.14.9-6.el7.src.rpm python-jaydebeapi-1.1.1-1.el7.src.rpm python-jedi-0.10.2-3.el7.src.rpm python-jsonschema-2.5.1-3.el7.src.rpm python-jupyter-core-4.3.0-1.el7.src.rpm python-keyring-5.0-3.el7.src.rpm python-lazr-config-2.1-5.el7.src.rpm python-lazr-delegates-2.0.3-5.el7.src.rpm python-lazr-smtptest-2.0.3-6.el7.src.rpm python-lazy-object-proxy-1.2.2-1.el7.src.rpm python-libdiscid-0.4.1-11.el7.src.rpm python-llfuse-1.0-1.el7.src.rpm python-locket-0.2.0-2.el7.src.rpm python-lz4-0.8.2-1.el7.src.rpm python-markdown-2.4.1-2.el7.src.rpm python-maxminddb-1.4.0-2.el7.src.rpm python-mccabe-0.6.1-6.el7.src.rpm python-mimeparse-1.6.0-4.el7.src.rpm python-mock-1.0.1-9.el7.src.rpm python-msgpack-0.5.6-4.el7.src.rpm python-multilib-1.2-2.el7.src.rpm python-nbformat-4.4.0-1.el7.src.rpm python-neovim-0.2.6-2.el7.src.rpm python-nose2-0.6.5-4.el7.src.rpm python-numexpr-2.5.2-2.el7.src.rpm python-olefile-0.45.1-1.el7.src.rpm python-openqa_client-1.3.1-1.el7.src.rpm python-pamela-0.3.0-1.el7.src.rpm python-paramiko-2.1.1-0.4.el7.src.rpm python-parso-0.3.1-1.el7.src.rpm python-pathspec-0.5.3-1.el7.src.rpm python-pdfminer-20160614-4.el7.src.rpm python-pdir2-0.3.0-5.el7.src.rpm python-pg8000-1.10.5-1.el7.src.rpm python-pika-0.10.0-9.el7.src.rpm python-pip-8.1.2-6.el7.src.rpm python-pkgconfig-1.1.0-7.el7.src.rpm python-pluginlib-0.6.1-1.el7.src.rpm python-polib-1.0.7-9.el7.src.rpm python-pretend-1.0.8-3.el7.src.rpm python-process-tests-1.0.0-9.el7.src.rpm python-productmd-1.16-1.el7.src.rpm python-prompt_toolkit-1.0.15-1.el7.src.rpm python-psutil-2.2.1-3.el7.src.rpm python-ptyprocess-0.5.1-6.el7.src.rpm python-pycodestyle-2.0.0-5.el7.src.rpm python-pycosat-0.6.3-1.el7.src.rpm python-pyev-0.9.0-0.1.20130610gite31d137.el7.src.rpm python-pymilter-1.0.2-4.el7.src.rpm python-pymssql-2.1.3-2.el7.src.rpm python-pypandoc-1.3.3-2.el7.src.rpm python-pyroute2-0.4.13-1.el7.src.rpm python-pyside-1.2.4-1.el7.src.rpm python-pysocks-1.6.7-1.el7.src.rpm python-pytest-catchlog-1.2.2-2.el7.src.rpm python-pytest-runner-3.0-1.el7.src.rpm python-pytest-xdist-1.17.1-2.el7.src.rpm python-pytoml-0.1.18-1.el7.src.rpm python-pyxs-0.4.1-1.el7.src.rpm python-regex-2018.07.11-1.el7.src.rpm python-rencode-1.0.5-3.el7.src.rpm python-responses-0.5.1-6.el7.src.rpm python-rope-0.11.0-1.el7.src.rpm python-ruamel-yaml-0.13.14-1.el7.src.rpm python-scandir-1.8-1.el7.src.rpm python-scripttest-1.3.0-9.el7.src.rpm python-setuptools_scm-1.10.1-2.el7.src.rpm python-sh-1.12.14-6.el7.src.rpm python-simplejson-3.10.0-1.el7.src.rpm python-simplepath-0.3.4-1.el7.src.rpm python-sphinx-autobuild-0.7.1-1.el7.src.rpm python-sphinx-theme-alabaster-0.7.9-1.el7.src.rpm python-spur-0.3.17-1.el7.src.rpm python-structlog-17.2.0-1.el7.src.rpm python-terminado-0.6-2.el7.src.rpm python-texttable-1.4.0-1.el7.src.rpm python-tinydb-3.7.0-1.el7.src.rpm python-toml-0.9.4-3.el7.src.rpm python-toolz-0.7.4-1.el7.src.rpm python-traitlets-4.3.2-2.el7.src.rpm python-trololio-1.0-1.el7.src.rpm python-typing-3.5.2.2-3.el7.src.rpm python-unidiff-0.5.4-1.el7.src.rpm python-vxi11-0.9-1.el7.src.rpm python-wcwidth-0.1.7-2.el7.src.rpm python-whoosh-2.7.4-3.el7.src.rpm python-wrapt-1.10.4-7.el7.src.rpm python-www-authenticate-0.9.2-3.el7.src.rpm python-xapp-1.0.1-7.el7.src.rpm python-xlrd-1.0.0-6.el7.src.rpm python-yara-3.8.1-2.el7.src.rpm python-zope-component-4.1.0-3.el7.src.rpm python-zope-configuration-4.0.3-11.el7.src.rpm python-zope-exceptions-4.0.5-4.el7.src.rpm python-zope-i18nmessageid-4.0.3-2.el7.src.rpm python-zope-schema-4.4.1-2.el7.src.rpm python-zope-testing-4.1.2-2.el7.src.rpm python3-Cython-0.23.5-1.el7.src.rpm python3-PyYAML-3.11-2.el7.src.rpm python3-backports-ssl_match_hostname-3.5.0.1-1.el7.src.rpm python3-cairo-1.10.0-24.el7.src.rpm python3-cffi-1.9.1-2.el7.src.rpm python3-chardet-2.3.0-4.el7.src.rpm python3-coverage-4.0.3-4.el7.src.rpm python3-cryptography-1.7.2-4.el7.src.rpm python3-cryptography-vectors-1.7.2-3.el7.src.rpm python3-dateutil-2.4.2-3.el7.src.rpm python3-dbus-1.2.4-2.el7.src.rpm python3-decorator-4.0.11-1.el7.src.rpm python3-dns-1.15.0-7.el7.src.rpm python3-docker-2.6.1-1.el7.src.rpm python3-docker-pycreds-0.2.1-1.el7.src.rpm python3-docutils-0.12-0.8.20140510svn7747.el7.src.rpm python3-gobject3-3.22.0-4.el7.src.rpm python3-greenlet-0.4.12-1.el7.src.rpm python3-gssapi-1.3.0-1.el7.src.rpm python3-idna-2.7-1.el7.src.rpm python3-jinja2-2.8-2.el7.src.rpm python3-jwt-1.6.4-1.el7.src.rpm python3-kitchen-1.2.5-1.el7.src.rpm python3-markupsafe-0.23-1.el7.src.rpm python3-mysql-1.3.12-1.el7.src.rpm python3-netaddr-0.7.18-2.el7.src.rpm python3-nose-1.3.7-3.el7.src.rpm python3-numpy-1.10.4-6.el7.src.rpm python3-ply-3.9-1.el7.src.rpm python3-py-1.4.30-2.el7.src.rpm python3-pyasn1-0.1.9-1.el7.src.rpm python3-pycparser-2.14-1.el7.src.rpm python3-pycurl-7.43.0-5.el7.src.rpm python3-pygments-2.2.0-1.el7.src.rpm python3-pytest-2.8.5-2.el7.src.rpm python3-pytest-cov-2.5.1-1.el7.src.rpm python3-pytz-2017.2-1.el7.src.rpm python3-pyusb-1.0.2-2.el7.src.rpm python3-requests-2.12.5-1.el7.src.rpm python3-scipy-0.18.1-1.el7.src.rpm python3-setproctitle-1.1.6-2.el7.src.rpm python3-setuptools-19.6.2-3.el7.src.rpm python3-simplepam-0.1.5-6.el7.src.rpm python3-six-1.11.0-1.el7.src.rpm python3-sqlalchemy-1.1.3-1.el7.src.rpm python3-suds-0.6-1.el7.src.rpm python3-tornado-4.4.2-1.el7.src.rpm python3-urllib3-1.19.1-3.el7.src.rpm python3-virtualenv-15.1.0-1.el7.src.rpm python3-websocket-client-0.47.0-1.el7.src.rpm python3-zope-event-4.2.0-1.el7.src.rpm python3-zope-interface-4.3.3-1.el7.src.rpm python34-3.4.9-1.el7.src.rpm pyxdg-0.25-6.el7.src.rpm root-6.14.04-1.el7.src.rpm scapy-2.4.0-2.el7.src.rpm spglib-1.10.3-1.el7.src.rpm uwsgi-2.0.17.1-1.el7.src.rpm xapps-1.0.4-12.el7.src.rpm xrootd-4.8.4-1.el7.src.rpm yubikey-piv-manager-1.4.2-5.el7.src.rpm -- Stephen J Smoogen.

5 5

EPEL: Python 3.4 will be EOL in March 2019
by Miro Hrončok 22 Oct '18

22 Oct '18

On 13.8.2018 11:49, Larry Hastings wrote on python-dev(a)python.org: > We of the core dev community commit to supporting Python releases for > five years. Releases get eighteen months of active bug fixes, followed > by three and a half years of security fixes. Python 3.4 turns 5 next > March--at which point we'll stop supporting it, and I'll retire as 3.4 > release manager. > > My plan is to make one final release on or around its fifth birthday > containing the last round of security fixes. That's about seven > months from now. See also PEP 429 -- Python 3.4 Release Schedule [0]. We have python34 and python36 in EPEL7. python34 being the "main" one and python36 the "other" one. The original plan [1] was that once the ecosystem is adapted well enough, we can switch what "main" and "other" is and eventually drop python34, creating room for maybe another python3 release to be the "other" next (python38 maybe?). Originally this was supposed to happen for each release [2], but this was later abandoned due to lack of manpower. Do we want to ever switch "main" with "other"? The problem here is: $ dnf repoquery --repo=epel7 --whatrequires 'python(abi) = 3.4' | pkgname | sort | uniq | wc -l 243 $ dnf repoquery --repo=epel7 --whatrequires 'python(abi) = 3.6' | pkgname | sort | uniq | wc -l 15 The original idea is that all packages would have the python3_other macros in them [2] and all we need is rebuild everything in proper order, maybe with some bootstrapping. However that never happened and most of the packages I've seen lack the python3_other bits (no statistics, just my impression). Is this something we want? If so, are the packagers willing to adapt their packages (as much as I'd like to do this, I lack the resources to hack on 228 packages)? The Python Maint team in Red Hat is here to help. However the drive would need to come from the EPEL community, as now we are only guessing what are the needs here. [0] https://www.python.org/dev/peps/pep-0429/ [1] https://lists.fedoraproject.org/archives/list/epel-devel@lists.fedoraprojec… [2] https://fedoraproject.org/wiki/User:Bkabrda/EPEL7_Python3 -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 2

test files in a python flat package
by Jun Aruga 17 Oct '18

17 Oct '18

I noticed that the test files (/usr/lib64/python3.7/test) for python37 are occupying more than 50% in the package size, when I was cleaning the files in my local machine. If we actually do not use the test files, I do not want to include the files in the python37 package. Just want to separate the files as python37-test sub package. Because of just reducing the file size. How do you think? I want to hear your opinions. Here is the bugzilla ticket. Separate the test directory as python37-test https://bugzilla.redhat.com/show_bug.cgi?id=1637959 -- Jun Aruga jaruga(a)redhat.com

2 1

Mass Python 2 packages Removal
by Miro Hrončok 15 Oct '18

15 Oct '18

Hi, this is about: https://fedoraproject.org/wiki/Changes/Mass_Python_2_Package_Removal We have ~300 bugz open and blocking https://bugzilla.redhat.com/show_bug.cgi?id=PY2REMOVAL Some of them are > 1 week old and hence anyone can go and remove the mentioned (sub)packages. I've retired the packages that were for retirement and I will retire more once they are unblocked (needs to go to compose -> portingdb before the automation recognizes it). I open more bugs as they are closed (I don't wan to deal with 1000 bugs at once). Retirement is easy. The problematic part is removing subpackages. I go and manually choose the packages that appear not to be just simple "py spec generates py2 and py3 subpackages" combo, but a bit more complicated, and I leave the "simple" ones alone for now. I'm theoretically able to remove a simple py2 subpackage in about a minute, yet it is tedious and after ~20 packages I need to stop (before I go crazy). We have something in between 900 and 2000 packages to remove (there are 900 leaves to remove and we may make another 1100 into leaves as we go). Is anyone willing to look into automating this? Write a spec depython2izer? This usually means: - removing python2- and python- BRs - removing %package -n python2 and it's metadata, description & %files - removing all %py2_build, %py2_install or their older manual variants - removing python2 tests from %check - (sometimes) removing %py3dir or python3 dir pushd/popd dance - (sometimes) removing with_python3 conditionals as they make no sense IIRC Zbyszek said at Flock that he might work on that. Is that still true? If not, shall I draft something? -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 8

Elliott Sales de Andrade (qulogic) is now in python{,-packagers}-sig
by Miro Hrončok 06 Oct '18

06 Oct '18

Welcome \o/ https://fedoraproject.org/wiki/User:Qulogic -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

2 1

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel October 2018