October 2016 - python-devel - Fedora Mailing-Lists

Python 3.6 and blocking-by-default os.urandom()

by Nick Coghlan

Hi folks, The long discussions upstream regarding the default behaviour of Python's os.urandom() API in Python 3.6 have come to an end, with Guido opting to make os.urandom() block implicitly waiting for the system RNG to be ready, even on Linux (where /dev/urandom doesn't do that). While I can see his point from a cross-platform language runtime design perspective, I'm still not sure it's the right answer for the Fedora system Python, as it doesn't play nice with ABRT, and goes against Ted T'so's decision to keep /dev/urandom non-blocking at the kernel level (risking confusion for folks that are Linux developers & operators first, and Pythonistas second). If "os.urandom() was called before system RNG is ready" throws an uncaught BlockingIOError, then we'll get a nice ABRT-friendly Python stack trace for people to debug (and perhaps figure out their VM entropy pool isn't being seeded properly, or their ARM hardware design needs a better source of entropy). By contrast, if it just blocks, then folks will be faced with a system hang, which they'll need to trace back to CPython being blocked on a kernel getrandom() call, and then infer from that that the system RNG isn't ready (for whatever reason), and they should probably do something about that. Patching this behaviour should be relatively straightforward - Python 3.5 currently makes this call in non-blocking mode and falls back to reading /dev/urandom directly in that case, so we'd just be forward porting the same logic to 3.6 and raising an exception instead of falling back to the file descriptor. Accordingly, what I propose we do is as follows: 1. Raise the concern in the F26 system-wide change proposal for migrating to Python 3.6 2. Apply the patch when the 3.6 beta releases are added to Fedora Rawhide 3. Decide whether or not to keep the patch based on ABRT results and other feedback on the Rawhide builds. Note that if the feedback on Rawhide shows that the change is helping people to find and diagnose VMs and hardware with improperly seeded entropy pools, that's a *good* thing: this proposed change is replacing Python 3.5's "/dev/urandom and os.urandom() may silently return statistically less-than-fully-random random numbers if the kernel entropy pool isn't seeded properly" with "os.urandom() will fail noisily in those cases, so you can either switch to the random module, or fix your entropy pool seeding". Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

6 years, 10 months

4
9
0 / 0

Python SIG on-boarding issue

by Dhanesh B. Sabane

CommOps is currently working on the Python SIG On-Boarding ticket [1] and one of the tasks we have identified is re-writing the Python SIG wiki page to make it more beginner-friendly. I was assigned with this particular task which is now complete [2] [3]. We received some feedback from mhroncok today during the CommOps meeting and following points were discussed: 1. Splitting Python SIG in two FAS groups: 1.1 *python-sig* for newcomers and interested members who hang out and help with Python on Fedora 1.2 *python-packagers* for proven members of the community / packagers who will also receive security-related bugs. Promising members from *python-sig* group can be promoted to *python-packagers* given that they have contributed to the package's git. 2. Python Features section on the wiki page needs some love. 3. Targeting specific information for newcomers to include in their self-introductions that would be helpful for SIG members to know where someone fits in the Python community and what their expertise is. We would like to hear your feedback on the draft and also on the aforementioned points. Everyone is welcome to leave a comment on the ticket[1]. [1] https://pagure.io/fedora-commops/issue/84 [2] https://fedoraproject.org/wiki/User:Dhanesh95/SIGs/Python [3] https://fedoraproject.org/wiki/User:Dhanesh95/SIGs/Python/Join --- Dhanesh B. Sabane

6 years, 11 months

6
11
0 / 0

Python 3.6 beta release in F26 Rawhide?

by Nick Coghlan

Hi folks, During the discussion of the os.urandom() change in Python 3.6 (the successor to the accidental change in Python 3.5 that is the subject of Tomas's emailing about rebasing to 3.5.2 in F24), we came to the conclusion it would be good to get 3.6 into Rawhide early enough to impact the upstream beta cycle: https://lists.fedorahosted.org/archives/list/python-devel@lists.fedorapro... However, there's no reference to that discussion in https://fedoraproject.org/wiki/Changes/Python3.6 - it only refers to incorporating a release candidate or the actual release, which will be too late for us to request changes to the upstream default os.urandom() behaviour if we find unexpected problems with it. With Python 3.6b2 recently pushed out the door, there's about two months (including two more beta releases) until the first release candidate in early December. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

7 years

3
14
0 / 0

Python packages tests directory co-ownership

by Athos Ribeiro

tl;dr: different packages own /usr/lib/pythonX/site-packages/tests and different files with the same name inside that directory. Hi all, I was going through a package review and realized the package under review owns /usr/lib/pythonX/site-packages/tests and some files inside the directory. since there were no subdirectories and there were no namespaces for the file names inside the directory, I went further and realized that the following packages are doing the same: python3-custodia-0:0.1.0-3.fc24.noarch python3-django-federated-login-0:1.0.0-9.fc24.noarch python3-journal-brief-0:1.1.3-3.fc24.noarch python3-oauth2-0:1.9.0-2.post1.fc24.noarch python3-repoze-who-plugins-sa-0:1.0.1-10.20160106gite1a36c5.fc24.noarch python-custodia-0:0.1.0-3.fc24.noarch python-django-federated-login-0:1.0.0-9.fc24.noarch python-libturpial-0:1.7.0-4.fc24.noarch python-repoze-who-plugins-sa-0:1.0.1-10.20160106gite1a36c5.fc24.noarch what's the problem with that? Well what if we would have a package named python-tests? shouldn't it own that directory? Also, since there are no namespaces inside that directory for each package owning files in there, they could end up owning different files with the same name. Actually, the following packages own /usr/lib/python3.5/site-packages/tests/__init__.py python3-custodia-0:0.1.0-3.fc24.noarch python3-django-federated-login-0:1.0.0-9.fc24.noarch python3-oauth2-0:1.9.0-2.post1.fc24.noarch python3-repoze-who-plugins-sa-0:1.0.1-10.20160106gite1a36c5.fc24.noarch or /usr/lib/python2.7/site-packages/tests/__init__.py python-custodia-0:0.1.0-3.fc24.noarch python-django-federated-login-0:1.0.0-9.fc24.noarch python-libturpial-0:1.7.0-4.fc24.noarch python-repoze-who-plugins-sa-0:1.0.1-10.20160106gite1a36c5.fc24.noarch Is that all really a problem or am I missing something in the guidelines? -- Athos Ribeiro http://www.ime.usp.br/~athoscr

7 years

2
1
0 / 0

python-setuptools explicit build dependency

by Germano Massullo

In spec files of Python based software, python-setuptools is a build requirement that is mandatory for EPEL7, on Fedora instead it is not required because it should already be a runtime dependency for python-devel. By the way I remember I have heard that in next future it should be added explicitly for various reasons. Since I am actually doing some edits on the spec files of all Python packages I mantain, I would like to know if what I have heard is true, so that I can add it such dependency now. Thank you

7 years, 1 month

2
1
0 / 0

%{python3_pkgversion} tag

by Germano Massullo

A proven packager edited one of the python libraries I maintain, to add some useful stuff. Among all the edits he made, there is one in particular that made me write this message. %{?python_provide:%python_provide python3-%{pypi_name}} has been replaced with %{?python_provide:%python_provide python%{python3_pkgversion}-%{pypi_name}} What is the advantage of using %{python3_pkgversion} tag? Have in future a package name like python31-foo ? Till now I have only seen names like python2-foo and python3-foo Thank you

7 years, 1 month

2
2
0 / 0

scikit-learn in Fedora 25 ?

by Tom Devel

I have used scikit-learn in several previous Fedora releases, however if I look at https://bodhi.fedoraproject.org/updates/?packages=python-scikit-learn there is no scikit-learn for Fedora 25 anymore. The latest stable version is 0.18.1 ( http://scikit-learn.org/dev/whats_new.html), this is a popular machine learning library. Will scikit-learn be included Fedora 25? Or am I searching incorrectly for it? Tom

7 years, 1 month

2
2
0 / 0

PEP: Distributing a Subset of the Standard Library

by Tomas Orsava

Hi! We have prepared a PEP that aims to standardize and improve what Fedora and other distributions have been doing for a while, that is splitting parts of Python's standard library to separate optional packages. Feedback is welcome: https://fedora-python.github.io/pep-drafts/pep-A.html Abstract Python is sometimes being distributed without its full standard library. However, there is as of yet no standardized way of dealing with importing a missing standard library module. This PEP proposes a mechanism for identifying which standard library modules are missing and puts forth a method of how attempts to import a missing standard library module should be handled. I'm leaving for PTO lasting till November 17th, after which we want to send this to the PEP editors upstream. Yours aye, Tomas Orsava

7 years, 1 month

1
0
0 / 0

python34 for epel6 ?

by Tim Orling

I came across a need for python34 on epel6. I did not see any BZ for it. Are there significant issues with packaging python34 for epel6? or just nobody wanted to do it yet? Cheers. --Tim P.S Thank you for python34 on epel7. worked like a charm. P.P.S. I realize rh-python34 is available on RHSCL, but it didn't seem to support "python3" in scripts...

7 years, 1 month

3
8
0 / 0

Self introduction - Fale

by Fabio Alessandro Locati

Hi, I've been packaging python packages for quite some time and since one month ago I also joined this mailing list, mainly because I was interested in reading it. I think that it would make sense to present myself and maybe help out more. I'm mainly interested in python packaging and ATM I'm Poc of 11 python packages and co-maintainer of other 9. Best, Fale -- Fabio Alessandro Locati Red Hat - Senior Consultant PGP Fingerprint: E815 3C49 2A8D FD8B 1CBD BC85 FDB3 DF20 B2DC 9C1B

7 years, 1 month

2
1
0 / 0

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

python-devel October 2016