I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
I can see the reason about packaging architecture specific packages, since they will benefit of optimizations and security flags that Fedora uses, but noarch packages are simply the source code that gets compiled at runtime by the Python interpreter, aren't they?
In what way is different from installing them by pip? Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
Mattia
On 08. 03. 21 18:33, Mattia Verga via devel wrote:
I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
You can use them as requirements for packaged applications.
I can see the reason about packaging architecture specific packages, since they will benefit of optimizations and security flags that Fedora uses, but noarch packages are simply the source code that gets compiled at runtime by the Python interpreter, aren't they?
In what way is different from installing them by pip? Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
See above. If they are required by something, it is necessary. Packaging leaf libraries as RPMs however indeed brings very little benefit.
On Mon, 2021-03-08 at 19:07 +0100, Miro Hrončok wrote:
On 08. 03. 21 18:33, Mattia Verga via devel wrote:
I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
You can use them as requirements for packaged applications.
I can see the reason about packaging architecture specific packages, since they will benefit of optimizations and security flags that Fedora uses, but noarch packages are simply the source code that gets compiled at runtime by the Python interpreter, aren't they?
In what way is different from installing them by pip? Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
See above. If they are required by something, it is necessary. Packaging leaf libraries as RPMs however indeed brings very little benefit.
by packaging you can avoid some of the mess that is seen https://github.com/pavoni/pywemo/issues/253 pip does not handle some use cases, rpm/dnf would throw an error and not let you shoot yourself in the foot
Denis
On 08.03.2021 18:33, Mattia Verga via devel wrote:
I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
1. Can be used in packaging of another packages. 2. Can be installed only once. Saves lots of disk space. 3. Can be easily updated to fix security vulnerabilities.
In what way is different from installing them by pip?
Using pip outside of the VENV/user_profile may destroy your system.
Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
Yes.
On Mon, 8 Mar 2021 at 12:34, Mattia Verga via devel < devel@lists.fedoraproject.org> wrote:
I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
I can see the reason about packaging architecture specific packages, since they will benefit of optimizations and security flags that Fedora uses, but noarch packages are simply the source code that gets compiled at runtime by the Python interpreter, aren't they?
In what way is different from installing them by pip? Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
The needs for packaging software usually come up as the following: 1. A lot of different installed packages like say fedpkg, koji, etc will need these packages either to run or to build itself. [A lot of these are things needed to build something else.] 2. Having those as known packages quantities allows others to test and duplicate what was used to build the OS. 3. Having those as known packages allows for people to avoid certain supply chain problems where the package is 'gone/unavailable' for whatever reason. 4. Bundling the versions means that you end up with a lot of unaudited duplication for a software set which is 'central' to the building, testing, and operations of Fedora (versus nodejs, ruby, perl, lua, rust, etc). 5. When problems do occur, there is an audit trail to see when the problem noarch entered the system, how long it was there, and what was known to use it.
The above doesn't mean that the operating system needs to do so.. or that the above solve the problems listed well. However they are some of the reasons why.
Mattia
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
Mattia Verga via devel wrote:
I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
[...]
In what way is different from installing them by pip?
· Users can install and use programs without caring about what programming language they are written in.
· Programs can depend on other programs written in other languages.
· Users don't need to run pip to check for Python program updates, cpan to check for Perl program updates, npm to check for Javascript program updates, gem to check for Ruby program updates, and so on and so forth. They can get all their updates with a single "yum update".
· It's easy to set traps on PyPI that trick users into downloading malware. I've never heard about any such problem in the Fedora repository. https://arstechnica.com/information-technology/2016/06/college-student-schoo... https://arstechnica.com/information-technology/2017/09/devs-unknowingly-use-... https://arstechnica.com/information-technology/2018/10/two-new-supply-chain-... https://arstechnica.com/information-technology/2021/02/supply-chain-attack-t...
Björn Persson
Same thing for all those drupal packages +modules drush and composer is all you need see: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
Good point but most of those modules are updated to the latest by pypl on https://release-monitoring.org/
You do make a point that it is more convenient to just do one update via more than one. Good point. Thanks.
You could just run a script that runs dnf update --refresh, pip-review, php composer.phar update and drush update. Instead of trying to use system rpm packages for everything? For system, python, php and drupal module, etc updates?
Dne 08. 03. 21 v 18:33 Mattia Verga via devel napsal(a):
In what way is different from installing them by pip? Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
1. You know from where the package comes. Verified using GPG checks. Helps to avoid Dependency Confusions attacks https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
2. You can audit files on your disk using rpm -qf /some/file
Nothing stops you to create these packages automatically using `pyp2rpm`.
On Tue, Mar 9, 2021 at 4:20 AM Miroslav Suchý msuchy@redhat.com wrote:
Dne 08. 03. 21 v 18:33 Mattia Verga via devel napsal(a):
In what way is different from installing them by pip? Does packaging them worth spending resources (disk space, Koji cycles) and packagers time?
- You know from where the package comes. Verified using GPG checks. Helps to avoid Dependency Confusions attacks https://medium.com/@alex.birsan/dependency-confusion-4a5d60fec610
It also avoids having an RPM update down the road replace or overwrite bits of the module, breaking functionality of other parts of the system such as RPM itself. and having an update replace a module with a newer and incompatible version unknown to *other* modules on the system.
- You can audit files on your disk using rpm -qf /some/file
Nothing stops you to create these packages automatically using `pyp2rpm`.
It can be awkward to resolve all the dependencies, especially when it has mismatched version dependencies on critical modules such as sphinx. Been there, done that, done this for awscli and Samba dependencies.
On the NeuroFedora side, having Python modules packaged in Fedora allows us to produce the ISO images with all the necessary tools for a particular domain (like comp-neuro).
I know that in FOSS we assume that everyone can use virtual environments and pip, but that's not necessarily true in a multi-disciplinary field like neuroscience. Lots of folks come from biology etc., and are not traditionally trained in computing. (Unfortunately, Matlab still seems to be prevalent in science/academia).
So, when we package stuff and provide a nice integrated environment to use, it makes researchers' lives much easier. For research tools, users should but do not generally run tests to check their tools for correctness either. Us packaging them up allows us to do this too.
So, it's a little work for us maintainers (noarch Python packages are usually easy to maintain), but it will hopefully encourage more people to move to Python + FOSS + Fedora.
On Mon, 08 Mar 2021 17:33:00 +0000 Mattia Verga via devel devel@lists.fedoraproject.org wrote:
I'm just wondering: what's the benefit of packaging Python noarch projects in Fedora?
Ob XKCD: https://xkcd.com/1987/
It is the same reason I build RPMs instead of installing Perl modules directly from CPAN.
Jim
-
Ankur Sinha -
Björn Persson -
Dennis Gilmore -
James Szinger -
Mattia Verga -
Miro Hrončok -
Miroslav Suchý -
Nico Kadel-Garcia -
Reon Beon -
Stephen John Smoogen -
Vitaly Zaitsev