Can we get rid of some Python dependencies in all Rust builds

[Miro Hrončok]

Hello Rust folks,

I see that all Rust crate packages require rust-packaging which in turn 
requires python3-rust2rpm. That brings in:

  - Python (obviously)
  - python-jinja2
  - python-requests
  - python-tqdm

I always forget about this when I try to do some impact checks in Copr. 
Currently, I am rebuilding all (transitive) build-dependents of 
python-markupsafe in Fedora 35 and 36 due to a requested update. This includes 
~2k rust packages that could probably be skipped, becasue markupsafe is 
required by jinja2.

Do all rust packages indeed need all 3 listed Python libs on runtime?

As far as I understand, Python is used in the dependency generators trough the 
cargo-inspector entry-point. So, let's live with that. jinja2, requests and 
tqdm all seem to only by imported from the rust2rpm entrypoint (rust2rpm.__main__).

Can we please get rid of this dependency? It could be done on several levels:

  1) upstream pyp2rpm, either or:
    a) split the project into inspector, rust2rpm and common
       -- I consider this tedious
    b) make rust2rpm CLI a Python extra
       -- I consider this bad UX, since the upstream project *is rust2rpm*

  2) downstream rust-pacakging, all of the following:
    - move crate-inspector bits to rust-pacakging
    - stop requiring python3-rust2rpm from rust-pacakging
    - use modern Python macros to install the entry_point via pip
      (this is needed to be able to run ti without the metadata installed)

I have a proof of concept for (2) but it requires some small adjustment 
upstream -- the inspector module must not import form __main__ (that is very 
bad habit): https://src.fedoraproject.org/rpms/rust-packaging/pull-request/3

WDYT?
-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Fabio Valentini]

On Mon, Jul 11, 2022 at 6:48 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  Hello Rust folks,
>
>  I see that all Rust crate packages require rust-packaging which in turn
>  requires python3-rust2rpm. That brings in:
>
>    - Python (obviously)
>    - python-jinja2
>    - python-requests
>    - python-tqdm
>
>  I always forget about this when I try to do some impact checks in Copr.
>  Currently, I am rebuilding all (transitive) build-dependents of
>  python-markupsafe in Fedora 35 and 36 due to a requested update. This includes
>  ~2k rust packages that could probably be skipped, becasue markupsafe is
>  required by jinja2.
>
>  Do all rust packages indeed need all 3 listed Python libs on runtime?
>
>  As far as I understand, Python is used in the dependency generators trough the
>  cargo-inspector entry-point. So, let's live with that. jinja2, requests and
>  tqdm all seem to only by imported from the rust2rpm entrypoint (rust2rpm.__main__).
>
>  Can we please get rid of this dependency? It could be done on several levels:
>
>    1) upstream pyp2rpm, either or:
>      a) split the project into inspector, rust2rpm and common
>         -- I consider this tedious
>      b) make rust2rpm CLI a Python extra
>         -- I consider this bad UX, since the upstream project *is rust2rpm*
>
>    2) downstream rust-pacakging, all of the following:
>      - move crate-inspector bits to rust-pacakging
>      - stop requiring python3-rust2rpm from rust-pacakging
>      - use modern Python macros to install the entry_point via pip
>        (this is needed to be able to run ti without the metadata installed)
>
>  I have a proof of concept for (2) but it requires some small adjustment
>  upstream -- the inspector module must not import form __main__ (that is very
>  bad habit): https://src.fedoraproject.org/rpms/rust-packaging/pull-request/3
>
>  WDYT? 
My PR to split core functionality from the things that need
third-party python packages has been merged:
https://pagure.io/fedora-rust/rust2rpm/pull-request/204

Once this is released as part of rust2rpm v22, we can adapt the
packaging so that cargo-inspector will no longer pull in any
third-party python packages, as it now only needs the python3
interpreter and only imports things from the python standard library.

I don't think we can get rid of the python3 interpreter dependency,
unless we rewrite the RPM dependency generator bits (cargo-inspector)
in some other language.
But other than that, does this improve the situation from your POV?

Fabio

[Miro Hrončok]

On 17. 07. 22 17:34, Fabio Valentini wrote:
>  My PR to split core functionality from the things that need
>  third-party python packages has been merged:
>  https://pagure.io/fedora-rust/rust2rpm/pull-request/204
>  
>  Once this is released as part of rust2rpm v22, we can adapt the
>  packaging so that cargo-inspector will no longer pull in any
>  third-party python packages, as it now only needs the python3
>  interpreter and only imports things from the python standard library.
>  
>  I don't think we can get rid of the python3 interpreter dependency,
>  unless we rewrite the RPM dependency generator bits (cargo-inspector)
>  in some other language. 
That's quite understandable. Could it be written in Rust? Not a priority, just 
curious.

>  But other than that, does this improve the situation from your POV?
By all means, it does! Thank you.

-- 
Miro Hrončok
-- 
Phone: [phone]
IRC: mhroncok

[Neal Gompa]

On Mon, Jul 18, 2022 at 12:03 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  On 17. 07. 22 17:34, Fabio Valentini wrote:
>  > My PR to split core functionality from the things that need
>  > third-party python packages has been merged:
>  > https://pagure.io/fedora-rust/rust2rpm/pull-request/204
>  >
>  > Once this is released as part of rust2rpm v22, we can adapt the
>  > packaging so that cargo-inspector will no longer pull in any
>  > third-party python packages, as it now only needs the python3
>  > interpreter and only imports things from the python standard library.
>  >
>  > I don't think we can get rid of the python3 interpreter dependency,
>  > unless we rewrite the RPM dependency generator bits (cargo-inspector)
>  > in some other language.
>
>  That's quite understandable. Could it be written in Rust? Not a priority, just
>  curious.
>  
I'd rather it not be written in Rust. That would make a number of
things too complicated, and Rust is a lot less accessible for people
to contribute to.



-- 
真実はいつも一つ！/ Always, there's only one truth!

[Fabio Valentini]

On Mon, Jul 18, 2022 at 6:08 PM Neal Gompa <ngompa13(a)gmail.com&gt; wrote:
>  On Mon, Jul 18, 2022 at 12:03 PM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:
>  >
>  > On 17. 07. 22 17:34, Fabio Valentini wrote:
>  > > My PR to split core functionality from the things that need
>  > > third-party python packages has been merged:
>  > > https://pagure.io/fedora-rust/rust2rpm/pull-request/204
>  > >
>  > > Once this is released as part of rust2rpm v22, we can adapt the
>  > > packaging so that cargo-inspector will no longer pull in any
>  > > third-party python packages, as it now only needs the python3
>  > > interpreter and only imports things from the python standard library.
>  > >
>  > > I don't think we can get rid of the python3 interpreter dependency,
>  > > unless we rewrite the RPM dependency generator bits (cargo-inspector)
>  > > in some other language.
>  >
>  > That's quite understandable. Could it be written in Rust? Not a priority, just
>  > curious. 
I mean, it *could* be rewritten in Rust. The project isn't that big.
And doing that would also let us reuse the same SemVer parsing
libraries that are used by cargo instead of using a custom
reimplementation in Python.
However, it would probably introduce a bootstrap problem for the Rust
stack (i.e. because the dependency generator for Rust crates now is
itself a Rust crate).
So we'd probably need to keep the Python version around, as well.

>  By all means, it does! Thank you. 
Great! I'm looking forward to the rust2rpm v22 release, it will bring
lots of quality-of-life improvements for Rust packagers, as well as
this dependency pruning.

>  I'd rather it not be written in Rust. That would make a number
> of
>  things too complicated, and Rust is a lot less accessible for people
>  to contribute to. 
Not sure if I agree with that. cargo-inspector is a tool that's used
exclusively by Rust RPM packaging macros, so we already require
contributors to use a different language (Python) that is, for all
intents and purposes, unrelated to both RPM macros and Rust. You could
make the argument that more people know how to write Python than know
how to write Rust, but maybe people who don't know how to write Rust
code shouldn't package Rust crates or make changes to Rust packaging
tools, in the first place? :)

Fabio