Re: Python packaging workflow future (setup.py install removal and PEP517)

[Filipe Laíns]

On Fri, 2020-06-05 at 22:49 +0200, Miro Hrončok wrote:
> On 05. 06. 20 22:30, Filipe Laíns wrote:
> > Hi,
>
> Hey Filipe.
>
> For Fedora related Python packaging topics, would you be OK to use a
> public 
> mailing list?
>
> https://lists.fedoraproject.org/archives/list/[e-mail]...

Sure :)
> > I am querying you a your current python packaging workflow. How do
> > you
> > handle the installation of python packages? setuptools, pip,
> > other...?
>
> For the old packages, we still very much use python setup.py install.
>
> For the new, we have provisional system that uses PEP 517/518.
>
> https://src.fedoraproject.org/rpms/pyproject-rpm-macros
>
> We have a draft of new Python packaging guidelines that use this:
>
> https://lists.fedoraproject.org/archives/list/[e-mail]...

Oh okay, that seems reasonable.
> > In Arch Linux we use setuptools (setup.py build && setup.py
> > install).
> > For PEP517 we generally use dephell to generate a setup.py script,
> > it
> > works most of the time but it's still an hack.
> >
> > Unfortunately, setup.py install will be removed soon (see [1]),
> > which
> > breaks our workflow. My proposed workflow alternative is to build
> > wheels (via PEP517 or setuptools directly) and install them.
>
> I will read that ticket later, it is very long :/
> > We don't think pip is a viable option for this because it requires
> > bootstrapping a big number of dependencies. Packaging a vendored
> > version of pip goes against our guidelines.
>
> We went the "lazy" way and our pip bundles the vendored deps. I am
> not very 
> happy about it, but it works. Our guidelines discourage it, but it is
> possible.

I see. I guess if we had absolutely no other option, that would be
possible too, but we really really don't want to.
> > I have been working on some the tooling that should be able to fit
> > our
> > needs: python-build[2] and python-install[3]. I have also been in
> > contact with a few PyPA members who have shown interest in having
> > such
> > tools.
>
> Nice. See also https://github.com/pradyunsg/installer/issues/1

Yeah, I have been participating in the discussions.
> > So, I would like to ask you if you are facing a similar problem? If
> > so,
> > I would like to get your opinion on this new tooling. Does it fix
> > your
> > problem? Do you have any reservations about this? Are there any
> > special
> > use-cases you would like to handle?
>
> Even if we create the entirely new packaging experience with PEP
> 517/518 based 
> macros, we have thousands of exciting packages that rely on python
> setup.py 
> install. If that gets removed, it will be very painful. If it is
> "just" 
> unsupported, we might be OK.

Right now it is only unsupported, but the maintainers said it will be
removed.

Can't you just adjust the macro definition? In the build step, instead
of setup.py install you run setup.py bdist_wheel, and then in the
install step you install the wheel. The new workflow should be
compatible, the different would be that instead of installing directly,
you build a wheel and then install the wheel.
> > [1] https://github.com/pypa/setuptools/issues/2088
> > [2] https://github.com/FFY00/python-build
> > [3] https://github.com/FFY00/python-install

Cheers,
Filipe Laíns