pyproject-rpm-macros: adding support for Pipfile?

[Michel Alexandre Salim]

Hi,

Is there interest in supporting parsing Pipfile for build requirements?

e.g. MonkeyType uses it to list its dependencies:

https://github.com/instagram/MonkeyType

For now I can just add requirements by hand, `%pyproject_buildrequires
-r` does not error out but also does not generate anything.

Thanks,

[Lumír Balhar]

Hi.

I'm not sure about it but if we'd want to support something like this, 
I'd use micropipenv [0] to transfer Pipfile[.lock] or poetry.lock into 
requirements.txt which we can then use with the existing logic.

Lumír

[0] https://github.com/thoth-station/micropipenv

On 12/17/21 23:26, Michel Alexandre Salim wrote:
> Hi,
>
> Is there interest in supporting parsing Pipfile for build requirements?
>
> e.g. MonkeyType uses it to list its dependencies:
>
> https://github.com/instagram/MonkeyType
>
> For now I can just add requirements by hand, `%pyproject_buildrequires
> -r` does not error out but also does not generate anything.
>
> Thanks,
>
> python-devel mailing list [e-mail]
> To unsubscribe send an email [e-mail]
> Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:https://lists.fedoraproject.org/archives/list/[e-mail]...
> Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure

[Miro Hrončok]

On 17. 12. 21 23:26, Michel Alexandre Salim wrote:
> Hi,
>
> Is there interest in supporting parsing Pipfile for build requirements?

Hi. Let me borrow an answer from a similar request somewhere else [1]:

"""Whatever comes out of this, we will probably strive for supporting existing 
standards, rather than specific tools that do their own thing."""

Supporting Pipfile is *not* planned. We should encourage upstreams to use a 
standardized way of specifying dependencies.

If you need to parse out dev-packages from Pipfile, you could use a tool like 
micorpipenv (as already suggested by Lumír) to create a txt file with list of 
dependencies, and pass that file to %pyproject_buildrequires. However note that 
MonkeyType includes stuff like flake8 in there, so you would need to filter it 
anyway.

I was recently thinking that there are many common but not standardized ways 
upstream specify their test dependencies, e.g.:

- setup.py: tests_require
  - Pipfile: dev-packages
  - pyproject.toml: tool.poetry.dev-dependencies

And I don't really want to keep snowballing such use-cases to 
%pyproject_buildrequires. However, we might be able to provide a Python 
*interface* for anybody to hook in their own way. Something like:

%pyproject_buildrequires -p my.module:function

And it would use this function to generate a list of additional deps. Packagers 
could then either include custom implementations as sources if they really 
wanted to, or even package their own "%pyproject_buildrequires plugins".
But maybe that's just too complex for a very narrow use case?

[1] 
https://github.com/readthedocs/readthedocs.org/issues/3181#issuecomment-9753...
> e.g. MonkeyType uses it to list its dependencies:
>
> https://github.com/instagram/MonkeyType
>
> For now I can just add requirements by hand, `%pyproject_buildrequires
> -r` does not error out but also does not generate anything.

Amongst the defaults (pip, toml), it should also generate setuptools, wheel:

https://github.com/Instagram/MonkeyType/blob/v21.5.0/pyproject.toml#L2

and mypy_extensions, libcst:

https://github.com/Instagram/MonkeyType/blob/v21.5.0/setup.py#L48

Could you please share the specfile, so I can debug this?

[Michel Alexandre Salim]

----------------------------------------

*From: *Miro Hrončok 
*To: *Fedora Python SIG ; Michel 
Alexandre Salim 
*Date: *Dec 19, 2021 03:58:57
*Subject: *Re: pyproject-rpm-macros: adding support for Pipfile?
> On 17. 12. 21 23:26, Michel Alexandre Salim wrote:
> > Hi,
> > Is there interest in supporting parsing Pipfile for build 
> > requirements?
>
> Hi. Let me borrow an answer from a similar request somewhere else [1]:
>
> """Whatever comes out of this, we will probably strive for supporting 
> existing standards, rather than specific tools that do their own 
> thing."""
>
> Supporting Pipfile is *not* planned. We should encourage upstreams to 
> use a standardized way of specifying dependencies.

Agreed, I'll see if I can nudge them, or bring up the issue with our 
internal Python team.
> If you need to parse out dev-packages from Pipfile, you could use a 
> tool like micorpipenv (as already suggested by Lumír) to create a txt 
> file with list of dependencies, and pass that file to 
> %pyproject_buildrequires. However note that MonkeyType includes stuff 
> like flake8 in there, so you would need to filter it anyway

Thanks Lumir! I'll definitely check micropipenv, some of the other tools 
I've found don't exactly work.
> I was recently thinking that there are many common but not standardized 
> ways upstream specify their test dependencies, e.g.:
>
> - setup.py: tests_require
>   - Pipfile: dev-packages
>   - pyproject.toml: tool.poetry.dev-dependencies
>
> And I don't really want to keep snowballing such use-cases to 
> %pyproject_buildrequires. However, we might be able to provide a Python 
> *interface* for anybody to hook in their own way. Something like:
>
> %pyproject_buildrequires -p my.module:function
>
> And it would use this function to generate a list of additional deps. 
> Packagers could then either include custom implementations as sources 
> if they really wanted to, or even package their own 
> "%pyproject_buildrequires plugins".
> But maybe that's just too complex for a very narrow use case?
>
> [1] 
> https://github.com/readthedocs/readthedocs.org/issues/3181#issuecomment-9753...
> > e.g. MonkeyType uses it to list its dependencies:
> > https://github.com/instagram/MonkeyType
> > For now I can just add requirements by hand, `%pyproject_buildrequires
> > -r` does not error out but also does not generate anything.
>
> Amongst the defaults (pip, toml), it should also generate setuptools, 
> wheel:
>
> https://github.com/Instagram/MonkeyType/blob/v21.5.0/pyproject.toml#L2
>
> and mypy_extensions, libcst:
>
> https://github.com/Instagram/MonkeyType/blob/v21.5.0/setup.py#L48
>
> Could you please share the specfile, so I can debug this?

Sure. As it turns out, most of the requirements are indeed superfluous, 
the test suite passed just fine with just this:

https://bugzilla.redhat.com/show_bug.cgi?id=2033816

It seems to work as expected with the demo provided in the example, but I 
haven't tried installing it in mock to make sure the runtime deps are 
complete.

Thanks,

[Miro Hrončok]

On 20. 12. 21 5:53, Michel Alexandre Salim wrote:
> > Could you please share the specfile, so I can debug this?
>
> Sure. As it turns out, most of the requirements are indeed superfluous, the 
> test suite passed just fine with just this:
>
> https://bugzilla.redhat.com/show_bug.cgi?id=2033816
>
> It seems to work as expected with the demo provided in the example, but I 
> haven't tried installing it in mock to make sure the runtime deps are complete.

Well at least they are the same as if installed with pip.