Re: Python packages with extras dependencies

[Miro Hrončok]

On 05. 02. 19 0:44, Eli Young wrote:
>  Python packages can specify extras dependencies, which are sets of
> dependencies not required for core functionality, and which generally correspond to some
> feature. These can then be specified by downstream consumers of the package. For example,
> requests has an entry in extras called security[1], which currently adds requirements of
> python packages pyOpenSSL >= 0.14, cryptography >= 1.3.4, and idna >= 2.0.0. A
> downstream consumer that wants to use this would add a dependency on requests[security].
>  
>   From what I can tell, the current practice in Fedora packaging is to ignore these. This
> simplifies packaging Python modules that have extras specified, but ultimately pushes the
> specification of those dependencies down into every consumer of the package, whether users
> or other packages.
>  
>  As an example of this, I currently maintain the python-dns-lexicon package, which
> provides a common CLI and API for various different DNS providers. Some of the providers
> have additional dependencies that are necessary to function, and which are specified as
> extras. The Plesk provider, for example, also requires python-xmltodict[2]. In line with
> what appears to standard practice, extra dependencies are not currently installed with the
> broader python-dns-lexicon package. If, however, a user or dependent package wants to
> utilize the Plesk functionality of python-dns-lexicon, they now need to know that
> python-xmltodict needs to be installed, and will need to check whenever the package
> updates as to whether or not that has changed.
>  
>  How should we be handling this? Right now, it seems that most packages follow this
> behavior of punting on the responsibility to package consumers. Should this continue? If
> not, how should we handle things? Should we just include all extras dependencies in the
> parent package? Alternatively, should we have dummy/meta subpackages for extras that
> require the parent package as well as any extras dependencies (e.g.
> python-dns-lexicon-plesk would require python-dns-lexicon and python-xmltodict)? 
This (metapackages) is what I've done in python-trimesh:

https://src.fedoraproject.org/rpms/python-trimesh/blob/master/f/python-tr...
https://src.fedoraproject.org/rpms/python-trimesh/blob/master/f/python-tr...

I'd still consider this on case by case basis instead of developing a general 
solution, sometimes a simple Recommends works. Sometimes, it's more complicated.

I'm CCing packaging and python to get more attention to this, but please keep 
the discussion on devel, so it's not shattered.


>  [1]: https://github.com/requests/requests/blob/v2.21.0/setup.py#L105
>  [2]: https://github.com/AnalogJ/lexicon/blob/v3.0.6/setup.py#L101 

-- 
Miro Hrončok
--
Phone: [phone]
IRC: mhroncok

Python packages with extras dependencies

[Igor Gnatenko]

In Rust we have similar problem (we have "features" than "extras") and
we
always package them as a subpackages.

https://src.fedoraproject.org/rpms/rust-serde/blob/master/f/rust-serde.spec
rust-serde-devel
rust-serde+alloc-devel
rust-serde+default-devel
rust-serde+derive-devel
rust-serde+rc-devel
rust-serde+serde_derive-devel
rust-serde+std-devel
rust-serde+unstable-devel

And the dependencies for all those subpackages are automatically generated.
We could do the same with Python.

On Tue, Feb 5, 2019 at 1:57 AM Miro Hrončok <mhroncok(a)redhat.com&gt; wrote:

>  On 05. 02. 19 0:44, Eli Young wrote:
>  > Python packages can specify extras dependencies, which are sets of
>  dependencies not required for core functionality, and which generally
>  correspond to some feature. These can then be specified by downstream
>  consumers of the package. For example, requests has an entry in extras
>  called security[1], which currently adds requirements of python packages
>  pyOpenSSL >= 0.14, cryptography >= 1.3.4, and idna >= 2.0.0. A downstream
>  consumer that wants to use this would add a dependency on
>  requests[security].
>  >
>  >  From what I can tell, the current practice in Fedora packaging is to
>  ignore these. This simplifies packaging Python modules that have extras
>  specified, but ultimately pushes the specification of those dependencies
>  down into every consumer of the package, whether users or other packages.
>  >
>  > As an example of this, I currently maintain the python-dns-lexicon
>  package, which provides a common CLI and API for various different DNS
>  providers. Some of the providers have additional dependencies that are
>  necessary to function, and which are specified as extras. The Plesk
>  provider, for example, also requires python-xmltodict[2]. In line with what
>  appears to standard practice, extra dependencies are not currently
>  installed with the broader python-dns-lexicon package. If, however, a user
>  or dependent package wants to utilize the Plesk functionality of
>  python-dns-lexicon, they now need to know that python-xmltodict needs to be
>  installed, and will need to check whenever the package updates as to
>  whether or not that has changed.
>  >
>  > How should we be handling this? Right now, it seems that most packages
>  follow this behavior of punting on the responsibility to package consumers.
>  Should this continue? If not, how should we handle things? Should we just
>  include all extras dependencies in the parent package? Alternatively,
>  should we have dummy/meta subpackages for extras that require the parent
>  package as well as any extras dependencies (e.g. python-dns-lexicon-plesk
>  would require python-dns-lexicon and python-xmltodict)?
>
>  This (metapackages) is what I've done in python-trimesh:
>
>
>  https://src.fedoraproject.org/rpms/python-trimesh/blob/master/f/python-tr...
>
>  https://src.fedoraproject.org/rpms/python-trimesh/blob/master/f/python-tr...
>
>  I'd still consider this on case by case basis instead of developing a
>  general
>  solution, sometimes a simple Recommends works. Sometimes, it's more
>  complicated.
>
>  I'm CCing packaging and python to get more attention to this, but please
>  keep
>  the discussion on devel, so it's not shattered.
>
>
>  > [1]: https://github.com/requests/requests/blob/v2.21.0/setup.py#L105
>  > [2]: https://github.com/AnalogJ/lexicon/blob/v3.0.6/setup.py#L101
>
>
>  --
>  Miro Hrončok
>  --
>  Phone: [phone]
>  IRC: mhroncok
>  _______________________________________________
>  python-devel mailing list -- python-devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to python-devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]...
>