On 07. 07. 22 14:00, Petr Viktorin wrote:
On 07. 07. 22 11:38, Miro Hrončok wrote:
> Hello Pythonistats, packagers,
>
> A handful of Fedora Python packages uses pytest-xdist to run tests in
> parallel like this:
>
> %pytest -n auto
>
> -n auto means pytest will spawn a number of workers processes equal to the
> number of available CPUs.
>
> In the spirit of other packaging guidelines, I believe we should use this
> instead:
>
> %pytest -n %{_smp_build_ncpus}
>
> This means the same thing in most of the ceases, but will limit the number of
> workers depending on other constraints in the spec or in the environment.
>
> Should I do this in a mass change? Not so many packages use pytest -n auto in
> the spec:
>
> $ rg -l -- '(-n|--numprocesses)(\s*|=)auto(\s|$)'
> ansible-bender.spec
> azure-cli.spec
> ocrmypdf.spec
> python-cartopy.spec
> python-GridDataFormats.spec
> python-hypothesis.spec
> python-matplotlib.spec
> python-mplcairo.spec
> python-rpmautospec.spec
> python-sqlalchemy.spec
> python-tox.spec
> python-xarray.spec
> python-zstandard.spec
> pythran.spec
> scipy.spec
>
> (Other packages have that in tox.ini or pytest config file, but I am not
> aiming at changing that here.)
>
> And, considering many other packages might want to benefit from that, should
> this be:
>
> 1) encouraged in the Python packaging guidelines
> 2) macronized (I was thinking %pytest_parallel, but TBD)
Or pytest-xdist could be taught to check an environment variable for `auto`,
making this seamless for packagers?
This could work, except that sometimes pytest-xdist is installed and we don't
want to run tests in parallel because they are not prepared for that. But I
guess an opt-out would still exist, e.g. setting %{_smp_build_ncpus} to 1 in
the %check section.
I will have a look if there is an environment variable we could use. We can
probably adjust PYTEST_ADDOPTS if we detect pytest-xdist is available.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok