[Bug 1662777] New: Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Bug ID: 1662777
           Summary: Review Request: python-pytest-randomly - Pytest plugin
                    to randomly order tests and control random.seed
           Product: Fedora
           Version: rawhide
          Hardware: All
                OS: Linux
            Status: NEW
         Component: Package Review
          Severity: medium
          Priority: medium
          Assignee: [e-mail]
          Reporter: [e-mail]
        QA Contact: [e-mail]
                CC: [e-mail]
  Target Milestone: ---
    Classification: Fedora

Spec URL:
https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-rand...
SRPM URL:
https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-rand...
Description: Pytest-randomly is a pytest plugin to randomly order tests and
control random.seed.
Fedora Account System Username: dcallagh

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Hirotaka Wakabayashi  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #1 from Hirotaka Wakabayashi  ---
Hello, this is an unofficial review. Please read it for your reference.

# Summary

1. Rpmlint on the src.rpm reported 1 errors, 0 warnings
2. Rpmlint on the binary rpm reported 2 errors, 0 warnings
3. %check should not be failed

# Details

## 1. Rpmlint on the src.rpm reported 1 errors, 0 warnings

no-changelogname-tag error found. You need add a changelog entry.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs

Here is the output in my fedora29 environment:
```
python-pytest-randomly.src: E: no-changelogname-tag
There is no %changelog tag in your spec file. To insert it, just insert a
'%changelog' in your spec file and rebuild it.

1 packages and 0 specfiles checked; 1 errors, 0 warnings.
```

## 2. Rpmlint on the binary rpm reported 2 errors, 0 warnings

no-changelogname-tag error and non-readable error found.

1. no-changelogname-tag
You need add a changelog entry.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#changelogs

2. non-readable
pytest_randomly.cpython-37-PYTEST.pyc is non-readable from users. Permissions
on files MUST be set properly.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_permissions

Here is the output in my fedora29 environment:
```
python3-pytest-randomly.noarch: E: no-changelogname-tag
There is no %changelog tag in your spec file. To insert it, just insert a
'%changelog' in your spec file and rebuild it.

python3-pytest-randomly.noarch: E: non-readable
/usr/lib/python3.7/site-packages/__pycache__/pytest_randomly.cpython-37-PYTEST.pyc
600
The file can't be read by everybody. Review if this is expected.

1 packages and 0 specfiles checked; 2 errors, 0 warnings.
```

## 3. %check should not be failed

%check failed and the status seems to be currently ignored.
tests should be executed and I think they should be executed correctly.
https://docs.fedoraproject.org/en-US/packaging-guidelines/#_test_suites

Here is the output of 'rpmbuild -ra
python-pytest-randomly-1.2.3-1.fc29.src.rpm'.
```
...snip...

+ pytest-3 tests/
============================= test session starts
==============================
platform linux -- Python 3.7.2, pytest-3.6.4, py-1.5.4, pluggy-0.6.0
Using --randomly-seed=1548246897
rootdir: /home/vagrant/rpmbuild/BUILD/pytest-randomly-1.2.3, inifile:
plugins: randomly-1.2.3
collected 20 items

tests/test_it.py ..........F...F.....                                    [100%]

...snip...
```

Thanks in advance,
Hirotaka Wakabayashi

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Zbigniew Jędrzejewski-Szmek  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #2 from Zbigniew Jędrzejewski-Szmek  ---
Looks good in general.
> pytest_randomly.cpython-37-PYTEST.pyc

Is this file created by pytest? It looks like it shouldn't be part of the
package at all.

The changelog is indeed missing.

The spec file and srpm don't match anymore. Please post an updated spec file
and srpm.

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Elliott Sales de Andrade  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[e-mail]

--- Comment #3 from Elliott Sales de Andrade  ---
You should set PYTHONDONTWRITEBYTECODE=1 when running tests so that you don't
get pytest bytecode in your rpms.

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #4 from Dan Callaghan  ---
Thanks for the tip about PYTHONDONTWRITEBYTECODE=1. Also it turns out the bug
in python-faker affecting the tests has now been fixed so they pass cleanly.

Here is an updated version with the tests properly enabled, no pytest bytecode,
and a proper %changelog:

https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-rand...
https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-rand...

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #5 from Zbigniew Jędrzejewski-Szmek  ---
Oops, upstream is at version 3.2.0 now. Please update!

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #6 from Dan Callaghan  ---
Oh yes indeed.

I updated the package but noticed that upstream accidentally dropped the test
suite from the latest tarballs. I filed a PR for it:
https://github.com/pytest-dev/pytest-randomly/pull/223

Let's see if they roll a new release for it soon...

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #7 from Zbigniew Jędrzejewski-Szmek  ---
Please post a new spec file and srpm.

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #8 from Dan Callaghan  ---
Yep I was just waiting to see upstream's response about the missing tests
before I post the new spec.

As I expected, they are resistant to including tests in the source tarball from
PyPI and suggest I should be using the Github tarballs instead. I guess
nowadays everyone sees PyPI as basically a binary distribution platform and not
a source hosting platform...

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #9 from Dan Callaghan  ---
Upstream has published a new release 3.2.1 with the tests included again.

Updated:
https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-rand...
https://fedorapeople.org/~dcallagh/python-pytest-randomly/python-pytest-rand...

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Zbigniew Jędrzejewski-Szmek  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |POST
           Assignee|[e-mail]    |[e-mail]
              Flags|                            |fedora-review+

--- Comment #10 from Zbigniew Jędrzejewski-Szmek  ---
+ package name is OK
+ latest version
+ license is acceptable for Fedora (BSD 3-clause)
+ license is specified correctly
+ builds and installs OK
+ rpmlint finds no issues

Package is APPROVED.

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #11 from Igor Gnatenko  ---
(fedscm-admin):  The Pagure repository was created at
https://src.fedoraproject.org/rpms/python-pytest-randomly

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Fedora Update System  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|POST                        |MODIFIED

--- Comment #12 from Fedora Update System  ---
FEDORA-2020-26bda81e92 has been submitted as an update to Fedora 31.
https://bodhi.fedoraproject.org/updates/FEDORA-2020-26bda81e92

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Fedora Update System  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|MODIFIED                    |ON_QA

--- Comment #13 from Fedora Update System  ---
python-pytest-randomly-3.2.1-1.fc31 has been pushed to the Fedora 31 testing
repository. If problems still persist, please make note of it in this bug
report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-26bda81e92

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #14 from Fedora Update System  ---
python-pytest-randomly-3.2.1-1.fc30 has been pushed to the Fedora 30 testing
repository. If problems still persist, please make note of it in this bug
report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here:
https://bodhi.fedoraproject.org/updates/FEDORA-2020-a76fa3f47e

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

Fedora Update System  changed:

What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ON_QA                       |CLOSED
         Resolution|---                         |ERRATA
        Last Closed|                            |2020-02-27 16:44:34

--- Comment #15 from Fedora Update System  ---
python-pytest-randomly-3.2.1-1.fc30 has been pushed to the Fedora 30 stable
repository. If problems still persist, please make note of it in this bug
report.

[Bug 1662777] Review Request: python-pytest-randomly - Pytest plugin to randomly order tests and control random.seed

https://bugzilla.redhat.com/show_bug.cgi?id=1662777

--- Comment #16 from Fedora Update System  ---
python-pytest-randomly-3.2.1-1.fc31 has been pushed to the Fedora 31 stable
repository. If problems still persist, please make note of it in this bug
report.