[PATCH 0/2] Streamline tests output

[Nir Soffer]

Streamline tests output by using normal mode and not listing skipped
tests in the short test summary.

Here is example output from travis build:

$ pytest
============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-4.3.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/travis/build/nirs/sanlock, inifile: tox.ini
collected 210 items                                                            
tests/daemon_test.py ...........                                         [  5%]
tests/direct_test.py ..                                                  [  6%]
tests/python_test.py ........ssssssssssss........ssssssssssssss......... [ 30%]
...............ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss. [ 64%]
ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.......... [ 99%]
..                                                                       [100%]
========================== slowest 10 test durations ===========================
6.02s call     tests/python_test.py::test_add_rem_lockspace_async
6.02s call     tests/python_test.py::test_add_rem_lockspace[1099511627776-1099510579200]
6.02s call     tests/python_test.py::test_add_rem_lockspace[1048576-0]
4.06s call     tests/python_test.py::test_acquire_release_resource[1048576-0]
4.05s call     tests/python_test.py::test_acquire_release_resource[1099511627776-1099510579200]
3.21s call     tests/daemon_test.py::test_create
3.12s call     tests/daemon_test.py::test_delete
3.09s call     tests/daemon_test.py::test_lookup
3.05s call     tests/daemon_test.py::test_lookup_missing
0.42s call     tests/daemon_test.py::test_start_after_kill
=================== 104 passed, 106 skipped in 46.73 seconds ===================
The command "pytest" exited with 0.

0.60s$ flake8 --statistics tests python
The command "flake8 --statistics tests python" exited with 0.

Based on commit edf1e08326daa39a2e453d936fa9a99dd0ec136c (python:
Convert example.py to python 3)

Nir Soffer (2):
  python: Don't report skipped tests
  python: Use pytest normal mode

README.dev | 9 +++++++++
 tox.ini    | 5 ++---
 2 files changed, 11 insertions(+), 3 deletions(-)

[PATCH 1/2] python: Don't report skipped tests

[Nir Soffer]

In the short test summary, report only (x)failed and (X)passed tests.
Skipped tests are normal, used when a test cannot run in the current
environment. (x)failed or (X)passed tests must be fixed and should be
listed.

Signed-off-by: Nir Soffer 
---
 tox.ini | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tox.ini b/tox.ini
index 0e337ce..0cadfb5 100644
--- a/tox.ini
+++ b/tox.ini
@@ -28,11 +28,11 @@ commands = flake8 --statistics tests python

[pytest]
 # Notes:
 # --basetemp: we must use /var/tmp as sanlock uses direct I/O.
 # -vv: increasing verbosify twice shows more detailed failures tracebacks.
-# -rxs: show extra test summary: (s)skipped, (x)failed
+# -rxX show extra test summary: (x)failed, (X)passed,
 # --durations: show slowest test duration
-addopts = -rxs -vv --basetemp=/var/tmp/sanlock --durations=10
+addopts = -rxX -vv --basetemp=/var/tmp/sanlock --durations=10

[flake8]
 show_source = True

[PATCH 2/2] python: Use pytest normal mode

[Nir Soffer]

We have too many tests to use verbose mode. If verbose more is needed it
can be enabled using:

tox -e py36 tests/python_test.py -- -vv

Or using environment variable:

export PYTEST_ADDOPTS=-vv

Signed-off-by: Nir Soffer 
---
 README.dev | 9 +++++++++
 tox.ini    | 3 +--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/README.dev b/README.dev
index 8ade5e6..7f3df2c 100644
--- a/README.dev
+++ b/README.dev
@@ -39,10 +39,19 @@ To run only tests matching the substring "foo":

To run basic lint and style check:

$ tox -e flake8

+Sometimes when debugging failing tests verbose mode is useful. You can
+enable it using:
+
+    $ tox -e py36 tests/python_test.py -- -vv
+
+Or using environment variable:
+
+    export PYTEST_ADDOPTS=-vv
+

Testing 4K support
 ==================

To enable the 4k tests, you need to setup 4k stroage for the tests:
diff --git a/tox.ini b/tox.ini
index 0cadfb5..4a28561 100644
--- a/tox.ini
+++ b/tox.ini
@@ -27,12 +27,11 @@ deps = flake8
 commands = flake8 --statistics tests python

[pytest]
 # Notes:
 # --basetemp: we must use /var/tmp as sanlock uses direct I/O.
-# -vv: increasing verbosify twice shows more detailed failures tracebacks.
 # -rxX show extra test summary: (x)failed, (X)passed,
 # --durations: show slowest test duration
-addopts = -rxX -vv --basetemp=/var/tmp/sanlock --durations=10
+addopts = -rxX --basetemp=/var/tmp/sanlock --durations=10

[flake8]
 show_source = True

[Vojtech Juranek]

LGTM
> Streamline tests output by using normal mode and not listing skipped
> tests in the short test summary.
>
> Here is example output from travis build:
>
> $ pytest
> ============================= test session starts
> ==============================

platform linux -- Python 3.7.1,
> pytest-4.3.1, py-1.7.0, pluggy-0.8.0 rootdir:
> /home/travis/build/nirs/sanlock, inifile: tox.ini
> collected 210 items

tests/daemon_test.py ...........
> [  5%] tests/direct_test.py ..                                            
>      [  6%] tests/python_test.py
> ........ssssssssssss........ssssssssssssss......... [ 30%]
> ...............ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss. [
> 64%]
> ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.......... [
> 99%] ..                                                                    
>   [100%] ========================== slowest 10 test durations
> =========================== 6.02s call    
> tests/python_test.py::test_add_rem_lockspace_async 6.02s call    
> tests/python_test.py::test_add_rem_lockspace[1099511627776-1099510579200]
> 6.02s call     tests/python_test.py::test_add_rem_lockspace[1048576-0]
> 4.06s call    
> tests/python_test.py::test_acquire_release_resource[1048576-0] 4.05s call
>
> tests/python_test.py::test_acquire_release_resource[1099511627776-109951057
> 9200] 3.21s call     tests/daemon_test.py::test_create
> 3.12s call     tests/daemon_test.py::test_delete
> 3.09s call     tests/daemon_test.py::test_lookup
> 3.05s call     tests/daemon_test.py::test_lookup_missing
> 0.42s call     tests/daemon_test.py::test_start_after_kill
> =================== 104 passed, 106 skipped in 46.73 seconds
> ===================

The command "pytest" exited with 0.
> 0.60s$ flake8 --statistics tests python
> The command "flake8 --statistics tests python" exited with 0.
>
> Based on commit edf1e08326daa39a2e453d936fa9a99dd0ec136c (python:
> Convert example.py to python 3)
>
> Nir Soffer (2):
>   python: Don't report skipped tests
>   python: Use pytest normal mode
>
> README.dev | 9 +++++++++
>  tox.ini    | 5 ++---
>  2 files changed, 11 insertions(+), 3 deletions(-)
>
> -- 
> 2.17.2
> _______________________________________________
> sanlock-devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
> Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
> Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]
> ed.org

[Nir Soffer]

On Mon, Jun 17, 2019 at 4:05 PM Vojtech Juranek  wrote:
> LGTM

Thanks, pushed
> > Streamline tests output by using normal mode and not listing skipped
> > tests in the short test summary.
> >
> > Here is example output from travis build:
> >
> > $ pytest
> > ============================= test session starts
> > ==============================
>
> platform linux -- Python 3.7.1,
> > pytest-4.3.1, py-1.7.0, pluggy-0.8.0 rootdir:
> > /home/travis/build/nirs/sanlock, inifile: tox.ini
> > collected 210 items
> >
>
> tests/daemon_test.py ...........
> > [  5%] tests/direct_test.py ..
> >  [  6%] tests/python_test.py
> >
> > ........ssssssssssss........ssssssssssssss......... [ 30%]
> > ...............ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.
>
> [
> > 64%]
> > ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss..........
>
> [
> > 99%] ..
> > [100%] ========================== slowest 10 test durations
> > =========================== 6.02s call
> > tests/python_test.py::test_add_rem_lockspace_async 6.02s call
> > tests/python_test.py::test_add_rem_lockspace[1099511627776-1099510579200]
> > 6.02s call     tests/python_test.py::test_add_rem_lockspace[1048576-0]
> > 4.06s call
> > tests/python_test.py::test_acquire_release_resource[1048576-0] 4.05s
>
> call
> >
>
> tests/python_test.py::test_acquire_release_resource[1099511627776-109951057
> > 9200] 3.21s call     tests/daemon_test.py::test_create
> > 3.12s call     tests/daemon_test.py::test_delete
> > 3.09s call     tests/daemon_test.py::test_lookup
> > 3.05s call     tests/daemon_test.py::test_lookup_missing
> > 0.42s call     tests/daemon_test.py::test_start_after_kill
> > =================== 104 passed, 106 skipped in 46.73 seconds
> > ===================
>
> The command "pytest" exited with 0.
> > 0.60s$ flake8 --statistics tests python
> > The command "flake8 --statistics tests python" exited with 0.
> >
> > Based on commit edf1e08326daa39a2e453d936fa9a99dd0ec136c (python:
> > Convert example.py to python 3)
> >
> > Nir Soffer (2):
> >   python: Don't report skipped tests
> >   python: Use pytest normal mode
> >
> > README.dev | 9 +++++++++
> >  tox.ini    | 5 ++---
> >  2 files changed, 11 insertions(+), 3 deletions(-)
> >
> > --
> > 2.17.2
> > _______________________________________________
> > sanlock-devel mailing list -- [e-mail]
> > To unsubscribe send an email to
>
> [e-mail]
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
> > Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
> > Archives:
>
> https://lists.fedorahosted.org/archives/list/[e-mail]
> > ed.org
>
> sanlock-devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail]...