[PATCH 0/2] Streamline tests output

[Nir Soffer]

Streamline tests output by using normal mode and not listing skipped
tests in the short test summary.


Here is example output from travis build:

$ pytest
============================= test session starts ==============================
platform linux -- Python 3.7.1, pytest-4.3.1, py-1.7.0, pluggy-0.8.0
rootdir: /home/travis/build/nirs/sanlock, inifile: tox.ini
collected 210 items                                                            
tests/daemon_test.py ...........                                         [  5%]
tests/direct_test.py ..                                                  [  6%]
tests/python_test.py ........ssssssssssss........ssssssssssssss......... [ 30%]
...............ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss. [ 64%]
ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.......... [ 99%]
..                                                                       [100%]
========================== slowest 10 test durations ===========================
6.02s call     tests/python_test.py::test_add_rem_lockspace_async
6.02s call     tests/python_test.py::test_add_rem_lockspace[1099511627776-1099510579200]
6.02s call     tests/python_test.py::test_add_rem_lockspace[1048576-0]
4.06s call     tests/python_test.py::test_acquire_release_resource[1048576-0]
4.05s call    
tests/python_test.py::test_acquire_release_resource[1099511627776-1099510579200]
3.21s call     tests/daemon_test.py::test_create
3.12s call     tests/daemon_test.py::test_delete
3.09s call     tests/daemon_test.py::test_lookup
3.05s call     tests/daemon_test.py::test_lookup_missing
0.42s call     tests/daemon_test.py::test_start_after_kill
=================== 104 passed, 106 skipped in 46.73 seconds ===================
The command "pytest" exited with 0.

0.60s$ flake8 --statistics tests python
The command "flake8 --statistics tests python" exited with 0.


Based on commit edf1e08326daa39a2e453d936fa9a99dd0ec136c (python:
Convert example.py to python 3)


Nir Soffer (2):
  python: Don't report skipped tests
  python: Use pytest normal mode

 README.dev | 9 +++++++++
 tox.ini    | 5 ++---
 2 files changed, 11 insertions(+), 3 deletions(-)

-- 
2.17.2

[PATCH 1/2] python: Don't report skipped tests

[Nir Soffer]

In the short test summary, report only (x)failed and (X)passed tests.
Skipped tests are normal, used when a test cannot run in the current
environment. (x)failed or (X)passed tests must be fixed and should be
listed.

Signed-off-by: Nir Soffer <nsoffer(a)redhat.com&gt;
---
 tox.ini | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tox.ini b/tox.ini
index 0e337ce..0cadfb5 100644
--- a/tox.ini
+++ b/tox.ini
@@ -28,11 +28,11 @@ commands = flake8 --statistics tests python
 
 [pytest]
 # Notes:
 # --basetemp: we must use /var/tmp as sanlock uses direct I/O.
 # -vv: increasing verbosify twice shows more detailed failures tracebacks.
-# -rxs: show extra test summary: (s)skipped, (x)failed
+# -rxX show extra test summary: (x)failed, (X)passed,
 # --durations: show slowest test duration
-addopts = -rxs -vv --basetemp=/var/tmp/sanlock --durations=10
+addopts = -rxX -vv --basetemp=/var/tmp/sanlock --durations=10
 
 [flake8]
 show_source = True
-- 
2.17.2

[PATCH 2/2] python: Use pytest normal mode

[Nir Soffer]

We have too many tests to use verbose mode. If verbose more is needed it
can be enabled using:

    tox -e py36 tests/python_test.py -- -vv

Or using environment variable:

    export PYTEST_ADDOPTS=-vv

Signed-off-by: Nir Soffer <nsoffer(a)redhat.com&gt;
---
 README.dev | 9 +++++++++
 tox.ini    | 3 +--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/README.dev b/README.dev
index 8ade5e6..7f3df2c 100644
--- a/README.dev
+++ b/README.dev
@@ -39,10 +39,19 @@ To run only tests matching the substring "foo":
 
 To run basic lint and style check:
 
     $ tox -e flake8
 
+Sometimes when debugging failing tests verbose mode is useful. You can
+enable it using:
+
+    $ tox -e py36 tests/python_test.py -- -vv
+
+Or using environment variable:
+
+    export PYTEST_ADDOPTS=-vv
+
 
 Testing 4K support
 ==================
 
 To enable the 4k tests, you need to setup 4k stroage for the tests:
diff --git a/tox.ini b/tox.ini
index 0cadfb5..4a28561 100644
--- a/tox.ini
+++ b/tox.ini
@@ -27,12 +27,11 @@ deps = flake8
 commands = flake8 --statistics tests python
 
 [pytest]
 # Notes:
 # --basetemp: we must use /var/tmp as sanlock uses direct I/O.
-# -vv: increasing verbosify twice shows more detailed failures tracebacks.
 # -rxX show extra test summary: (x)failed, (X)passed,
 # --durations: show slowest test duration
-addopts = -rxX -vv --basetemp=/var/tmp/sanlock --durations=10
+addopts = -rxX --basetemp=/var/tmp/sanlock --durations=10
 
 [flake8]
 show_source = True
-- 
2.17.2

[Vojtech Juranek]

LGTM

>  Streamline tests output by using normal mode and not listing skipped
>  tests in the short test summary.
>  
>  
>  Here is example output from travis build:
>  
>  $ pytest
>  ============================= test session starts
>  ============================== 
 platform linux -- Python 3.7.1,
>  pytest-4.3.1, py-1.7.0, pluggy-0.8.0 rootdir:
>  /home/travis/build/nirs/sanlock, inifile: tox.ini
>  collected 210 items                                                         
>     
 tests/daemon_test.py ...........                                       
>   [  5%] tests/direct_test.py ..                                      
>      
>       [  6%] tests/python_test.py
>  ........ssssssssssss........ssssssssssssss......... [ 30%]
>  ...............ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss. [
>  64%]
>  ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.......... [
>  99%] ..                                                                    
>    [100%] ========================== slowest 10 test durations
>  =========================== 6.02s call    
>  tests/python_test.py::test_add_rem_lockspace_async 6.02s call    
>  tests/python_test.py::test_add_rem_lockspace[1099511627776-1099510579200]
>  6.02s call     tests/python_test.py::test_add_rem_lockspace[1048576-0]
>  4.06s call    
>  tests/python_test.py::test_acquire_release_resource[1048576-0] 4.05s call  
>   
>  tests/python_test.py::test_acquire_release_resource[1099511627776-109951057
>  9200] 3.21s call     tests/daemon_test.py::test_create
>  3.12s call     tests/daemon_test.py::test_delete
>  3.09s call     tests/daemon_test.py::test_lookup
>  3.05s call     tests/daemon_test.py::test_lookup_missing
>  0.42s call     tests/daemon_test.py::test_start_after_kill
>  =================== 104 passed, 106 skipped in 46.73 seconds
>  =================== 
 The command "pytest" exited with 0.
>  
>  0.60s$ flake8 --statistics tests python
>  The command "flake8 --statistics tests python" exited with 0.
>  
>  
>  Based on commit edf1e08326daa39a2e453d936fa9a99dd0ec136c (python:
>  Convert example.py to python 3)
>  
>  
>  Nir Soffer (2):
>    python: Don't report skipped tests
>    python: Use pytest normal mode
>  
>   README.dev | 9 +++++++++
>   tox.ini    | 5 ++---
>   2 files changed, 11 insertions(+), 3 deletions(-)
>  
>  -- 
>  2.17.2
>  _______________________________________________
>  sanlock-devel mailing list -- sanlock-devel(a)lists.fedorahosted.org
>  To unsubscribe send an email to sanlock-devel-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
>  Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
>  Archives:
>  https://lists.fedorahosted.org/archives/list/[e-mail]
>  ed.org

[Nir Soffer]

On Mon, Jun 17, 2019 at 4:05 PM Vojtech Juranek <vjuranek(a)redhat.com&gt; wrote:

>  LGTM
>  
Thanks, pushed



>  > Streamline tests output by using normal mode and not listing skipped
>  > tests in the short test summary.
>  >
>  >
>  > Here is example output from travis build:
>  >
>  > $ pytest
>  > ============================= test session starts
>  > ==============================
>   platform linux -- Python 3.7.1,
>  > pytest-4.3.1, py-1.7.0, pluggy-0.8.0 rootdir:
>  > /home/travis/build/nirs/sanlock, inifile: tox.ini
>  > collected 210 items
>
>  >
>   tests/daemon_test.py ...........
>  >  [  5%] tests/direct_test.py ..
>
>  >      [  6%] tests/python_test.py
>  > ........ssssssssssss........ssssssssssssss......... [ 30%]
>  > ...............ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.
>  [
>  > 64%]
>  > ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss.ss..........
>  [
>  > 99%] ..
>
>  >   [100%] ========================== slowest 10 test durations
>  > =========================== 6.02s call
>  > tests/python_test.py::test_add_rem_lockspace_async 6.02s call
>  > tests/python_test.py::test_add_rem_lockspace[1099511627776-1099510579200]
>  > 6.02s call     tests/python_test.py::test_add_rem_lockspace[1048576-0]
>  > 4.06s call
>  > tests/python_test.py::test_acquire_release_resource[1048576-0] 4.05s
>  call
>  >
>  >
>  tests/python_test.py::test_acquire_release_resource[1099511627776-109951057
>  > 9200] 3.21s call     tests/daemon_test.py::test_create
>  > 3.12s call     tests/daemon_test.py::test_delete
>  > 3.09s call     tests/daemon_test.py::test_lookup
>  > 3.05s call     tests/daemon_test.py::test_lookup_missing
>  > 0.42s call     tests/daemon_test.py::test_start_after_kill
>  > =================== 104 passed, 106 skipped in 46.73 seconds
>  > ===================
>   The command "pytest" exited with 0.
>  >
>  > 0.60s$ flake8 --statistics tests python
>  > The command "flake8 --statistics tests python" exited with 0.
>  >
>  >
>  > Based on commit edf1e08326daa39a2e453d936fa9a99dd0ec136c (python:
>  > Convert example.py to python 3)
>  >
>  >
>  > Nir Soffer (2):
>  >   python: Don't report skipped tests
>  >   python: Use pytest normal mode
>  >
>  >  README.dev | 9 +++++++++
>  >  tox.ini    | 5 ++---
>  >  2 files changed, 11 insertions(+), 3 deletions(-)
>  >
>  > --
>  > 2.17.2
>  > _______________________________________________
>  > sanlock-devel mailing list -- sanlock-devel(a)lists.fedorahosted.org
>  > To unsubscribe send an email to
>  sanlock-devel-leave(a)lists.fedorahosted.org
>  > Fedora Code of Conduct:
>  > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List
>  > Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List
>  > Archives:
>  >
>  https://lists.fedorahosted.org/archives/list/[e-mail]
>  > ed.org
>
>  _______________________________________________
>  sanlock-devel mailing list -- sanlock-devel(a)lists.fedorahosted.org
>  To unsubscribe send an email to sanlock-devel-leave(a)lists.fedorahosted.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedorahosted.org/archives/list/[e-mail]...
>