May 2019 - sanlock-devel - Fedora Mailing-Lists

[PATCH] tests: Simplify 4k tests setup and teardown

by Nir Soffer

Previously you had to setup the environment manually to enable the 4k tests, and pass environment variable to tests. The tests used single path, so you had to run the tests twice to test both 4k block device and filesystem backed by a 4k block device. Add setup and teardown scripts to make it easier to enable the 4k tests. Run the setup scripts once to setup your environment: $ tests/setup.sh This script create both a 4k block device and filesystem backed by a 4k device. The user_4k_path fixture enables the tests if the device and mount exists, and run the test twice, once with the loop device, and once with the filesystem. To clean up the environment, run: $ tests/teardown.sh Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- The patch is also available in: https://github.com/nirs/sanlock/commits/4k README.dev | 45 +++++++++++++-------------------------------- tests/config.py | 4 ++++ tests/conftest.py | 33 +++++++++------------------------ tests/setup.sh | 41 +++++++++++++++++++++++++++++++++++++++++ tests/teardown.sh | 27 +++++++++++++++++++++++++++ 5 files changed, 94 insertions(+), 56 deletions(-) create mode 100644 tests/config.py create mode 100755 tests/setup.sh create mode 100755 tests/teardown.sh diff --git a/README.dev b/README.dev index bfb170c..64feb9a 100644 --- a/README.dev +++ b/README.dev @@ -27,44 +27,25 @@ To run only tests matching the substring "foo": Testing 4K support ================== -This requires manual setup for creating a block device using 4k sector size, -optionally creating and mounting a file system, and passing the test path to -tests. The setup must be done as root, but the tests can run later as the -current user. +To enable the 4k tests, you need to setup your environment. -Common setup ------------- + $ tests/setup.sh -1. Create a backing file: +This creates two loop devices with 4k sector size; one for testing +sanlock with 4k block device, and the other for testing with a +filesystem backed by a 4k block device. - $ truncate -s 1G /var/tmp/backing +To teardown your environment: -2. Create a loop device with 4k sector size: + $ tests/teardown.sh - $ sudo losetup -f /var/tmp/backing --show --sector-size=4096 - /dev/loop2 +The script unmounts the filesystem and detaches the loop devices. -3. Change the device (or mountpoint) owner to current user +Both scripts use configuraion values in tests/config.py. +This module is imported by the python test modules. - $ sudo chown $USER:$USER /dev/loop2 - -Testing 4k block device ------------------------ - -Run the tests with USER_4K_PATH environment variable: - - $ USER_4K_PATH=/dev/loop2 tox -e py27 - -Testing 4k file system ----------------------- - -To test file system on top of 4k block device, create a file system on the -device, mount it, and create a file for testing lockspace or resources: - - $ sudo mkfs.xfs /dev/loop2 - $ sudo mount /dev/loop2 /tmp/sanlock-4k - $ sudo chown $USER:$USER /tmp/sanlock-4k - $ truncate -s 128m /tmp/sanlock-4k/disk - $ USER_4K_PATH=/tmp/sanlock-4k/disk tox -e py27 +The scripts use sudo to perform privileged operations. The best way to +use them is to setup the environment once at the start of the session, +and teardown when you finish. diff --git a/tests/config.py b/tests/config.py new file mode 100644 index 0000000..23f3545 --- /dev/null +++ b/tests/config.py @@ -0,0 +1,4 @@ +# Test configuration, used both by the shell and python. + +base_dir="/var/tmp/sanlock-4k" +dev_size=1073741824 diff --git a/tests/conftest.py b/tests/conftest.py index bb95d45..8037a24 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -5,10 +5,11 @@ Fixtures for sanlock testing. import os import stat import pytest +from . import config from . import util @pytest.fixture def sanlock_daemon(): @@ -24,36 +25,20 @@ def sanlock_daemon(): # which takes about 3 seconds, slowing down the tests. p.kill() p.wait() -(a)pytest.fixture -def user_4k_path(): +(a)pytest.fixture(params=[ + pytest.param("loop1", id="block"), + pytest.param("mnt/file", id="file") +]) +def user_4k_path(request): """ A path to block device or file on file system on top of 4k block device, provided by the user. - The user must create the block device or the file system before running the - tests, and specify the path to the file in the USER_4K_PATH environment - variable. - - If USER_4K_PATH was not specified, tests using this fixture will be skipped. - If USER_4K_PATH was specified but does not exist, or is not a file or block - device, RuntimeError is raised. - - Return path to the user specified file. + If 4k tests were not enabled, skip the tests. """ - path = os.environ.get("USER_4K_PATH") - if path is None: - pytest.skip("USER_4K_PATH pointing to a 4k block device or file was " - "not specified") - + path = os.path.join(config.base_dir, request.param) if not os.path.exists(path): - raise RuntimeError("USER_4K_PATH {!r} does not exist".format(path)) - - mode = os.stat(path).st_mode - if not (stat.S_ISBLK(mode) or stat.S_ISREG(mode)): - raise RuntimeError( - "USER_4K_PATH {!r} is not a block device or regular file" - .format(path)) - + pytest.skip("4k tests disabled, run tests/setup.sh to enable") return path diff --git a/tests/setup.sh b/tests/setup.sh new file mode 100755 index 0000000..7f9046b --- /dev/null +++ b/tests/setup.sh @@ -0,0 +1,41 @@ +#!/bin/sh -e +# Run this once to setup your environment for testing sanlock. +# +# The script creates 2 loop devices, and link them to: +# +# $base_dir/loop{1,2} +# +# And mount a filesystem on: +# +# $base_dir/mnt +# +# The teardown script unmount and detach the loop devices and remove the links. +# + +. tests/config.py + +mkdir -p "$base_dir" + +if [ ! -e "$base_dir/loop1" ]; then + echo " Creating 4k loop device $base_dir/loop1" + truncate -s $dev_size "$base_dir/backing1" + loop1=$(sudo losetup -f "$base_dir/backing1" --show --sector-size 4096) + ln -s "$loop1" "$base_dir/loop1" + sudo chown $USER:$USER "$base_dir/loop1" +fi + +if [ ! -e "$base_dir/loop2" ]; then + echo " Creating 4k loop device $base_dir/loop2" + truncate -s $dev_size "$base_dir/backing2" + loop2=$(sudo losetup -f "$base_dir/backing2" --show --sector-size 4096) + ln -s "$loop2" "$base_dir/loop2" + + echo " Mounting 4k filesystem at $base_dir/mnt" + sudo mkfs.xfs -q "$base_dir/loop2" + mkdir -p "$base_dir/mnt" + sudo mount "$base_dir/loop2" "$base_dir/mnt" + sudo chown $USER:$USER "$base_dir/mnt" + touch "$base_dir/mnt/file" +fi + +# vim: set expandtab shiftwidth=4 tabstop=4 : diff --git a/tests/teardown.sh b/tests/teardown.sh new file mode 100755 index 0000000..9342e0a --- /dev/null +++ b/tests/teardown.sh @@ -0,0 +1,27 @@ +#!/bin/sh +# Run this to teardown your environment. +# +# The script undoes the actions perfomed in setup.sh. + +. tests/config.py + +if grep -q "$base_dir/mnt" /proc/self/mounts; then + echo " Umounting $base_dir/mnt" + sudo umount "$base_dir/mnt" +fi + +if [ -h "$base_dir/loop1" ]; then + echo " Detaching $base_dir/loop2" + sudo losetup -d "$base_dir/loop2" +fi + +if [ -h "$base_dir/loop2" ]; then + echo " Detaching $base_dir/loop1" + sudo losetup -d "$base_dir/loop1" +fi + +echo " Cleaning up $base_dir" +rm -f "$base_dir/loop"* +rm -f "$base_dir/backing"* + +# vim: set expandtab shiftwidth=4 tabstop=4 : -- 2.17.2

4 years, 11 months

3
4
0 / 0

[PATCH] gitignore: Add python related files

by Nir Soffer

Ignore files generated when building and running the tests. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- .gitignore | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/.gitignore b/.gitignore index dec4e89..d506b71 100644 --- a/.gitignore +++ b/.gitignore @@ -1,20 +1,27 @@ -*.swp *.co +*.swp +*.tar.gz +.tox/ .vimdir cscope.* -wdmd/libwdmd.so* -wdmd/wdmd -wdmd/wdmd_client +python/*.so +python/build/ +python/usr/ +reset/sanlk-reset +reset/sanlk-resetd +sanlock.spec +src/libsanlock.pc src/libsanlock.so* +src/libsanlock_client.pc src/libsanlock_client.so* src/sanlock -python/build/ -python/usr/ +tests/*.pyc +tests/__pycache__/ tests/devcount tests/killpath tests/sanlk_client tests/sanlk_load tests/sanlk_path -tests/__pycache__/ -sanlock.spec -*.tar +wdmd/libwdmd.so* +wdmd/wdmd +wdmd/wdmd_client -- 2.17.2

4 years, 11 months

1
0
0 / 0

[sanlock] branch master updated: python: Add vim modeline

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. The following commit(s) were added to refs/heads/master by this push: new 8ffc57a python: Add vim modeline 8ffc57a is described below commit 8ffc57aa276f6c3ae78d9287acba0f4d0a0ed3e9 Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Fri May 17 23:12:50 2019 +0300 python: Add vim modeline Sanlock is using tabs for indentation, but the python bindings is using spaces. Add a modeline to help keep the formatting consistent. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- python/sanlock.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/python/sanlock.c b/python/sanlock.c index 0b9a61e..e710f89 100644 --- a/python/sanlock.c +++ b/python/sanlock.c @@ -1760,3 +1760,5 @@ initsanlock(void) } } + +/* vim: set expandtab shiftwidth=4 tabstop=4 : */ -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

[PATCH 0/5] Preparation patches for python 3

by Nir Soffer

These patches should be included in sanlock 3.8, but they should be harmless if we include them in 3.7. Amit Bawer (5): readme: Add prerequisites section tox: Add python version as make argument tests: Use python3 and python2 compatible socket exception type tests: Lockspace and resource names should be read as bytes tests: Enforce absolute import behavior for python2 tests README.dev | 12 ++++++++++++ python/Makefile | 6 ++++-- sanlock.spec.in | 3 ++- tests/conftest.py | 1 + tests/daemon_test.py | 11 ++++++----- tests/direct_test.py | 1 + tests/python_test.py | 19 ++++++++++--------- tests/util.py | 5 +++-- tox.ini | 3 ++- 9 files changed, 41 insertions(+), 20 deletions(-) -- 2.17.2

4 years, 11 months

4
13
0 / 0

[sanlock] 06/06: sanlock: Fail read_resource_owners with incorrect align size

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. commit 9488ad21dc4f7612708cd15d97cde07b18e6bd08 Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Sun May 12 04:04:58 2019 +0300 sanlock: Fail read_resource_owners with incorrect align size Previously if sector size was not set, we modified both sector size and align size, dropping the caller align size silently. This can cause two issues: - User align size was correct, but sanlock guessed the wrong value - in this case sanlock logs a debug message and perform another read. - User align size was incorrect, but sanlock guess was correct, hiding a bug in caller code. Change the behaviour to be more consistent with the way we validate sector size, and reject invalid value set by the user. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- src/resource.c | 17 +++++++++++++---- tests/python_test.py | 1 - 2 files changed, 13 insertions(+), 5 deletions(-) diff --git a/src/resource.c b/src/resource.c index 21dbe84..76d506c 100644 --- a/src/resource.c +++ b/src/resource.c @@ -170,6 +170,7 @@ int read_resource_owners(struct task *task, struct token *token, char *lease_buf = NULL; char *hosts_buf = NULL; const int sector_size_set = token->sector_size != 0; + const int align_size_set = token->align_size != 0; int align_size; int host_count = 0; int i, rv; @@ -177,11 +178,12 @@ int read_resource_owners(struct task *task, struct token *token, disk = &token->disks[0]; /* If sector size not set, start with the smaller one. */ - - if (!sector_size_set) { + if (!sector_size_set) token->sector_size = 512; - token->align_size = sector_size_to_align_size_old(512); - } + + /* If align size not set, default to the older align. */ + if (!align_size_set) + token->align_size = sector_size_to_align_size_old(token->sector_size); /* we could in-line paxos_read_buf here like we do in read_mode_block */ retry: @@ -213,6 +215,13 @@ int read_resource_owners(struct task *task, struct token *token, goto out; } + if (align_size_set && token->align_size != align_size) { + log_errot(token, "read_resource_owners invalid align_size: %d actual: %d", + token->align_size, align_size); + rv = -EINVAL; + goto out; + } + /* * If the caller did not specify sector size and our guess was wrong, * retry with the actual value diff --git a/tests/python_test.py b/tests/python_test.py index 49460b5..12358c7 100644 --- a/tests/python_test.py +++ b/tests/python_test.py @@ -237,7 +237,6 @@ def test_read_resource_owners_4k_invalid_sector_size( assert e.value.errno == errno.EINVAL -(a)pytest.mark.xfail(reason="fallback hides invalid user value") def test_read_resource_owners_invalid_align_size(tmpdir, sanlock_daemon): path = str(tmpdir.join("path")) util.create_file(path, 1024**3) -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

[sanlock] 05/06: tests: Add failing test for invalid align size

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. commit 4de75bcb5d590af75157d3ce86ae50eed426100a Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Sun May 12 04:19:28 2019 +0300 tests: Add failing test for invalid align size sanlock_read_resource_owners() fallback mechanism hides invalid align size specified by the caller. Add a test reproducing this issue, and specifying the wanted behaviour. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- tests/python_test.py | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/tests/python_test.py b/tests/python_test.py index d70187c..49460b5 100644 --- a/tests/python_test.py +++ b/tests/python_test.py @@ -23,7 +23,8 @@ LARGE_FILE_SIZE = 1024**4 LOCKSPACE_SIZE = 1024**2 MIN_RES_SIZE = 1024**2 -ALIGNMENT_1M = 1024**2 +ALIGNMENT_1M = 1 * 1024**2 +ALIGNMENT_2M = 2 * 1024**2 SECTOR_SIZE_512 = 512 SECTOR_SIZE_4K = 4096 @@ -236,6 +237,29 @@ def test_read_resource_owners_4k_invalid_sector_size( assert e.value.errno == errno.EINVAL +(a)pytest.mark.xfail(reason="fallback hides invalid user value") +def test_read_resource_owners_invalid_align_size(tmpdir, sanlock_daemon): + path = str(tmpdir.join("path")) + util.create_file(path, 1024**3) + disks = [(path, 0)] + + sanlock.write_resource( + "ls_name", + "res_name", + disks, + align=ALIGNMENT_1M, + sector=SECTOR_SIZE_512) + + with pytest.raises(sanlock.SanlockException) as e: + sanlock.read_resource_owners( + "ls_name", + "res_name", + disks, + align=ALIGNMENT_2M, + sector=SECTOR_SIZE_512) + assert e.value.errno == errno.EINVAL + + @pytest.mark.parametrize("size,offset", [ # Smallest offset. (MIN_RES_SIZE, 0), -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

[sanlock] 04/06: sanlock: Fail read_resource_owners() with wrong sector size

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. commit 1c8f054606a24988fdf6417b4d7b578fb482bc9a Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Tue May 7 01:05:27 2019 +0300 sanlock: Fail read_resource_owners() with wrong sector size If sanlock_read_resource_owners() is called with wrong sector size, fail instead of falling back to actual sector size. This is more consistent with sanlock_read_resource() and other other functions. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- src/resource.c | 22 ++++++++++++++++------ tests/python_test.py | 1 - 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/src/resource.c b/src/resource.c index a783106..21dbe84 100644 --- a/src/resource.c +++ b/src/resource.c @@ -169,18 +169,16 @@ int read_resource_owners(struct task *task, struct token *token, char *lease_buf_dblock; char *lease_buf = NULL; char *hosts_buf = NULL; + const int sector_size_set = token->sector_size != 0; int align_size; int host_count = 0; int i, rv; disk = &token->disks[0]; - /* - * We don't know the sector_size of the resource until the leader - * record has been read, start with the smaller size. - */ + /* If sector size not set, start with the smaller one. */ - if (!token->sector_size) { + if (!sector_size_set) { token->sector_size = 512; token->align_size = sector_size_to_align_size_old(512); } @@ -206,9 +204,21 @@ int read_resource_owners(struct task *task, struct token *token, if (!align_size) align_size = sector_size_to_align_size_old(leader.sector_size); + /* If caller specified values are incorrect, fail. */ + + if (sector_size_set && token->sector_size != leader.sector_size) { + log_errot(token, "read_resource_owners invalid sector_size: %d actual: %d", + token->sector_size, leader.sector_size); + rv = -EINVAL; + goto out; + } + + /* + * If the caller did not specify sector size and our guess was wrong, + * retry with the actual value + */ if ((token->sector_size != leader.sector_size) || (token->align_size != align_size)) { - /* initial sizes were wrong */ log_debug("read_resource_owners rereading with correct sizses"); token->sector_size = leader.sector_size; token->align_size = align_size; diff --git a/tests/python_test.py b/tests/python_test.py index edb6de7..d70187c 100644 --- a/tests/python_test.py +++ b/tests/python_test.py @@ -219,7 +219,6 @@ def test_read_resource_4k_invalid_sector_size(sanlock_daemon, user_4k_path): assert e.value.errno == errno.EINVAL -(a)pytest.mark.xfail(reason="fallback hides wrong value from caller") def test_read_resource_owners_4k_invalid_sector_size( sanlock_daemon, user_4k_path): disks = [(user_4k_path, 0)] -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

[sanlock] 03/06: tests: Test wrong sector size with 4k storage

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. commit d30a6d6282fa824beda2e7f67cb6b7a4720d954d Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Tue May 7 00:56:05 2019 +0300 tests: Test wrong sector size with 4k storage When using 4k disk and sector=512, we expect sanlock to fail with SanlockException(EINVAL) when writing or reading. Two tests are marked as expected failure (xfail): - write_resource() - not sure how sanlock can succeed with wrong sector size - this may be a bug. - read_resource_owners() - sanlock uses a fallback mechanism, hiding wrong value from the user. I think we can make sanlock more strict. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- tests/python_test.py | 59 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 59 insertions(+) diff --git a/tests/python_test.py b/tests/python_test.py index eb93762..edb6de7 100644 --- a/tests/python_test.py +++ b/tests/python_test.py @@ -96,6 +96,22 @@ def test_write_lockspace_4k(user_4k_path, sanlock_daemon, align): util.check_guard(user_4k_path, align) +def test_write_lockspace_4k_invalid_sector_size(sanlock_daemon, user_4k_path): + with pytest.raises(sanlock.SanlockException) as e: + sanlock.write_lockspace( + "name", user_4k_path, iotimeout=1, sector=SECTOR_SIZE_512) + assert e.value.errno == errno.EINVAL + + +def test_read_lockspace_4k_invalid_sector_size(sanlock_daemon, user_4k_path): + sanlock.write_lockspace( + "name", user_4k_path, iotimeout=1, sector=SECTOR_SIZE_4K) + + with pytest.raises(sanlock.SanlockException) as e: + sanlock.read_lockspace(user_4k_path, sector=SECTOR_SIZE_512) + assert e.value.errno == errno.EINVAL + + @pytest.mark.parametrize("size,offset", [ # Smallest offset. (MIN_RES_SIZE, 0), @@ -178,6 +194,49 @@ def test_write_resource_4k(sanlock_daemon, user_4k_path, align): util.check_guard(user_4k_path, align) +(a)pytest.mark.xfail(reason="need to investigate why the call succeed") +def test_write_resource_4k_invalid_sector_size(sanlock_daemon, user_4k_path): + disks = [(user_4k_path, 0)] + + with pytest.raises(sanlock.SanlockException) as e: + sanlock.write_resource( + "ls_name", "res_name", disks, sector=SECTOR_SIZE_512) + assert e.value.errno == errno.EINVAL + + +def test_read_resource_4k_invalid_sector_size(sanlock_daemon, user_4k_path): + disks = [(user_4k_path, 0)] + + sanlock.write_resource( + "ls_name", + "res_name", + disks, + align=ALIGNMENT_1M, + sector=SECTOR_SIZE_4K) + + with pytest.raises(sanlock.SanlockException) as e: + sanlock.read_resource(user_4k_path, sector=SECTOR_SIZE_512) + assert e.value.errno == errno.EINVAL + + +(a)pytest.mark.xfail(reason="fallback hides wrong value from caller") +def test_read_resource_owners_4k_invalid_sector_size( + sanlock_daemon, user_4k_path): + disks = [(user_4k_path, 0)] + + sanlock.write_resource( + "ls_name", + "res_name", + disks, + align=ALIGNMENT_1M, + sector=SECTOR_SIZE_4K) + + with pytest.raises(sanlock.SanlockException) as e: + sanlock.read_resource_owners( + "ls_name", "res_name", disks, sector=SECTOR_SIZE_512) + assert e.value.errno == errno.EINVAL + + @pytest.mark.parametrize("size,offset", [ # Smallest offset. (MIN_RES_SIZE, 0), -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

[sanlock] 02/06: tests: Start tests for 4k storage

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. commit 7ba5700dc1c08735da0dd897bcf94ae30d294af1 Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Sun May 5 17:03:21 2019 +0300 tests: Start tests for 4k storage We added API for using 4k sector size and various alignments, but there are no tests for the new APIs with disks using 4k sector size. Testing this is complicated since creating a block device and mounting file systems requires root, and we like to run the tests as a regular user. Add new tests for writing and reading lockspace and resource using 4k storage provided by the user. If a path is not specified the tests are skipped. To run the tests, you need to create a block device using 4k sector size and optionally create a file system and mount it. The process is explained in README.dev. Then pass the path of the block device or file on the new file system to the tests: USER_4K_PATH=/dev/loop2 tox -e py27 Adding and removing lockspace and acquiring resources is not tested yet with 4k storage. Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- README.dev | 44 +++++++++++++++++++++++++++++++++++ tests/conftest.py | 36 +++++++++++++++++++++++++++++ tests/python_test.py | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++ tests/util.py | 34 ++++++++++++++++++--------- tox.ini | 2 +- 5 files changed, 169 insertions(+), 12 deletions(-) diff --git a/README.dev b/README.dev index 3519993..bfb170c 100644 --- a/README.dev +++ b/README.dev @@ -24,3 +24,47 @@ To run only test from some modules: To run only tests matching the substring "foo": $ tox -- -k foo + + +Testing 4K support +================== + +This requires manual setup for creating a block device using 4k sector size, +optionally creating and mounting a file system, and passing the test path to +tests. The setup must be done as root, but the tests can run later as the +current user. + +Common setup +------------ + +1. Create a backing file: + + $ truncate -s 1G /var/tmp/backing + +2. Create a loop device with 4k sector size: + + $ sudo losetup -f /var/tmp/backing --show --sector-size=4096 + /dev/loop2 + +3. Change the device (or mountpoint) owner to current user + + $ sudo chown $USER:$USER /dev/loop2 + +Testing 4k block device +----------------------- + +Run the tests with USER_4K_PATH environment variable: + + $ USER_4K_PATH=/dev/loop2 tox -e py27 + +Testing 4k file system +---------------------- + +To test file system on top of 4k block device, create a file system on the +device, mount it, and create a file for testing lockspace or resources: + + $ sudo mkfs.xfs /dev/loop2 + $ sudo mount /dev/loop2 /tmp/sanlock-4k + $ sudo chown $USER:$USER /tmp/sanlock-4k + $ truncate -s 128m /tmp/sanlock-4k/disk + $ USER_4K_PATH=/tmp/sanlock-4k/disk tox -e py27 diff --git a/tests/conftest.py b/tests/conftest.py index 6b8b0e8..bb95d45 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -2,6 +2,9 @@ Fixtures for sanlock testing. """ +import os +import stat + import pytest from . import util @@ -21,3 +24,36 @@ def sanlock_daemon(): # which takes about 3 seconds, slowing down the tests. p.kill() p.wait() + + +(a)pytest.fixture +def user_4k_path(): + """ + A path to block device or file on file system on top of 4k block device, + provided by the user. + + The user must create the block device or the file system before running the + tests, and specify the path to the file in the USER_4K_PATH environment + variable. + + If USER_4K_PATH was not specified, tests using this fixture will be skipped. + If USER_4K_PATH was specified but does not exist, or is not a file or block + device, RuntimeError is raised. + + Return path to the user specified file. + """ + path = os.environ.get("USER_4K_PATH") + if path is None: + pytest.skip("USER_4K_PATH pointing to a 4k block device or file was " + "not specified") + + if not os.path.exists(path): + raise RuntimeError("USER_4K_PATH {!r} does not exist".format(path)) + + mode = os.stat(path).st_mode + if not (stat.S_ISBLK(mode) or stat.S_ISREG(mode)): + raise RuntimeError( + "USER_4K_PATH {!r} is not a block device or regular file" + .format(path)) + + return path diff --git a/tests/python_test.py b/tests/python_test.py index 1a6bbbd..eb93762 100644 --- a/tests/python_test.py +++ b/tests/python_test.py @@ -25,6 +25,7 @@ MIN_RES_SIZE = 1024**2 ALIGNMENT_1M = 1024**2 SECTOR_SIZE_512 = 512 +SECTOR_SIZE_4K = 4096 @pytest.mark.parametrize("size,offset", [ @@ -66,6 +67,35 @@ def test_write_lockspace(tmpdir, sanlock_daemon, size, offset): util.check_guard(path, size) +(a)pytest.mark.parametrize("align", sanlock.ALIGN_SIZE) +def test_write_lockspace_4k(user_4k_path, sanlock_daemon, align): + + # Poison lockspace area, ensuring that previous tests will not break this + # test, and sanlock does not write beyond the lockspace area. + with io.open(user_4k_path, "rb+") as f: + f.write(align * b"x") + util.write_guard(user_4k_path, align) + + sanlock.write_lockspace( + "name", user_4k_path, iotimeout=1, align=align, sector=SECTOR_SIZE_4K) + + ls = sanlock.read_lockspace( + user_4k_path, align=align, sector=SECTOR_SIZE_4K) + + assert ls == {"iotimeout": 1, "lockspace": "name"} + + acquired = sanlock.inq_lockspace("name", 1, user_4k_path, wait=False) + assert acquired is False + + # Verify that lockspace was written. + with io.open(user_4k_path, "rb") as f: + magic, = struct.unpack("< I", f.read(4)) + assert magic == constants.DELTA_DISK_MAGIC + + # Check that sanlock did not write beyond the lockspace area. + util.check_guard(user_4k_path, align) + + @pytest.mark.parametrize("size,offset", [ # Smallest offset. (MIN_RES_SIZE, 0), @@ -113,6 +143,41 @@ def test_write_resource(tmpdir, sanlock_daemon, size, offset): util.check_guard(path, size) +(a)pytest.mark.parametrize("align", sanlock.ALIGN_SIZE) +def test_write_resource_4k(sanlock_daemon, user_4k_path, align): + disks = [(user_4k_path, 0)] + + # Poison resource area, ensuring that previous tests will not break this + # test, and sanlock does not write beyond the lockspace area. + with io.open(user_4k_path, "rb+") as f: + f.write(align * b"x") + util.write_guard(user_4k_path, align) + + sanlock.write_resource( + "ls_name", "res_name", disks, align=align, sector=SECTOR_SIZE_4K) + + res = sanlock.read_resource( + user_4k_path, align=align, sector=SECTOR_SIZE_4K) + + assert res == { + "lockspace": "ls_name", + "resource": "res_name", + "version": 0 + } + + owners = sanlock.read_resource_owners( + "ls_name", "res_name", disks, align=align, sector=SECTOR_SIZE_4K) + assert owners == [] + + # Verify that resource was written. + with io.open(user_4k_path, "rb") as f: + magic, = struct.unpack("< I", f.read(4)) + assert magic == constants.PAXOS_DISK_MAGIC + + # Check that sanlock did not write beyond the lockspace area. + util.check_guard(user_4k_path, align) + + @pytest.mark.parametrize("size,offset", [ # Smallest offset. (MIN_RES_SIZE, 0), diff --git a/tests/util.py b/tests/util.py index a6b53e3..0695d8c 100644 --- a/tests/util.py +++ b/tests/util.py @@ -13,6 +13,9 @@ import time TESTDIR = os.path.dirname(__file__) SANLOCK = os.path.join(TESTDIR, os.pardir, "src", "sanlock") +GUARD = b"X" +GUARD_SIZE = 4096 + class TimeoutExpired(Exception): """ Raised when timeout expired """ @@ -97,28 +100,37 @@ def wait_for_termination(p, timeout): time.sleep(0.05) -def create_file(path, size, guard=b"X", guard_size=4096): +def create_file(path, size, guard=True): """ Create sparse file of size bytes. - If guard is set, add a guard area after the end of the file and fill it - with guard bytes. This allows testing that the code under test do not write - anything after the end of the file. + If guard is True, add a guard area beyond the end of the file. """ with io.open(path, "wb") as f: f.truncate(size) - if guard: - f.seek(size) - f.write(guard * guard_size) + + if guard: + write_guard(path, size) + + +def write_guard(path, offset): + """ + Write guard areas at offset and fill with guard byte. + + Use check_guard() to verify that nothing was written to the guard area. + """ + with io.open(path, "rb+") as f: + f.seek(offset) + f.write(GUARD * GUARD_SIZE) -def check_guard(path, size, guard=b"X", guard_size=4096): +def check_guard(path, offset): """ - Assert that a file ends with a guard area filled with guard bytes. + Assert that guard area at offset was not modified. """ with io.open(path, "rb") as f: - f.seek(size) - assert f.read() == guard * guard_size + f.seek(offset) + assert f.read(GUARD_SIZE) == GUARD * GUARD_SIZE def check_rindex_entry(entry, name, offset=None, flags=None): diff --git a/tox.ini b/tox.ini index 1c2349e..d21f2f3 100644 --- a/tox.ini +++ b/tox.ini @@ -9,7 +9,7 @@ skipsdist = True skip_missing_interpreters = True [testenv] -passenv = USER +passenv = * setenv = LD_LIBRARY_PATH={env:PWD}/wdmd:{env:PWD}/src SANLOCK_PRIVILEGED=0 -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

[sanlock] 01/06: python: Add align and sector arguments to read_resource_owners()

by pagure＠pagure.io

This is an automated email from the git hooks/post-receive script. nsoffer pushed a commit to branch master in repository sanlock. commit 67b246873ff433c332d757ba3af3272930b64d8f Author: Nir Soffer <nsoffer(a)redhat.com> AuthorDate: Tue May 7 00:49:05 2019 +0300 python: Add align and sector arguments to read_resource_owners() We forgot to add the flags to this function, and it works with 4k storage because sanlock implements a fallback mechanism. Since we require align and sector arguments in read_resource(), we should also require them for consistency in read_resource_owners(). Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- python/example.py | 2 +- python/sanlock.c | 24 +++++++++++++++++++----- 2 files changed, 20 insertions(+), 6 deletions(-) diff --git a/python/example.py b/python/example.py index cb8f3e9..6ef9f7d 100644 --- a/python/example.py +++ b/python/example.py @@ -54,7 +54,7 @@ def main(): time.sleep(5) print "Resource '%s' owners: " % RESOURCE_NAME, \ sanlock.read_resource_owners( - LOCKSPACE_NAME, RESOURCE_NAME, SNLK_DISKS) + LOCKSPACE_NAME, RESOURCE_NAME, SNLK_DISKS, align=1048576, sector=512) print "Releasing '%s' on '%s'" % (RESOURCE_NAME, LOCKSPACE_NAME) sanlock.release(LOCKSPACE_NAME, RESOURCE_NAME, SNLK_DISKS, slkfd=fd) except Exception as e: diff --git a/python/sanlock.c b/python/sanlock.c index d6841e8..0b9a61e 100644 --- a/python/sanlock.c +++ b/python/sanlock.c @@ -1197,26 +1197,32 @@ exit_fail: /* read_resource_owners */ PyDoc_STRVAR(pydoc_read_resource_owners, "\ -read_resource_owners(lockspace, resource, disks) -> list\n\ +read_resource_owners(lockspace, resource, disks, align=1048576, sector=512) \ +-> list\n\ Returns the list of hosts owning a resource, the list is not filtered and\n\ it might contain hosts that are currently failing or dead. The hosts are\n\ returned in the same format used by get_hosts.\n\ -The disks must be in the format: [(path, offset), ... ]"); +The disks must be in the format: [(path, offset), ... ].\n\ +Align can be one of (1048576, 2097152, 4194304, 8388608).\n\ +Sector can be one of (512, 4096)."); static PyObject * py_read_resource_owners(PyObject *self __unused, PyObject *args, PyObject *keywds) { int rv, hss_count = 0; + int sector = SECTOR_SIZE_512; + long align = ALIGNMENT_1M; const char *lockspace, *resource; struct sanlk_resource *res = NULL; struct sanlk_host *hss = NULL; PyObject *disks, *ls_list = NULL; - static char *kwlist[] = {"lockspace", "resource", "disks", NULL}; + static char *kwlist[] = {"lockspace", "resource", "disks", "align", + "sector", NULL}; /* parse python tuple */ - if (!PyArg_ParseTupleAndKeywords(args, keywds, "ssO!", kwlist, - &lockspace, &resource, &PyList_Type, &disks)) { + if (!PyArg_ParseTupleAndKeywords(args, keywds, "ssO!|li", kwlist, + &lockspace, &resource, &PyList_Type, &disks, &align, &sector)) { return NULL; } @@ -1229,6 +1235,14 @@ py_read_resource_owners(PyObject *self __unused, PyObject *args, PyObject *keywd strncpy(res->lockspace_name, lockspace, SANLK_NAME_LEN); strncpy(res->name, resource, SANLK_NAME_LEN); + /* set resource alignment and sector flags */ + + if (add_align_flag(align, &res->flags) == -1) + goto exit_fail; + + if (add_sector_flag(sector, &res->flags) == -1) + goto exit_fail; + /* read resource owners (gil disabled) */ Py_BEGIN_ALLOW_THREADS rv = sanlock_read_resource_owners(res, 0, &hss, &hss_count); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

4 years, 11 months

1
0
0 / 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

sanlock-devel May 2019