vdsm-patches August 2015

vdsm-patches@lists.fedorahosted.org

47 participants
674 discussions

Change in vdsm[master]: iscsicred: Use local iscsi credentials database
by Nir Soffer 05 Aug '15

05 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: iscsicred: Use local iscsi credentials database ...................................................................... iscsicred: Use local iscsi credentials database When discovering send targets or logging in to targets, use the local iscsi credentials database. Change-Id: I065619f31fbefeb5b96bc86ed9da81ce17c1c47d Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 24 insertions(+), 12 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/58/43258/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 1b9de05..a236b23 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -57,6 +57,7 @@ import image import volume import iscsi +import iscsicred import misc from misc import deprecated import taskManager @@ -234,17 +235,34 @@ conDict.get('initiatorName', None), conDict.get('netIfaceName', None)) - cred = None - username = conDict.get('user', None) - password = conDict.get('password', None) - if username or password: - cred = iscsi.ChapCredentials(username, password) + cred = _iscsi_chap_credentials(conDict) params = storageServer.IscsiConnectionParameters(target, iface, cred) else: raise se.StorageServerActionError() return storageServer.ConnectionInfo(typeName, params) + + +def _iscsi_chap_credentials(con): + """ + Return iscsi.ChapCredentials from local credentials database of from + connection info. + + TODO: prefer more specific credentials over less specific: + iqn > ip:port > host > engine. + """ + try: + credentials = iscsicred.get_credentials('host') + return iscsi.ChapCredentials(credentials['username'], + credentials['password']) + except iscsicred.NotFound: + username = con.get('user') + if username: + password = con.get('password') + return iscsi.ChapCredentials(username, password) + + return None class HSM(object): @@ -3028,16 +3046,10 @@ """ ip = con['connection'] port = int(con['port']) - username = con['user'] - password = con['password'] - if username == "": - username = password = None iface = iscsi.IscsiInterface("default") portal = iscsi.IscsiPortal(ip, port) - cred = None - if username or password: - cred = iscsi.ChapCredentials(username, password) + cred = _iscsi_chap_credentials(con) try: targets = iscsi.discoverSendTargets(iface, portal, cred) -- To view, visit https://gerrit.ovirt.org/43258 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I065619f31fbefeb5b96bc86ed9da81ce17c1c47d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 16

Change in vdsm[master]: iscsicred: Install the iscsi-cred directory
by Nir Soffer 05 Aug '15

05 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: iscsicred: Install the iscsi-cred directory ...................................................................... iscsicred: Install the iscsi-cred directory This directory will be populated with iscsi target credential files by the system administrator. Change-Id: Iea9359291a4bea9fa0d93d52f44f1e0af5235a4c Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M debian/vdsm.dirs M debian/vdsm.postinst M vdsm.spec.in M vdsm/Makefile.am 4 files changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/57/43257/1 diff --git a/debian/vdsm.dirs b/debian/vdsm.dirs index ee2bccd..3fd61f5 100644 --- a/debian/vdsm.dirs +++ b/debian/vdsm.dirs @@ -5,6 +5,7 @@ etc/pki/vdsm/libvirt-spice etc/vdsm etc/vdsm/mom.d +etc/vdsm/iscsi-cred rhev rhev/data-center rhev/data-center/hsm-tasks diff --git a/debian/vdsm.postinst b/debian/vdsm.postinst index 5458974..88644cb 100644 --- a/debian/vdsm.postinst +++ b/debian/vdsm.postinst @@ -66,6 +66,7 @@ $INSTALLDIR /var/run/vdsm/trackedInterfaces $INSTALLDIR /var/log/vdsm $INSTALLDIR /var/log/vdsm/backup + install -dDm 0700 /etc/vdsm/iscsi-cred chmod g+x /var/lib/libvirt/qemu chmod 644 /etc/iscsi/initiatorname.iscsi diff --git a/vdsm.spec.in b/vdsm.spec.in index de15c12..733a6e9 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -805,6 +805,8 @@ %dir %{_libexecdir}/%{vdsm_name} %dir %{_sysconfdir}/%{vdsm_name} %dir %{_sysconfdir}/%{vdsm_name}/mom.d +%dir %{_sysconfdir}/%{vdsm_name}/iscsi-cred +%attr(0700, root, root) %{_sysconfdir}/%{vdsm_name}/iscsi-cred %dir %{_datadir}/%{vdsm_name} %dir %{_datadir}/%{vdsm_name}/network %dir %{_datadir}/%{vdsm_name}/network/configurators diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 691fb8b..866ee6b 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -158,6 +158,7 @@ $(MKDIR_P) $(DESTDIR)$(vdsmlibdir)/upgrade $(MKDIR_P) $(DESTDIR)$(vdsmbackupdir) $(MKDIR_P) $(DESTDIR)$(localstatedir)/lib/libvirt/qemu/channels + $(MKDIR_P) $(DESTDIR)$(vdsmconfdir)/iscsi-cred uninstall-local: \ uninstall-data-bonding-defaults \ -- To view, visit https://gerrit.ovirt.org/43257 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iea9359291a4bea9fa0d93d52f44f1e0af5235a4c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 9

Change in vdsm[master]: iscsicred: Support per-host iscsi credentials database
by Nir Soffer 05 Aug '15

05 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: iscsicred: Support per-host iscsi credentials database ...................................................................... iscsicred: Support per-host iscsi credentials database Some users need unique host iSCSI credentials for each target, but Engine supports only same credentials for all hosts. This series adds support for simple per-host iSCSI credentials database. Credentials are stored in /etc/vdsm/iscsi-cred/<target-name> Credentials file format: username = foo:bar password = 12345678 This patch adds the iscsicred module, providing readonly access to the iSCSI credentials database. Change-Id: I8f6a838f6b8e132d6b0c1a8135f2c28ef1e7f847 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M debian/vdsm.install M tests/Makefile.am A tests/iscsicred_test.py M vdsm.spec.in M vdsm/storage/Makefile.am A vdsm/storage/iscsicred.py M vdsm/supervdsmServer 7 files changed, 207 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/79/43179/1 diff --git a/debian/vdsm.install b/debian/vdsm.install index 35fcba7..c7cc90e 100644 --- a/debian/vdsm.install +++ b/debian/vdsm.install @@ -110,6 +110,7 @@ ./usr/share/vdsm/storage/imageSharing.py ./usr/share/vdsm/storage/iscsi.py ./usr/share/vdsm/storage/iscsiadm.py +./usr/share/vdsm/storage/iscsicred.py ./usr/share/vdsm/storage/localFsSD.py ./usr/share/vdsm/storage/lvm.env ./usr/share/vdsm/storage/lvm.py diff --git a/tests/Makefile.am b/tests/Makefile.am index b9b8e72..30ee221 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -56,6 +56,7 @@ iproute2Tests.py \ ipwrapperTests.py \ iscsiTests.py \ + iscsicred_test.py \ jsonRpcHelper.py \ jsonRpcTests.py \ libvirtconnectionTests.py \ diff --git a/tests/iscsicred_test.py b/tests/iscsicred_test.py new file mode 100644 index 0000000..715dded --- /dev/null +++ b/tests/iscsicred_test.py @@ -0,0 +1,121 @@ +# +# Copyright 2015 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +from contextlib import contextmanager +import os + +from testlib import VdsmTestCase +from testlib import namedTemporaryDir +from monkeypatch import MonkeyPatchScope, MonkeyClass + +from vdsm.password import ProtectedPassword +from storage import iscsicred + + +class FakeSupervdsm(object): + + def getProxy(self): + return self + + def readTargetCredfile(self, targetName): + return iscsicred._read_credfile(targetName) + + +@MonkeyClass(iscsicred, "supervdsm", FakeSupervdsm()) +class IscsicredTests(VdsmTestCase): + + TARGET = "iqn.1994-05.com.redhat:target7" + + def test_not_found(self): + self.assertRaises(iscsicred.NotFound, iscsicred.get_credentials, + self.TARGET) + + def test_simple(self): + data = "".join(["username=foo:bar\n", + "password=12345678\n"]) + with credfile(self.TARGET, data): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {"username": "foo:bar", + "password": ProtectedPassword("12345678")}) + + def test_whitespace(self): + data = "".join([" username = foo:bar \n", + " password = 12345678 \n"]) + with credfile(self.TARGET, data): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {"username": "foo:bar", + "password": ProtectedPassword("12345678")}) + + def test_any_whitespace(self): + data = "".join(["\t\tusername\t\t=\t\tfoo:bar\t\t\n", + "\t\tpassword\t\t=\t\t12345678\t\t\n"]) + with credfile(self.TARGET, data): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {"username": "foo:bar", + "password": ProtectedPassword("12345678")}) + + def test_empty_lines(self): + data = "".join(["\n", + "username=foo:bar\n", + "\n", + "password=12345678\n", + "\n"]) + with credfile(self.TARGET, data): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {"username": "foo:bar", + "password": ProtectedPassword("12345678")}) + + def test_no_eol(self): + data = "".join(["username=foo:bar\n", + "password=12345678"]) + with credfile(self.TARGET, data): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {"username": "foo:bar", + "password": ProtectedPassword("12345678")}) + + def test_extra_key(self): + data = "".join(["username=foo:bar\n", + "password=12345678\n" + "extra=value\n"]) + with credfile(self.TARGET, data): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {"username": "foo:bar", + "password": ProtectedPassword("12345678"), + "extra": "value"}) + + def test_empty(self): + with credfile(self.TARGET, ""): + d = iscsicred.get_credentials(self.TARGET) + self.assertEqual(d, {}) + + def test_invalid(self): + with credfile(self.TARGET, "invalid line\n"): + with self.assertRaises(iscsicred.InvalidCredfile): + iscsicred.get_credentials(self.TARGET) + + +@contextmanager +def credfile(target, data): + with namedTemporaryDir() as tmpdir: + with MonkeyPatchScope([(iscsicred, "DIR", tmpdir)]): + filename = os.path.join(tmpdir, target) + with open(filename, "wb") as f: + f.write(data) + yield diff --git a/vdsm.spec.in b/vdsm.spec.in index 58bf5ef..844a041 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -976,6 +976,7 @@ %{_datadir}/%{vdsm_name}/storage/image.py* %{_datadir}/%{vdsm_name}/storage/imageSharing.py* %{_datadir}/%{vdsm_name}/storage/iscsiadm.py* +%{_datadir}/%{vdsm_name}/storage/iscsicred.py* %{_datadir}/%{vdsm_name}/storage/iscsi.py* %{_datadir}/%{vdsm_name}/storage/localFsSD.py* %{_datadir}/%{vdsm_name}/storage/lvm.env diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am index 4ebe0a2..5e64c44 100644 --- a/vdsm/storage/Makefile.am +++ b/vdsm/storage/Makefile.am @@ -42,6 +42,7 @@ image.py \ imageSharing.py \ iscsiadm.py \ + iscsicred.py \ iscsi.py \ localFsSD.py \ lvm.py \ diff --git a/vdsm/storage/iscsicred.py b/vdsm/storage/iscsicred.py new file mode 100644 index 0000000..7e6bfa3 --- /dev/null +++ b/vdsm/storage/iscsicred.py @@ -0,0 +1,76 @@ +# +# Copyright 2015 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +""" +iscsicred - access iscsi credetials database + +This module provides readonly access to per-host iscsi credentails. Each host +in a cluster may be configured with unique credentatils for each iscsi target. +""" + +import errno +import os +from vdsm import password +import supervdsm + +DIR = "/etc/vdsm/iscsi-cred" + + +class NotFound(Exception): + """ Raised when getting non-existent cred file """ + + +class InvalidCredfile(Exception): + """ Raised if cred file cannot be parsed """ + + +def get_credentials(target_name): + data = supervdsm.getProxy().readTargetCredfile(target_name) + return _parse_credfile(data) + + +def _read_credfile(target_name): + """ + Read /etc/vdsm/iscsi-cred/target_name. + + Called from supervdsm since credfiles are root-readonly files. + """ + filename = os.path.join(DIR, target_name) + try: + with open(filename, "rb") as f: + return f.read() + except IOError as e: + if e.errno != errno.ENOENT: + raise + raise NotFound(target_name) + + +def _parse_credfile(data): + cred = {} + for line in data.splitlines(): + line = line.rstrip() + if not line: + continue + if "=" not in line: + raise InvalidCredfile("Invalid line: %r" % line) + key, value = [s.strip() for s in line.split("=", 1)] + if key == "password": + value = password.ProtectedPassword(value) + cred[key] = value + return cred diff --git a/vdsm/supervdsmServer b/vdsm/supervdsmServer index 36c67b1..1d2dd41 100755 --- a/vdsm/supervdsmServer +++ b/vdsm/supervdsmServer @@ -69,6 +69,7 @@ from storage.iscsi import readSessionInfo as _readSessionInfo from supervdsm import _SuperVdsmManager from storage import hba +from storage import iscsicred from storage import multipath from storage.fileUtils import chown, resolveGid, resolveUid from storage.fileUtils import validateAccess as _validateAccess @@ -167,6 +168,11 @@ return _readSessionInfo(sessionID) @logDecorator + def readTargetCredfile(self, target): + data = iscsicred._read_credfile(target) + return password.ProtectedPassword(data) + + @logDecorator def getPathsStatus(self): return _getPathsStatus() -- To view, visit https://gerrit.ovirt.org/43179 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8f6a838f6b8e132d6b0c1a8135f2c28ef1e7f847 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

5 24

Change in vdsm[master]: tests: Enhance permutations tests
by Nir Soffer 05 Aug '15

05 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: tests: Enhance permutations tests ...................................................................... tests: Enhance permutations tests - Move nested class to module to avoid clumsy self.Class syntax. We don't really need to have a nested class since it is not a TestCase class. - Test instance methods instead of class functions - Separate tests for unrelated features. We don't want tests for expanded method to fail because we did not remove the original method. - Use hasattr() for attribute testing - Add test for sub-classing permuted class. Change-Id: I3e28fb348e688970611ddd2e66ed6869047b5317 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/testlibTests.py 1 file changed, 49 insertions(+), 15 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/14/44214/1 diff --git a/tests/testlibTests.py b/tests/testlibTests.py index d631aaf..0e6f604 100644 --- a/tests/testlibTests.py +++ b/tests/testlibTests.py @@ -130,22 +130,56 @@ pass +@expandPermutations +class Permutated(object): + + @permutations([[1, 2], [3, 4]]) + def fn(self, a, b): + return a, b + + +@expandPermutations +class SubPermuated(Permutated): + + @permutations([[1], [2]]) + def fn2(self, param): + return param + + class TestPermutationExpansion(VdsmTestCase): - @expandPermutations - class _Permutations(object): - @permutations([[False], [True]]) - def fn(self, param): - return param - def test_expansion(self): - self.assertNotIn('fn', dir(self._Permutations)) - self.assertIn('fn(False)', dir(self._Permutations)) - self.assertIn('fn(True)', dir(self._Permutations)) + def setUp(self): + self.instance = Permutated() - def test_parameter_values(self): - self.assertFalse(getattr(self._Permutations(), 'fn(False)')()) - self.assertTrue(getattr(self._Permutations(), 'fn(True)')()) + def test_expand_new_methods(self): + self.assertTrue(hasattr(self.instance, 'fn(1, 2)')) + self.assertTrue(hasattr(self.instance, 'fn(3, 4)')) - def test_expanded_attributes(self): - fn = getattr(self._Permutations, 'fn(False)') - self.assertNotIn(PERMUTATION_ATTR, dir(fn)) + def test_remove_expanded_method(self): + self.assertFalse(hasattr(self.instance, "fn")) + + def test_invoke_expanded_1(self): + expanded_method = getattr(self.instance, "fn(1, 2)") + self.assertEqual((1, 2), expanded_method()) + + def test_invoke_expanded_2(self): + expanded_method = getattr(self.instance, "fn(3, 4)") + self.assertEqual((3, 4), expanded_method()) + + def test_clear_permuations_attribute(self): + fn = getattr(Permutated, 'fn(1, 2)') + self.assertFalse(hasattr(fn, PERMUTATION_ATTR)) + + +class TestSubPermuated(VdsmTestCase): + + def setUp(self): + self.instance = SubPermuated() + + def test_super_method(self): + expanded_method = getattr(self.instance, "fn(1, 2)") + self.assertEqual((1, 2), expanded_method()) + + def test_sub_method(self): + expanded_method = getattr(self.instance, "fn2(1)") + self.assertEqual(1, expanded_method()) -- To view, visit https://gerrit.ovirt.org/44214 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3e28fb348e688970611ddd2e66ed6869047b5317 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 6

Change in vdsm[master]: Experiment with delay after running tc
by Nir Soffer 04 Aug '15

04 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: Experiment with delay after running tc ...................................................................... Experiment with delay after running tc Change-Id: I37c7928d5e57c3555bd3467ec60293705bbc2ac5 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/nettestlib.py 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/44375/1 diff --git a/tests/nettestlib.py b/tests/nettestlib.py index 51bafc4..8e62cab 100644 --- a/tests/nettestlib.py +++ b/tests/nettestlib.py @@ -24,6 +24,7 @@ import platform import signal import struct +import time from multiprocessing import Process from nose.plugins.skip import SkipTest @@ -176,6 +177,11 @@ except ExecError as e: raise SkipTest("%r has failed: %s\nDo you have Traffic Control kernel " "modules installed?" % (EXT_TC, e.err)) + # FIXME: Deleting a bridge fails randomally becasue the bridge is up, + # altough we bring the interface down before deleting it. We probably + # need to wait for tc events, but monitoring 'tc' cause a segfault. + # Hopefully this delay will avoid the failures. + time.sleep(0.1) finally: dev.delDevice() -- To view, visit https://gerrit.ovirt.org/44375 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I37c7928d5e57c3555bd3467ec60293705bbc2ac5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

2 6

Change in vdsm[master]: register: general improvements
by Douglas Schilling Landgraf 04 Aug '15

04 Aug '15

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: register: general improvements ...................................................................... register: general improvements - replace --node-fqdn to --node-address for a better naming fit - remove unused socket library - improvement in docstring Change-Id: I5182c32e927e45da18e15e6a0f8fb2c6632de370 Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M lib/vdsm/tool/register.py 1 file changed, 11 insertions(+), 12 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/69/44369/1 diff --git a/lib/vdsm/tool/register.py b/lib/vdsm/tool/register.py index b5c2654..c170316 100644 --- a/lib/vdsm/tool/register.py +++ b/lib/vdsm/tool/register.py @@ -21,7 +21,6 @@ import logging import os import pwd -import socket import ssl import sys import tempfile @@ -40,13 +39,13 @@ def __init__(self, engine_fqdn, engine_https_port=None, fingerprint=None, ssh_port=None, ssh_user=None, check_fqdn=True, - vdsm_port=None, node_fqdn=None, + vdsm_port=None, node_address=None, node_name=None): """ Attributes: - engine_fqdn - Engine FQDN - engine_https_port - Engine http port + engine_fqdn - Engine FQDN or IP address + engine_https_port - Engine https port fingeprint - Fingerprint to be validated ssh_user - SSH user that will establish the connection from Engine @@ -54,7 +53,7 @@ check_fqdn - Validate Engine FQDN against CA (True or False) Default is TRUE vdsm_port - VDSM listen port - node_fqdn - Specify node FQDN + node_address - Specify node address or FQDN node_name - Specify node name """ self.logger = self._set_logger() @@ -87,8 +86,8 @@ self.fprint = fingerprint self.logger.debug("Fingerprint: {fp}".format(fp=self.fprint)) - self.node_fqdn = node_fqdn - self.logger.debug("Node FQDN: {nf}".format(nf=self.node_fqdn)) + self.node_address = node_address + self.logger.debug("Node address: {nf}".format(nf=self.node_address)) self.node_name = node_name self.logger.debug("Node name: {na}".format(na=self.node_name)) @@ -151,8 +150,8 @@ if self.node_name is not None: ureg += "&name={name}".format(name=self.node_name) - if self.node_fqdn is not None: - ureg += "&address={fqdn}".format(fqdn=self.node_fqdn) + if self.node_address is not None: + ureg += "&address={fqdn}".format(fqdn=self.node_address) self.url_reg = "{e}{u}".format(e=self.engine_url, u=ureg) @@ -378,7 +377,7 @@ ) parser.add_argument( - '--node-fqdn', + '--node-address', help="Define node FQDN or IP address." " If not provided, will be used system host name", ) @@ -392,7 +391,7 @@ parser.add_argument( '--engine-fqdn', - help="Engine FQDN (See also: --check-fqdn)", + help="Engine FQDN or IP address (See also: --check-fqdn)", required=True ) @@ -441,7 +440,7 @@ reg = Register(engine_fqdn=args.engine_fqdn, engine_https_port=args.engine_https_port, vdsm_port=args.vdsm_port, - node_fqdn=args.node_fqdn, + node_address=args.node_address, node_name=args.node_name, ssh_user=args.ssh_user, ssh_port=args.ssh_port, -- To view, visit https://gerrit.ovirt.org/44369 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5182c32e927e45da18e15e6a0f8fb2c6632de370 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

3 6

Change in vdsm[master]: Test randomness when running under mock
by Nir Soffer 04 Aug '15

04 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: Test randomness when running under mock ...................................................................... Test randomness when running under mock Add two tests that will always fail, showing random values generated by random.randint() and os.urandom(). If we get different random values on each run, this it shows that we don't need the randomness hack added in https://gerrit.ovirt.org/41079 Change-Id: I90fbc6cbc6ef9a9acbdc2964b92f1c4eb72df97d Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/testlibTests.py 1 file changed, 9 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/44356/1 diff --git a/tests/testlibTests.py b/tests/testlibTests.py index d631aaf..b2a6489 100644 --- a/tests/testlibTests.py +++ b/tests/testlibTests.py @@ -149,3 +149,12 @@ def test_expanded_attributes(self): fn = getattr(self._Permutations, 'fn(False)') self.assertNotIn(PERMUTATION_ATTR, dir(fn)) + + +class TestRandomness(VdsmTestCase): + def test_randint(self): + import random + self.assertEqual([0] * 10, [random.randint(0, 1000) for _ in range(10)]) + def test_urandom(self): + import os + self.assertEqual("0" * 20, os.urandom(10).encode('hex')) -- To view, visit https://gerrit.ovirt.org/44356 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I90fbc6cbc6ef9a9acbdc2964b92f1c4eb72df97d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

2 4

Change in vdsm[master]: register: use node name and fqdn when needed
by Douglas Schilling Landgraf 04 Aug '15

04 Aug '15

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: register: use node name and fqdn when needed ...................................................................... register: use node name and fqdn when needed Currently, we collect host fqdn before registering but it's not required. This patch make registration be triggered without setting host fqdn, it will use node IP address. Change-Id: I91c834e10cade4878791be43201ab0966bf251b4 Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M lib/vdsm/tool/register.py 1 file changed, 9 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/21/44221/1 diff --git a/lib/vdsm/tool/register.py b/lib/vdsm/tool/register.py index ce9cdbe..3656f1d 100644 --- a/lib/vdsm/tool/register.py +++ b/lib/vdsm/tool/register.py @@ -84,16 +84,10 @@ self.fprint = fingerprint self.logger.debug("Fingerprint: {fp}".format(fp=self.fprint)) - if node_fqdn is None: - self.node_fqdn = socket.gethostname() - else: - self.node_fqdn = node_fqdn + self.node_fqdn = node_fqdn self.logger.debug("Node FQDN: {nf}".format(nf=self.node_fqdn)) - if node_name is None: - self.node_name = socket.gethostname().split(".")[0] - else: - self.node_name = node_name + self.node_name = node_name self.logger.debug("Node name: {na}".format(na=self.node_name)) if ssh_user is None: @@ -145,14 +139,18 @@ uc=ucmd, c="get-ssh-trust") - ureg = "{uc}register&name={name}&address={fqdn}&sshUser={sshu}&" \ + ureg = "{uc}register&sshUser={sshu}&" \ "sshPort={sshp}&port={mp}".format(uc=ucmd, - name=self.node_name, - fqdn=self.node_fqdn, sshu=self.ssh_user, sshp=self.ssh_port, mp=self.vdsm_port) + if self.node_name is not None: + ureg += "&name={name}".format(name=self.node_name) + + if self.node_fqdn is not None: + ureg += "&address={fqdn}".format(fqdn=self.node_fqdn) + self.url_reg = "{e}{u}".format(e=self.engine_url, u=ureg) self.logger.debug("Download CA via: {u}".format(u=self.url_CA)) -- To view, visit https://gerrit.ovirt.org/44221 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I91c834e10cade4878791be43201ab0966bf251b4 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

3 9

Change in vdsm[master]: networkTests: fix slow tests broken by fb3a4bd4
by osvoboda＠redhat.com 04 Aug '15

04 Aug '15

Ondřej Svoboda has uploaded a new change for review. Change subject: networkTests: fix slow tests broken by fb3a4bd4 ...................................................................... networkTests: fix slow tests broken by fb3a4bd4 Change-Id: Ifc7efbfec1afcce44653c1b1eff9e840b9afbf24 Signed-off-by: Ondřej Svoboda <osvoboda(a)redhat.com> --- M tests/functional/networkTests.py 1 file changed, 1 insertion(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/39/44339/1 diff --git a/tests/functional/networkTests.py b/tests/functional/networkTests.py index 4b4dbf8..7526ded 100644 --- a/tests/functional/networkTests.py +++ b/tests/functional/networkTests.py @@ -2366,7 +2366,7 @@ @cleanupNet def testSetupNetworksEmergencyDevicesCleanupVlanOverwrite(self, bridged): with dummyIf(2) as nics: - nic, = nics + nic = nics[0] network = {NETWORK_NAME: {'vlan': VLAN_ID, 'bridged': bridged, 'nic': nic}} status, msg = self.setupNetworks(network, {}, NOCHK) @@ -2391,7 +2391,6 @@ @cleanupNet def testSetupNetworksEmergencyDevicesCleanupBondOverwrite(self, bridged): with dummyIf(2) as nics: - nic, = nics network = {NETWORK_NAME: {'bridged': bridged, 'bonding': BONDING_NAME}} bonding = {BONDING_NAME: {'nics': nics}} -- To view, visit https://gerrit.ovirt.org/44339 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifc7efbfec1afcce44653c1b1eff9e840b9afbf24 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ondřej Svoboda <osvoboda(a)redhat.com>

4 6

Change in vdsm[master]: tests: Extract nettestlib module
by Nir Soffer 04 Aug '15

04 Aug '15

Nir Soffer has uploaded a new change for review. Change subject: tests: Extract nettestlib module ...................................................................... tests: Extract nettestlib module ipwrapperTests was using tcTests as a library. Move the shared code to new module and use it from both tcTests and ipwrapperTests. Change-Id: I22b491120eb8be5b7ad0087ce21c489e6c04e24e Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/Makefile.am M tests/ipwrapperTests.py A tests/nettestlib.py M tests/tcTests.py 4 files changed, 182 insertions(+), 159 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/44212/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 3152735..174982c 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -189,6 +189,7 @@ apiData.py \ hookValidation.py \ monkeypatch.py \ + nettestlib.py \ storagefakelib.py \ storagetestlib.py \ testlib.py \ diff --git a/tests/ipwrapperTests.py b/tests/ipwrapperTests.py index bd3007d..bcef590 100644 --- a/tests/ipwrapperTests.py +++ b/tests/ipwrapperTests.py @@ -22,8 +22,8 @@ from vdsm import ipwrapper from vdsm.ipwrapper import Route from vdsm.ipwrapper import Rule -import tcTests +from nettestlib import Bridge, checkDependencies from testlib import VdsmTestCase as TestCaseBase @@ -95,8 +95,8 @@ @ValidateRunningAsRoot def setUp(self): - tcTests._checkDependencies() - self._bridge = tcTests._Bridge() + checkDependencies() + self._bridge = Bridge() self._bridge.addDevice() def tearDown(self): @@ -113,8 +113,8 @@ @ValidateRunningAsRoot def setUp(self): - tcTests._checkDependencies() - self._bridge = tcTests._Bridge() + checkDependencies() + self._bridge = Bridge() self._bridge.addDevice() def tearDown(self): @@ -140,10 +140,10 @@ @ValidateRunningAsRoot def setUp(self): - tcTests._checkDependencies() + checkDependencies() # First 3 Hebrew letters # See http://unicode.org/charts/PDF/U0590.pdf - self._bridge = tcTests._Bridge("\xd7\x90\xd7\x91\xd7\x92") + self._bridge = Bridge("\xd7\x90\xd7\x91\xd7\x92") self._bridge.addDevice() def tearDown(self): diff --git a/tests/nettestlib.py b/tests/nettestlib.py new file mode 100644 index 0000000..44a831d --- /dev/null +++ b/tests/nettestlib.py @@ -0,0 +1,166 @@ +# Copyright 2015 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import errno +import fcntl +import os +import platform +import random +import signal +import struct +from multiprocessing import Process + +from nose.plugins.skip import SkipTest + +from vdsm.constants import EXT_BRCTL, EXT_TC +from vdsm.netlink import monitor +from vdsm.utils import execCmd, random_iface_name + +EXT_IP = "/sbin/ip" + + +class ExecError(RuntimeError): + def __init__(self, msg, out, err): + super(ExecError, self).__init__(msg) + self.out = out + self.err = err + + +def check_call(cmd): + rc, out, err = execCmd(cmd, raw=True) + if rc != 0: + raise ExecError( + 'Command %s returned non-zero exit status %s.' % (cmd, rc), + out, err) + + +class Interface(): + + def __init__(self, prefix='vdsm-'): + self.devName = random_iface_name(prefix) + + def _ifUp(self): + check_call([EXT_IP, "link", "set", self.devName, "up"]) + + def _ifDown(self): + with monitor.Monitor(groups=('link',), timeout=2) as mon: + check_call([EXT_IP, "link", "set", self.devName, "down"]) + for event in mon: + if (event.get('name') == self.devName and + event.get('state') == 'down'): + return + + def __str__(self): + return "<{1} {2!r}>".format(self.__class__.__name__, self.devName) + + +class Bridge(Interface): + + def addDevice(self): + check_call([EXT_BRCTL, 'addbr', self.devName]) + # learning interval is different on different kernels, so set it + # explicit for 2.x kernels + if os.uname()[2].startswith("2"): + check_call([EXT_BRCTL, 'setfd', self.devName, '0']) + check_call([EXT_BRCTL, 'setageing', self.devName, '0']) + self._ifUp() + + def delDevice(self): + self._ifDown() + check_call([EXT_BRCTL, 'delbr', self.devName]) + + def addIf(self, dev): + check_call([EXT_BRCTL, 'addif', self.devName, dev]) + + +def _listenOnDevice(fd, icmp): + while True: + packet = os.read(fd, 2048) + # check if it is an IP packet + if (packet[12:14] == chr(0x08) + chr(0x00)): + if packet == icmp: + return + + +class Tap(Interface): + + _IFF_TAP = 0x0002 + _IFF_NO_PI = 0x1000 + arch = platform.machine() + if arch == 'x86_64': + _TUNSETIFF = 0x400454ca + elif arch == 'ppc64': + _TUNSETIFF = 0x800454ca + else: + raise SkipTest("Unsupported Architecture %s" % arch) + + _deviceListener = None + + def addDevice(self): + self._cloneDevice = open('/dev/net/tun', 'r+b') + ifr = struct.pack('16sH', self.devName, self._IFF_TAP | + self._IFF_NO_PI) + fcntl.ioctl(self._cloneDevice, self._TUNSETIFF, ifr) + self._ifUp() + + def delDevice(self): + self._ifDown() + self._cloneDevice.close() + + def startListener(self, icmp): + self._deviceListener = Process(target=_listenOnDevice, + args=(self._cloneDevice.fileno(), icmp)) + self._deviceListener.start() + + def isListenerAlive(self): + if self._deviceListener: + return self._deviceListener.is_alive() + else: + return False + + def stopListener(self): + if self._deviceListener: + os.kill(self._deviceListener.pid, signal.SIGKILL) + self._deviceListener.join() + + def writeToDevice(self, icmp): + os.write(self._cloneDevice.fileno(), icmp) + + +def checkDependencies(): + + # hack to avoid name collision in concurrently-running mock-based tests + random.jumpahead(os.getpid()) + + dev = Bridge() + try: + dev.addDevice() + except OSError as e: + if e.errno == errno.ENOENT: + raise SkipTest("Cannot run %r: %s\nDo you have bridge-utils " + "installed?" % (EXT_BRCTL, e)) + raise + + try: + check_call([EXT_TC, 'qdisc', 'add', 'dev', dev.devName, 'ingress']) + except ExecError as e: + raise SkipTest("%r has failed: %s\nDo you have Traffic Control kernel " + "modules installed?" % (EXT_TC, e.err)) + finally: + dev.delDevice() diff --git a/tests/tcTests.py b/tests/tcTests.py index 942f63f..8c894ba 100644 --- a/tests/tcTests.py +++ b/tests/tcTests.py @@ -19,173 +19,29 @@ # Refer to the README and COPYING files for full details of the license # -import errno -import random import time import os -import signal import six import sys -from multiprocessing import Process from binascii import unhexlify from itertools import izip_longest from subprocess import Popen, PIPE -import fcntl -import struct from testlib import VdsmTestCase as TestCaseBase from testValidation import ValidateRunningAsRoot +from nettestlib import Bridge, Tap, checkDependencies -from vdsm.constants import EXT_BRCTL, EXT_TC -from vdsm.netlink import monitor -from vdsm.utils import execCmd, random_iface_name - -from nose.plugins.skip import SkipTest - +from vdsm.constants import EXT_TC from network import tc -import platform - -EXT_IP = "/sbin/ip" - - -class ExecError(RuntimeError): - def __init__(self, msg, out, err): - super(ExecError, self).__init__(msg) - self.out = out - self.err = err - - -def check_call(cmd): - rc, out, err = execCmd(cmd, raw=True) - if rc != 0: - raise ExecError( - 'Command %s returned non-zero exit status %s.' % (cmd, rc), - out, err) - - -class _Interface(): - - def __init__(self, prefix='vdsm-'): - self.devName = random_iface_name(prefix) - - def _ifUp(self): - check_call([EXT_IP, "link", "set", self.devName, "up"]) - - def _ifDown(self): - with monitor.Monitor(groups=('link',), timeout=2) as mon: - check_call([EXT_IP, "link", "set", self.devName, "down"]) - for event in mon: - if (event.get('name') == self.devName and - event.get('state') == 'down'): - return - - def __str__(self): - return "<{1} {2!r}>".format(self.__class__.__name__, self.devName) - - -class _Bridge(_Interface): - - def addDevice(self): - check_call([EXT_BRCTL, 'addbr', self.devName]) - # learning interval is different on different kernels, so set it - # explicit for 2.x kernels - if os.uname()[2].startswith("2"): - check_call([EXT_BRCTL, 'setfd', self.devName, '0']) - check_call([EXT_BRCTL, 'setageing', self.devName, '0']) - self._ifUp() - - def delDevice(self): - self._ifDown() - check_call([EXT_BRCTL, 'delbr', self.devName]) - - def addIf(self, dev): - check_call([EXT_BRCTL, 'addif', self.devName, dev]) - - -def _listenOnDevice(fd, icmp): - while True: - packet = os.read(fd, 2048) - # check if it is an IP packet - if (packet[12:14] == chr(0x08) + chr(0x00)): - if packet == icmp: - return - - -class Tap(_Interface): - - _IFF_TAP = 0x0002 - _IFF_NO_PI = 0x1000 - arch = platform.machine() - if arch == 'x86_64': - _TUNSETIFF = 0x400454ca - elif arch == 'ppc64': - _TUNSETIFF = 0x800454ca - else: - raise SkipTest("Unsupported Architecture %s" % arch) - - _deviceListener = None - - def addDevice(self): - self._cloneDevice = open('/dev/net/tun', 'r+b') - ifr = struct.pack('16sH', self.devName, self._IFF_TAP | - self._IFF_NO_PI) - fcntl.ioctl(self._cloneDevice, self._TUNSETIFF, ifr) - self._ifUp() - - def delDevice(self): - self._ifDown() - self._cloneDevice.close() - - def startListener(self, icmp): - self._deviceListener = Process(target=_listenOnDevice, - args=(self._cloneDevice.fileno(), icmp)) - self._deviceListener.start() - - def isListenerAlive(self): - if self._deviceListener: - return self._deviceListener.is_alive() - else: - return False - - def stopListener(self): - if self._deviceListener: - os.kill(self._deviceListener.pid, signal.SIGKILL) - self._deviceListener.join() - - def writeToDevice(self, icmp): - os.write(self._cloneDevice.fileno(), icmp) - - -def _checkDependencies(): - - # hack to avoid name collision in concurrently-running mock-based tests - random.jumpahead(os.getpid()) - - dev = _Bridge() - try: - dev.addDevice() - except OSError as e: - if e.errno == errno.ENOENT: - raise SkipTest("Cannot run %r: %s\nDo you have bridge-utils " - "installed?" % (EXT_BRCTL, e)) - raise - - try: - check_call([EXT_TC, 'qdisc', 'add', 'dev', dev.devName, 'ingress']) - except ExecError as e: - raise SkipTest("%r has failed: %s\nDo you have Traffic Control kernel " - "modules installed?" % (EXT_TC, e.err)) - finally: - dev.delDevice() class TestQdisc(TestCaseBase): @ValidateRunningAsRoot def setUp(self): - _checkDependencies() - self._bridge = _Bridge() + checkDependencies() + self._bridge = Bridge() self._bridge.addDevice() def tearDown(self): @@ -431,13 +287,13 @@ @ValidateRunningAsRoot def setUp(self): - _checkDependencies() + checkDependencies() self._tap0 = Tap() self._tap1 = Tap() self._tap2 = Tap() - self._bridge0 = _Bridge('src-') - self._bridge1 = _Bridge('target-') - self._bridge2 = _Bridge('target2-') + self._bridge0 = Bridge('src-') + self._bridge1 = Bridge('target-') + self._bridge2 = Bridge('target2-') self._devices = [self._tap0, self._tap1, self._tap2, self._bridge0, self._bridge1, self._bridge2] # If setUp raise, teardown is not called, so we should either succeed, -- To view, visit https://gerrit.ovirt.org/44212 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I22b491120eb8be5b7ad0087ce21c489e6c04e24e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 16

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches August 2015