vdsm-patches December 2013

vdsm-patches@lists.fedorahosted.org

48 participants
493 discussions

Change in vdsm[master]: Explicit call for libvirtd service manipulations during conf...
by ybronhei＠redhat.com 12 Jan '14

12 Jan '14

Yaniv Bronhaim has uploaded a new change for review. Change subject: Explicit call for libvirtd service manipulations during configure ...................................................................... Explicit call for libvirtd service manipulations during configure We used service.py package to run operations on running services. service.py tries all alternatives until reaching included one (systemd, initctl..). Over RHEL we prefer to use initctl to manage libvirtd (as it helps us to avoid respawn script). For that we need explicitly to use initctl manipulation for libvirtd. Change-Id: I08bbf2bb2aac950bf15210018adfb296a8f3fda3 Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1047514 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M lib/vdsm/tool/configurator.py 1 file changed, 32 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/43/22843/1 diff --git a/lib/vdsm/tool/configurator.py b/lib/vdsm/tool/configurator.py index 869d774..1cabe28 100644 --- a/lib/vdsm/tool/configurator.py +++ b/lib/vdsm/tool/configurator.py @@ -178,6 +178,35 @@ ) +# The above helper functions should provide service manipulation using service +# package. Some services can be used only by specific mechanisem as libvirtd +# which is managed by upstart. using systemctl in this case will give invalid +# results. +def _getServiceStatus(s): + if s is "libvirtd": + ret = service._initctlStatus(s) + else: + ret = service.service_status(s) + return ret + + +def _stopService(s): + if s is "libvirtd": + ret = service._initctlStop(s) + else: + ret = service.service_stop(s) + return ret + + +def _startService(s): + if s is "libvirtd": + ret = service._initctlStart(s) + else: + ret = service.service_start(s) + + return ret + + @expose("configure") def configure(*args): """ @@ -197,7 +226,7 @@ services = [] for c in configurer_to_trigger: for s in c.getServices(): - if service.service_status(s) == 0: + if _getServiceStatus(s) == 0: if not args.force: raise RuntimeError( "Cannot configure while %s is running" % s @@ -205,13 +234,13 @@ services.append(s) for s in services: - service.service_stop(s) + _stopService(s) for c in configurer_to_trigger: c.configure() for s in reversed(services): - service.service_start(s) + _startService(s) @expose("is-configured") -- To view, visit http://gerrit.ovirt.org/22843 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I08bbf2bb2aac950bf15210018adfb296a8f3fda3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

4 5

Change in vdsm[ovirt-3.3]: utils: Create AsyncProcessOperation
by smizrahi＠redhat.com 12 Jan '14

12 Jan '14

Saggi Mizrahi has uploaded a new change for review. Change subject: utils: Create AsyncProcessOperation ...................................................................... utils: Create AsyncProcessOperation This is an object to wrap a process so it can be treated as an AsyncOperation. Features: * Takes care of pid collection * Has pluggable output parsing * Process appropriate implementation of wait() and stop() Related-Bug-Url: http://bugzilla.redhat.com/964595 Change-Id: I79d0eefc9d917a4a93916d52867fb4f1e793c60e Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M lib/vdsm/utils.py M tests/utilsTests.py 2 files changed, 113 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/35/21135/1 diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py index d3e0c2b..14f9dfa 100644 --- a/lib/vdsm/utils.py +++ b/lib/vdsm/utils.py @@ -48,6 +48,7 @@ import subprocess import threading import time +import zombiereaper from cpopen import CPopen as BetterPopen from config import config @@ -914,6 +915,55 @@ time.sleep(sleep) +class AsyncProcessOperation(object): + def __init__(self, proc, resultParser=None): + """Wraps a running process operation. + + resultParser should be of type callback(rc, out, err) and can return + anything or throw exceptions.""" + self._lock = threading.Lock() + + self._result = None + self._resultParser = resultParser + + self._proc = proc + + def wait(self, timeout=None, cond=None): + """Waits until the process has exited, the timeout has been reached or + the condition has been met""" + return self._proc.wait(timeout, cond) + + def stop(self): + """Stops the running operation, effectively sending a kill signal to + the process""" + self._proc.kill() + + def result(self): + """Returns the result in the as a tuple of (result, error). + If the operation is still running it will block until it returns. + + If no resultParser has been set the default result + is (rc, out, err) """ + with self._lock: + if self._result is None: + out, err = self._proc.communicate() + rc = self._proc.returncode + if self._resultParser is not None: + try: + self._result = (self._resultParser(rc, out, err), + None) + except Exception as e: + self._result = (None, e) + else: + self._result = ((rc, out, err), None) + + return self._result + + def __del__(self): + if self._proc.returncode is None: + zombiereaper.autoReapPID(self._proc.pid) + + def panic(msg): logging.error("Panic: %s", msg, exc_info=True) os.killpg(0, 9) diff --git a/tests/utilsTests.py b/tests/utilsTests.py index a6a222d..656c121 100644 --- a/tests/utilsTests.py +++ b/tests/utilsTests.py @@ -22,6 +22,7 @@ from testrunner import VdsmTestCase as TestCaseBase from vdsm import utils from storage import misc +import time class RetryTests(TestCaseBase): @@ -80,3 +81,65 @@ class GeneralUtilsTests(TestCaseBase): def testPanic(self): self.assertRaises(AssertionError, utils.panic, "panic test") + + +class AsyncProcessOperationTests(TestCaseBase): + def _echo(self, text): + proc = misc.execCmd(["echo", "-n", "test"], sync=False) + + def parse(rc, out, err): + return out + + return utils.AsyncProcessOperation(proc, parse) + + def _sleep(self, t): + proc = misc.execCmd(["sleep", str(t)], sync=False) + return utils.AsyncProcessOperation(proc) + + def _fail(self, t): + proc = misc.execCmd(["sleep", str(t)], sync=False) + + def parse(rc, out, err): + raise Exception("TEST!!!") + + return utils.AsyncProcessOperation(proc, parse) + + def test(self): + aop = self._sleep(1) + self.assertEquals(aop.result(), ((0, "", ""), None)) + + def testAlreadyExitedSuccess(self): + aop = self._sleep(0) + time.sleep(1) + self.assertEquals(aop.result(), ((0, "", ""), None)) + + def testAlreadyExitedFail(self): + aop = self._sleep("hello") + time.sleep(1) + ((rc, out, err), err) = aop.result() + self.assertEquals(err, None) + self.assertEquals(rc, 1) + + def testWait(self): + aop = self._sleep(1) + aop.wait(timeout=2) + + def testParser(self): + aop = self._echo("test") + self.assertEquals(aop.result(), ("test", None)) + + def testStop(self): + aop = self._sleep(10) + aop.stop() + + start = time.time() + aop.result() + end = time.time() + duration = end - start + self.assertTrue(duration < 2) + + def testException(self): + aop = self._fail(1) + res, err = aop.result() + self.assertEquals(res, None) + self.assertNotEquals(err, None) -- To view, visit http://gerrit.ovirt.org/21135 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I79d0eefc9d917a4a93916d52867fb4f1e793c60e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-3.3 Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

3 6

Change in vdsm[master]: gluster: Add force option to Volume Create command
by avishwan＠redhat.com 12 Jan '14

12 Jan '14

Aravinda VK has uploaded a new change for review. Change subject: gluster: Add force option to Volume Create command ...................................................................... gluster: Add force option to Volume Create command glusterfs cli will fail if system's root partition is used as brick dirs. If user wants to use system's root partition as bricks dir then "force" option can be used in gluster cli. Added the "force" parameter to vdsm verb glusterVolumeCreate It will not break backward compatibility since it utilizes existing optional parameter. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1008942 Change-Id: I3b3a6f79c1473c4bc5d7976de20c03e1ac513af0 Signed-off-by: Aravinda VK <avishwan(a)redhat.com> --- M client/vdsClientGluster.py M vdsm/gluster/api.py M vdsm/gluster/cli.py M vdsm/gluster/vdsmapi-gluster-schema.json 4 files changed, 13 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/21082/1 diff --git a/client/vdsClientGluster.py b/client/vdsClientGluster.py index 65e6282..7353e13 100644 --- a/client/vdsClientGluster.py +++ b/client/vdsClientGluster.py @@ -39,10 +39,11 @@ stripeCount = params.get('stripe', '') transport = params.get('transport', '') transportList = transport.strip().split(',') if transport else [] + force = {"force": (params.get('force', 'no').upper() == 'YES')} status = self.s.glusterVolumeCreate(volumeName, brickList, replicaCount, stripeCount, - transportList) + transportList, force) return status['status']['code'], status['status']['message'] def do_glusterVolumesList(self, args): @@ -449,7 +450,8 @@ {'glusterVolumeCreate': ( serv.do_glusterVolumeCreate, ('volumeName=<volume_name> bricks=<brick[,brick, ...]> ' - '[replica=<count>] [stripe=<count>] [transport={tcp|rdma}]\n\t' + '[replica=<count>] [stripe=<count>] [transport={tcp|rdma}] ' + '[force={yes|no}]\n\t' '<volume_name> is name of new volume', '<brick[,brick, ...]> is brick(s) which will be used to ' 'create volume', diff --git a/vdsm/gluster/api.py b/vdsm/gluster/api.py index 193d6bd..1078700 100644 --- a/vdsm/gluster/api.py +++ b/vdsm/gluster/api.py @@ -78,10 +78,11 @@ @exportAsVerb def volumeCreate(self, volumeName, brickList, replicaCount=0, - stripeCount=0, transportList=[], options=None): + stripeCount=0, transportList=[], options={}): + force = options.get("force") return self.svdsmProxy.glusterVolumeCreate(volumeName, brickList, replicaCount, stripeCount, - transportList) + transportList, force) @exportAsVerb def volumeStart(self, volumeName, force=False, options=None): diff --git a/vdsm/gluster/cli.py b/vdsm/gluster/cli.py index a654b56..25fd8ba 100644 --- a/vdsm/gluster/cli.py +++ b/vdsm/gluster/cli.py @@ -445,7 +445,7 @@ @makePublic def volumeCreate(volumeName, brickList, replicaCount=0, stripeCount=0, - transportList=[]): + transportList=[], force=False): command = _getGlusterVolCmd() + ["create", volumeName] if stripeCount: command += ["stripe", "%s" % stripeCount] @@ -454,6 +454,10 @@ if transportList: command += ["transport", ','.join(transportList)] command += brickList + + if force: + command.append('force') + try: xmltree = _execGlusterXml(command) except ge.GlusterCmdFailedException as e: diff --git a/vdsm/gluster/vdsmapi-gluster-schema.json b/vdsm/gluster/vdsmapi-gluster-schema.json index 7de762a..3bab622 100644 --- a/vdsm/gluster/vdsmapi-gluster-schema.json +++ b/vdsm/gluster/vdsmapi-gluster-schema.json @@ -558,7 +558,7 @@ # Since: 4.10.3 ## {'command': {'class': 'GlusterVolume', 'name': 'create'}, - 'data': {'volumeName': 'str', 'bricklist': ['str'], 'replicaCount': 'int', 'stripeCount': 'int', 'transportList': ['transType']}, + 'data': {'volumeName': 'str', 'bricklist': ['str'], 'replicaCount': 'int', 'stripeCount': 'int', 'transportList': ['transType'], 'force': 'bool'}, 'returns': 'UUID'} ## -- To view, visit http://gerrit.ovirt.org/21082 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3b3a6f79c1473c4bc5d7976de20c03e1ac513af0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Aravinda VK <avishwan(a)redhat.com>

6 21

Change in vdsm[master]: [5/x] Use getDeviceByIP(server IP) for lastClientIface
by amuller＠redhat.com 12 Jan '14

12 Jan '14

Assaf Muller has uploaded a new change for review. Change subject: [5/x] Use getDeviceByIP(server IP) for lastClientIface ...................................................................... [5/x] Use getDeviceByIP(server IP) for lastClientIface Change-Id: Ic17e91c80560cd3acb73e7a64879257a6e07b1fe Signed-off-by: Assaf Muller <amuller(a)redhat.com> --- M vdsm/BindingXMLRPC.py 1 file changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/42/22842/1 diff --git a/vdsm/BindingXMLRPC.py b/vdsm/BindingXMLRPC.py index 2364684..a10e952 100644 --- a/vdsm/BindingXMLRPC.py +++ b/vdsm/BindingXMLRPC.py @@ -28,7 +28,7 @@ from vdsm import constants from vdsm import utils from vdsm.define import doneCode, errCode -from vdsm.netinfo import getRouteDeviceTo +from vdsm.netinfo import getDeviceByIP import API from vdsm.exception import VdsmException try: @@ -115,6 +115,7 @@ class LoggingHandler(basehandler): def setup(self): threadLocal.client = self.client_address[0] + threadLocal.server = self.request.getsockname()[0] return basehandler.setup(self) def parse_request(self): @@ -295,8 +296,8 @@ ret = api.getCapabilities() ret['info']['management_ip'] = self.serverIP ret['info']['lastClient'] = self.cif.threadLocal.client - ret['info']['lastClientIface'] = getRouteDeviceTo( - destinationIP=self.cif.threadLocal.client) + ret['info']['lastClientIface'] = getDeviceByIP( + self.cif.threadLocal.server) return ret def getHardwareInfo(self): -- To view, visit http://gerrit.ovirt.org/22842 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic17e91c80560cd3acb73e7a64879257a6e07b1fe Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Assaf Muller <amuller(a)redhat.com>

3 11

Change in vdsm[master]: [4/x] Added netinfo:getDeviceByIP and a test
by amuller＠redhat.com 12 Jan '14

12 Jan '14

Assaf Muller has uploaded a new change for review. Change subject: [4/x] Added netinfo:getDeviceByIP and a test ...................................................................... [4/x] Added netinfo:getDeviceByIP and a test Change-Id: I3563ae9b3e199e29ea2227463b9e4a3752cb99a0 Signed-off-by: Assaf Muller <amuller(a)redhat.com> --- M lib/vdsm/netinfo.py M tests/netinfoTests.py 2 files changed, 28 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/41/22841/1 diff --git a/lib/vdsm/netinfo.py b/lib/vdsm/netinfo.py index d000fe4..d8af7bc 100644 --- a/lib/vdsm/netinfo.py +++ b/lib/vdsm/netinfo.py @@ -615,6 +615,23 @@ return '' +def getDeviceByIP(ip): + """ + Get network device by IP address + :param ip: String representing IPv4 or IPv6, but not link-local IPv6 + """ + for info in ethtool.get_interfaces_info(ethtool.get_active_devices()): + for ipv4addr in info.get_ipv4_addresses(): + if ip in (ipv4addr.address, IPv4toMapped(ipv4addr.address)): + return info.device + + for ipv6addr in info.get_ipv6_addresses(): + if ip == ipv6addr.address: + return info.device + + return '' + + class NetInfo(object): def __init__(self, _netinfo=None): if _netinfo is None: diff --git a/tests/netinfoTests.py b/tests/netinfoTests.py index da233e0..5207c0d 100644 --- a/tests/netinfoTests.py +++ b/tests/netinfoTests.py @@ -24,6 +24,8 @@ import tempfile from xml.dom import minidom +import ethtool + from vdsm import ipwrapper from vdsm import netconfpersistence from vdsm import netinfo @@ -111,6 +113,15 @@ def testIPv4toMapped(self): self.assertEqual('::ffff:127.0.0.1', netinfo.IPv4toMapped('127.0.0.1')) + def testGetDeviceByIP(self): + for dev in ethtool.get_interfaces_info(ethtool.get_active_devices()): + ipaddrs = [ipv6.address for ipv6 in dev.get_ipv6_addresses() + if ipv6.scope != 'link'] + if dev.ipv4_address is not None: + ipaddrs.append(dev.ipv4_address) + for ip in ipaddrs: + self.assertTrue(dev.device == netinfo.getDeviceByIP(ip)) + def _testNics(self): """Creates a test fixture so that nics() reports: physical nics: em, me, me0, me1, hid0 and hideous -- To view, visit http://gerrit.ovirt.org/22841 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3563ae9b3e199e29ea2227463b9e4a3752cb99a0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Assaf Muller <amuller(a)redhat.com>

3 13

Change in vdsm[ovirt-3.3]: iscsi: Iscsi rescan cleanup
by smizrahi＠redhat.com 12 Jan '14

12 Jan '14

Saggi Mizrahi has uploaded a new change for review. Change subject: iscsi: Iscsi rescan cleanup ...................................................................... iscsi: Iscsi rescan cleanup * Remove forceIscsiScan(): iscsiadm already does that when refreshing sessions * Cleanup iscsi.rescan(): Replace complex logic with a simpler implementation since new iscsiadm takes care of all of this itself * Make iscsi.rescan() more testable * Test confusing timeout semantics of iscsi.rescan() Bug-Url: https://bugzilla.redhat.com/964595 Change-Id: I842eb37cea3bd5e8cd357a310dd966081b242abd Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M tests/Makefile.am A tests/iscsiTests.py M vdsm.spec.in M vdsm/storage/iscsi.py M vdsm/storage/iscsiadm.py M vdsm/storage/multipath.py M vdsm/supervdsmServer 7 files changed, 111 insertions(+), 62 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/21136/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index f87eddd..5efae38 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -37,6 +37,7 @@ guestIFTests.py \ hooksTests.py \ ipwrapperTests.py \ + iscsiTests.py \ jsonRpcTests.py \ jsonRpcUtils.py \ libvirtconnectionTests.py \ diff --git a/tests/iscsiTests.py b/tests/iscsiTests.py new file mode 100644 index 0000000..44f6962 --- /dev/null +++ b/tests/iscsiTests.py @@ -0,0 +1,80 @@ +import threading +import time +from contextlib import contextmanager + +from testrunner import VdsmTestCase as TestCaseBase + +from storage import iscsi + + +class AsyncStubOperation(object): + def __init__(self, timeout): + self._evt = threading.Event() + if timeout == 0: + self._evt.set() + else: + threading.Timer(timeout, self._evt.set) + + def wait(self, timeout=None, cond=None): + if cond is not None: + raise Exception("TODO!!!") + + self._evt.wait(timeout) + + def stop(self): + self._evt.set() + + def result(self): + if self._evt.is_set(): + return (None, None) + else: + return None + + +class RescanTimeoutTests(TestCaseBase): + def setUp(self): + self._iscsiadm_rescan_async = \ + iscsi.iscsiadm.session_rescan_async + iscsi.iscsiadm.session_rescan_async = self._iscsi_stub + self._timeout = 0 + + def tearDown(self): + iscsi.iscsiadm.session_rescan_async = \ + self._iscsiadm_rescan_async + + def _iscsi_stub(self): + return AsyncStubOperation(self._timeout) + + @contextmanager + def assertMaxDuration(self, maxtime): + start = time.time() + try: + yield + finally: + end = time.time() + elapsed = end - start + if maxtime < elapsed: + self.fail("Operation was too slow %fs > %fs" % + (elapsed, maxtime)) + + @contextmanager + def assertMinDuration(self, mintime): + start = time.time() + try: + yield + finally: + end = time.time() + elapsed = end - start + if mintime > elapsed: + self.fail("Operation was too fast %fs > %fs" % + (elapsed, mintime)) + + def testFast(self): + self._timeout = 0 + with self.assertMinDuration(2): + iscsi.rescan(2, 4) + + def testSlow(self): + self._timeout = 60 + with self.assertMaxDuration(3): + iscsi.rescan(1, 2) diff --git a/vdsm.spec.in b/vdsm.spec.in index b0b89a1..cae9cab 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -146,7 +146,7 @@ # Update the qemu-kvm requires when block_stream will be included Requires: qemu-kvm >= 2:0.12.1.2-2.295.el6_3.4 Requires: qemu-img >= 2:0.12.1.2-2.295.el6_3.4 -Requires: iscsi-initiator-utils >= 6.2.0.872-15 +Requires: iscsi-initiator-utils >= 6.2.0.873-3 Requires: device-mapper-multipath >= 0.4.9-52 Requires: e2fsprogs >= 1.41.12-11 Requires: fence-agents diff --git a/vdsm/storage/iscsi.py b/vdsm/storage/iscsi.py index 7da94ab..f4fdceb 100644 --- a/vdsm/storage/iscsi.py +++ b/vdsm/storage/iscsi.py @@ -30,7 +30,6 @@ import time from collections import namedtuple -from vdsm import constants import misc from vdsm.config import config import devicemapper @@ -373,38 +372,15 @@ @misc.samplingmethod -def rescan(): - try: - iscsiadm.session_rescan() - except iscsiadm.IscsiError: - pass +def rescan(minTimeout=None, maxTimeout=None): + # FIXME: This whole thing is wrong from the core. We need to make rescan + # completely async and have methods timeout on their own if they + # can't find the devices they are looking for + if minTimeout is None: + minTimeout = config.getint('irs', 'scsi_rescan_minimal_timeout') + if maxTimeout is None: + maxTimeout = config.getint('irs', 'scsi_rescan_maximal_timeout') - -(a)misc.samplingmethod -def forceScsiScan(): - processes = [] - minTimeout = config.getint('irs', 'scsi_rescan_minimal_timeout') - maxTimeout = config.getint('irs', 'scsi_rescan_maximal_timeout') - for hba in glob.glob(SCAN_PATTERN): - cmd = [constants.EXT_DD, 'of=' + hba] - p = misc.execCmd(cmd, sudo=False, sync=False) - try: - p.stdin.write("- - -") - p.stdin.flush() - p.stdin.close() - except OSError as e: - if p.wait(0) is False: - log.error("pid %s still running", p.pid) - log.warning("Error in rescan of hba:%s with returncode:%s and " - "error message: %s", hba, p.returncode, - p.stderr.read(1000)) - if e.errno != errno.EPIPE: - raise - else: - log.warning("Ignoring error in rescan of hba %s: ", - hba, exc_info=True) - continue - processes.append((hba, p)) if (minTimeout > maxTimeout or minTimeout < 0): minTimeout = 2 maxTimeout = 30 @@ -412,24 +388,13 @@ "illegal value: scsi_rescan_minimal_timeout or " "scsi_rescan_maximal_timeout. Set to %s and %s seconds " "respectively.", minTimeout, maxTimeout) + log.debug("Performing SCSI scan, this will take up to %s seconds", maxTimeout) + + rescanOp = iscsiadm.session_rescan_async() time.sleep(minTimeout) - for i in xrange(maxTimeout - minTimeout): - for p in processes[:]: - (hba, proc) = p - if proc.wait(0): - if proc.returncode != 0: - log.warning('returncode for: %s is: %s', hba, - proc.returncode) - processes.remove(p) - if not processes: - break - else: - time.sleep(1) - else: - log.warning("Still waiting for scsi scan of hbas: %s", - tuple(hba for p in processes)) + rescanOp.wait(timeout=(maxTimeout - minTimeout)) def devIsiSCSI(dev): diff --git a/vdsm/storage/iscsiadm.py b/vdsm/storage/iscsiadm.py index cd28db8..9bb27ea 100644 --- a/vdsm/storage/iscsiadm.py +++ b/vdsm/storage/iscsiadm.py @@ -1,6 +1,7 @@ from threading import Lock import misc from vdsm import constants +from vdsm.utils import AsyncProcessOperation # iscsiadm exit statuses ISCSI_ERR_SESS_EXISTS = 15 @@ -71,7 +72,7 @@ _iscsiadmLock = Lock() -def _runCmd(args, hideValue=False): +def _runCmd(args, hideValue=False, sync=True): # FIXME: I don't use supervdsm because this entire module has to just be # run as root and there is no such feature yet in supervdsm. When such # feature exists please change this. @@ -88,7 +89,7 @@ if i < (len(printCmd) - 1): printCmd[i + 1] = "****" - return misc.execCmd(cmd, printable=printCmd, sudo=True) + return misc.execCmd(cmd, printable=printCmd, sudo=True, sync=sync) def iface_exists(interfaceName): @@ -294,12 +295,21 @@ raise IscsiNodeError(rc, out, err) -def session_rescan(): - rc, out, err = _runCmd(["-m", "session", "-R"]) - if rc == 0: - return +def session_rescan_async(): + proc = _runCmd(["-m", "session", "-R"], sync=False) - raise IscsiSessionError(rc, out, err) + def parse_result(rc, out, err): + if rc == 0: + return + + raise IscsiSessionError(rc, out, err) + + return AsyncProcessOperation(proc, parse_result) + + +def session_rescan(): + aop = session_rescan_async() + return aop.result() def session_logout(sessionId): diff --git a/vdsm/storage/multipath.py b/vdsm/storage/multipath.py index 1f9773c..29851f3 100644 --- a/vdsm/storage/multipath.py +++ b/vdsm/storage/multipath.py @@ -105,8 +105,6 @@ # First ask iSCSI to rescan all its sessions iscsi.rescan() - supervdsm.getProxy().forceScsiScan() - # Now let multipath daemon pick up new devices cmd = [constants.EXT_MULTIPATH, "-r"] misc.execCmd(cmd, sudo=True) diff --git a/vdsm/supervdsmServer b/vdsm/supervdsmServer index 40ec9df..6c2d3f5 100755 --- a/vdsm/supervdsmServer +++ b/vdsm/supervdsmServer @@ -51,7 +51,6 @@ from lsblk import getLsBlk as _getLsBlk from storage.multipath import getScsiSerial as _getScsiSerial -from storage.iscsi import forceScsiScan as _forceScsiScan from storage.iscsi import getDevIscsiInfo as _getdeviSCSIinfo from storage.iscsi import readSessionInfo as _readSessionInfo from supervdsm import _SuperVdsmManager @@ -142,10 +141,6 @@ @logDecorator def getScsiSerial(self, *args, **kwargs): return _getScsiSerial(*args, **kwargs) - - @logDecorator - def forceScsiScan(self, *args, **kwargs): - return _forceScsiScan(*args, **kwargs) @logDecorator def removeDeviceMapping(self, devName): -- To view, visit http://gerrit.ovirt.org/21136 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I842eb37cea3bd5e8cd357a310dd966081b242abd Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-3.3 Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

4 7

Change in vdsm[ovirt-3.3]: configNetwork: Fix bridgeless broken network deletion
by asegurap＠redhat.com 12 Jan '14

12 Jan '14

Hello Dan Kenigsberg, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/22436 to review the following change. Change subject: configNetwork: Fix bridgeless broken network deletion ...................................................................... configNetwork: Fix bridgeless broken network deletion When trying to delete a broken bridgeless vdsm network, if the iface referenced by the network definition was missing, delNetwork would not get any device on its call for vlan, nic and bonds, but objectivize would be called nonetheless, failing with a: ERR_BAD_PARAMS: 'Network defined without devices.' The fix consists on just removing the network from libvirt when the backing device is missing. Change-Id: I92c641d91da9ef7eab876ed8c77fff7a9becb503 Bug-Url: https://bugzilla.redhat.com/1020356 Signed-off-by: Antoni S. Puimedon <asegurap(a)redhat.com> Reviewed-on: http://gerrit.ovirt.org/22354 Reviewed-by: Dan Kenigsberg <danken(a)redhat.com> --- M tests/functional/networkTests.py M tests/functional/utils.py M vdsm/configNetwork.py 3 files changed, 34 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/22436/1 diff --git a/tests/functional/networkTests.py b/tests/functional/networkTests.py index 3e0eed8..c010bcc 100644 --- a/tests/functional/networkTests.py +++ b/tests/functional/networkTests.py @@ -27,6 +27,7 @@ from testValidation import RequireDummyMod, ValidateRunningAsRoot import dummy +import ipwrapper from utils import cleanupNet, restoreNetConfig, SUCCESS, VdsProxy, cleanupRules from vdsm.ipwrapper import (ruleAdd, ruleDel, routeAdd, routeDel, routeExists, @@ -1405,3 +1406,27 @@ status, msg = self.vdsm_net.setupNetworks(delete_networks, {}, {}) self.assertEqual(status, SUCCESS, msg) + + @RequireDummyMod + @ValidateRunningAsRoot + def testBrokenBridgelessNetReplacement(self): + with dummyIf(1) as nics: + nic, = nics + network = {NETWORK_NAME: {'nic': nic, 'vlan': VLAN_ID, + 'bridged': False}} + status, msg = self.vdsm_net.setupNetworks(network, {}, + {'connectivityCheck': 0}) + self.assertEqual(status, SUCCESS, msg) + self.assertNetworkExists(NETWORK_NAME) + ipwrapper.linkDel(nic + '.' + VLAN_ID) + self.vdsm_net.refreshNetinfo() + self.assertNetworkDoesntExist(NETWORK_NAME) + status, msg = self.vdsm_net.setupNetworks(network, {}, + {'connectivityCheck': 0}) + self.assertEqual(status, SUCCESS, msg) + self.assertNetworkExists(NETWORK_NAME) + network[NETWORK_NAME] = {'remove': True} + status, msg = self.vdsm_net.setupNetworks(network, {}, + {'connectivityCheck': 0}) + self.assertEqual(status, SUCCESS, msg) + self.assertNetworkDoesntExist(NETWORK_NAME) diff --git a/tests/functional/utils.py b/tests/functional/utils.py index f80175e..d8bc939 100644 --- a/tests/functional/utils.py +++ b/tests/functional/utils.py @@ -139,6 +139,10 @@ self.vdscli.setSafeNetworkConfig() @netinfo_altering + def refreshNetinfo(self): + pass + + @netinfo_altering def restoreNetConfig(self): restoreNetConfig() diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 28c1200..97d3f09 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -27,6 +27,7 @@ from vdsm import utils from storage.misc import execCmd import neterrors as ne +from netconf import libvirtCfg from neterrors import ConfigNetworkError from vdsm import netinfo from netconf.ifcfg import ConfigWriter @@ -263,6 +264,10 @@ _netinfo.networks[network] = netAttr if _netinfo.networks[network]['bridged']: _netinfo.networks[network]['ports'] = ConfigWriter.ifcfgPorts(network) + elif not os.path.exists('/sys/class/net/' + netAttr['iface']): + # Bridgeless broken network without underlying device + libvirtCfg.removeNetwork(network) + return delNetwork(network, configurator=configurator, force=True, implicitBonding=False, _netinfo=_netinfo) -- To view, visit http://gerrit.ovirt.org/22436 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I92c641d91da9ef7eab876ed8c77fff7a9becb503 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-3.3 Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>

4 15

Change in vdsm[master]: utils.py: Add hostUUID retrieval support for KVM on PPC64
by Pradipta Kumar Banerjee 11 Jan '14

11 Jan '14

Pradipta Banerjee has uploaded a new change for review. Change subject: utils.py: Add hostUUID retrieval support for KVM on PPC64 ...................................................................... utils.py: Add hostUUID retrieval support for KVM on PPC64 Change-Id: I7a557b096550631e340fcc61bcb3fa4c3ae73837 Signed-off-by: Pradipta Kr. Banerjee <bpradip(a)in.ibm.com> --- M vdsm/utils.py 1 file changed, 16 insertions(+), 10 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/37/8637/1 diff --git a/vdsm/utils.py b/vdsm/utils.py index 0dbb342..80582d4 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -37,6 +37,7 @@ import functools import stat import glob +import platform import ethtool @@ -701,19 +702,23 @@ with open(constants.P_VDSM_NODE_ID) as f: __hostUUID = f.readline().replace("\n", "") else: - p = subprocess.Popen([constants.EXT_SUDO, + arch = platform.machine() + if arch == 'x86_64': + p = subprocess.Popen([constants.EXT_SUDO, constants.EXT_DMIDECODE, "-s", "system-uuid"], - close_fds=True, stdin=subprocess.PIPE, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) - out, err = p.communicate() - out = '\n'.join( line for line in out.splitlines() + close_fds=True, stdin=subprocess.PIPE, + stdout=subprocess.PIPE, stderr=subprocess.PIPE) + out, err = p.communicate() + out = '\n'.join( line for line in out.splitlines() if not line.startswith('#') ) - if p.returncode == 0 and 'Not' not in out: - #Avoid error string - 'Not Settable' or 'Not Present' - __hostUUID = out.strip() - else: - logging.warning('Could not find host UUID.') + if p.returncode == 0 and 'Not' not in out: + #Avoid error string - 'Not Settable' or 'Not Present' + __hostUUID = out.strip() + elif arch == 'ppc64': + if os.path.exists('/proc/device-tree/system-id'): + #eg. output IBM,03061C14A + __hostUUID = file('/proc/device-tree/system-id').readline().replace(",", "") try: mac = sorted(_getAllMacs())[0] @@ -724,6 +729,7 @@ if __hostUUID != "None": __hostUUID += "_" + mac else: + logging.warning('Could not find host UUID') __hostUUID = "_" + mac except: logging.error("Error retrieving host UUID", exc_info=True) -- To view, visit http://gerrit.ovirt.org/8637 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7a557b096550631e340fcc61bcb3fa4c3ae73837 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Pradipta Banerjee <bpradip(a)in.ibm.com>

8 30

Change in vdsm[ovirt-3.3]: vdsm: support VIR_MIGRATE_ABORT_ON_ERROR
by peet＠redhat.com 10 Jan '14

10 Jan '14

Hello Saggi Mizrahi, Antoni Segura Puimedon, Vinzenz Feenstra, Dan Kenigsberg, Michal Skrivanek, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/20434 to review the following change. Change subject: vdsm: support VIR_MIGRATE_ABORT_ON_ERROR ...................................................................... vdsm: support VIR_MIGRATE_ABORT_ON_ERROR Abort VM migration on EIO by default. The flag is supported since libvirt 1.0.1 upstream, so use it only in appropriate cluster versions. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=961154 Signed-off-by: Peter V. Saveliev <peet(a)redhat.com> Change-Id: Ic7f715c51f28ef2cd01fb95d42553ca10c79ea80 Reviewed-on: http://gerrit.ovirt.org/17422 Reviewed-by: Michal Skrivanek <michal.skrivanek(a)redhat.com> Reviewed-by: Antoni Segura Puimedon <asegurap(a)redhat.com> Reviewed-by: Saggi Mizrahi <smizrahi(a)redhat.com> Reviewed-by: Vinzenz Feenstra <vfeenstr(a)redhat.com> Reviewed-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/vm.py M vdsm_api/vdsmapi-schema.json 2 files changed, 9 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/34/20434/1 diff --git a/vdsm/vm.py b/vdsm/vm.py index 1a16d53..4e29b71 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -127,7 +127,7 @@ def __init__(self, vm, dst='', dstparams='', mode='remote', method='online', - tunneled=False, dstqemu='', **kwargs): + tunneled=False, dstqemu='', abortOnError=False, **kwargs): self.log = vm.log self._vm = vm self._dst = dst @@ -136,6 +136,7 @@ self._dstparams = dstparams self._machineParams = {} self._tunneled = utils.tobool(tunneled) + self._abortOnError = utils.tobool(abortOnError) self._dstqemu = dstqemu self._downtime = kwargs.get('downtime') or \ config.get('vars', 'migration_downtime') @@ -381,7 +382,9 @@ libvirt.VIR_MIGRATE_LIVE | libvirt.VIR_MIGRATE_PEER2PEER | (libvirt.VIR_MIGRATE_TUNNELLED if - self._tunneled else 0), + self._tunneled else 0) | + (libvirt.VIR_MIGRATE_ABORT_ON_ERROR if + self._abortOnError else 0), None, maxBandwidth) finally: t.cancel() diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 60b4bd5..aae6c6e 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -5726,6 +5726,9 @@ # # @tunneled: #optional use libvirt's tunneled migration. default it False. # +# @abortOnError: #optional Abort migration on I/O errors and refuse to migrate +# VMs, paused due to EIO +# # @dstqemu: #optional The destination's host address dedicated for migration. # # Since: 4.10.0 @@ -5733,7 +5736,7 @@ {'type': 'MigrateParams', 'data': {'vmId': 'UUID', 'dst': 'str', 'dstparams': 'str', '*mode': 'MigrateMode', '*method': 'MigrateMethod', - '*tunneled': 'bool', 'dstqemu': 'str'}} + '*tunneled': 'bool', '*abortOnError': 'bool', 'dstqemu': 'str'}} ## # @VM.migrate: -- To view, visit http://gerrit.ovirt.org/20434 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic7f715c51f28ef2cd01fb95d42553ca10c79ea80 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-3.3 Gerrit-Owner: Peter V. Saveliev <peet(a)redhat.com> Gerrit-Reviewer: Antoni Segura Puimedon <asegurap(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Michal Skrivanek <michal.skrivanek(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Vinzenz Feenstra <vfeenstr(a)redhat.com>

4 7

Change in vdsm[master]: Bug-Url: clientIF: Check device visibility on failed hotplug...
by Nir Soffer 09 Jan '14

09 Jan '14

Nir Soffer has uploaded a new change for review. Change subject: Bug-Url: clientIF: Check device visibility on failed hotplugDisk ...................................................................... Bug-Url: clientIF: Check device visibility on failed hotplugDisk According to the analysis in the bug, invoking vmHotplugDisk right after connectStorageServer may not leave enough time for multipath to create the device, leading to incorrect failure. The original fix failed to invoke multipath.rescan(), and the bug was verified by QE and used by customers for the last 6 month. We believe that the problem is a tiny race that is unlikely to happen, but it is possible and should be fixed. This patch implements the original fix in a different way, calling getDevicesVisibility(), which invokes multipath.rescan() if a device is missing. This is the second patch in series of patches fixing the regressions introduced by commit 15c7f74365cb5d2c9258bc333c441941c6e42bdb. Change-Id: I3d75ac53ca809b7a25d7fb237ec661dd865e31fc Bug-Url: https://bugzilla.redhat.com/923773 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/clientIF.py 1 file changed, 4 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/81/21181/1 diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index 137ba4e..367416d 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -282,15 +282,16 @@ # GUID drive format elif "GUID" in drive: - volPath = os.path.join("/dev/mapper", drive["GUID"]) - - if not os.path.exists(volPath): + res = self.irs.getDevicesVisibility([drive["GUID"]]) + if not res["visible"][drive["GUID"]]: raise vm.VolumeError(drive) res = self.irs.appropriateDevice(drive["GUID"], vmId) if res['status']['code']: raise vm.VolumeError(drive) + volPath = os.path.join("/dev/mapper", drive["GUID"]) + # UUID drive format elif "UUID" in drive: volPath = self._getUUIDSpecPath(drive["UUID"]) -- To view, visit http://gerrit.ovirt.org/21181 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3d75ac53ca809b7a25d7fb237ec661dd865e31fc Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

7 39

← Newer
1
...
23
24
25
26
27
28
29
...
50
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches December 2013