November 2012 - vdsm-patches - Fedora mailing-lists

Change in vdsm[master]: persist state in _changeBlockDev
by lvroyce＠linux.vnet.ibm.com 27 Oct '13

27 Oct '13

Royce Lv has uploaded a new change for review. Change subject: persist state in _changeBlockDev ...................................................................... persist state in _changeBlockDev persist state to make vm.conf survive recovery Change-Id: I220b72ebb360356092a2dccc25dfb9f63f638607 Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/8617/1 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index bd43b56..abd8440 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -2028,6 +2028,8 @@ ['message']}} self.cif.teardownVolumePath(self.conf.get(vmDev)) self.conf[vmDev] = path + # persist state to keep vm conf consistency in recovery + self.saveState() return {'status': doneCode, 'vmList': self.status()} def setTicket(self, otp, seconds, connAct, params): -- To view, visit http://gerrit.ovirt.org/8617 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I220b72ebb360356092a2dccc25dfb9f63f638607 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

2 2

Change in vdsm[master]: Move start_needed_srv and shutdown_conflicting_srv to vdsm-t...
by wenyi＠linux.vnet.ibm.com 27 Oct '13

27 Oct '13

Wenyi Gao has uploaded a new change for review. Change subject: Move start_needed_srv and shutdown_conflicting_srv to vdsm-tool ...................................................................... Move start_needed_srv and shutdown_conflicting_srv to vdsm-tool Change-Id: I54eca4b2cc3a17e9819a0188e5dc4c8bd28161da Signed-off-by: Wenyi Gao <wenyi(a)linux.vnet.ibm.com> --- M .gitignore M vdsm-tool/Makefile.am A vdsm-tool/base.py A vdsm-tool/needed_service.py.in M vdsm.spec.in M vdsm/vdsmd.init.in 6 files changed, 145 insertions(+), 42 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/59/8759/1 diff --git a/.gitignore b/.gitignore index 49a78ef..f9c26d4 100644 --- a/.gitignore +++ b/.gitignore @@ -41,6 +41,7 @@ vdsm/vdsmd.8 vdsm/vdsmd.init vdsm-tool/load_needed_modules.py +vdsm-tool/needed_service.py vdsm-tool/validate_ovirt_certs.py vdsm_cli/vdsClient vdsm_cli/vdscli.py diff --git a/vdsm-tool/Makefile.am b/vdsm-tool/Makefile.am index 997e339..0ed3226 100644 --- a/vdsm-tool/Makefile.am +++ b/vdsm-tool/Makefile.am @@ -27,7 +27,9 @@ dist_vdsmtool_DATA = \ __init__.py \ + base.py \ load_needed_modules.py \ + needed_service.py \ passwd.py \ validate_ovirt_certs.py \ $(NULL) diff --git a/vdsm-tool/base.py b/vdsm-tool/base.py new file mode 100644 index 0000000..9ded9a1 --- /dev/null +++ b/vdsm-tool/base.py @@ -0,0 +1,77 @@ +# Copyright IBM, Corp. 2012 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import subprocess + + +def exec_command(argv): + """ + This function executes a given shell command. + """ + try: + p = subprocess.Popen(argv, stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + out, err = p.communicate() + rc = p.returncode + except: + raise Exception('Failed to execute "%s"', ' '.join(argv), + exc_info=True) + + return (rc, out, err) + + +def call_init_function(function, *args, **kwargs): + """ + Call a function from /etc/init.d/functions. + """ + allowed_init_function = ['success', 'failure', 'pidofproc', + 'killproc', 'daemon'] + rc = -1 + srccmd = 'SYSTEMCTL_SKIP_REDIRECT=true source /etc/init.d/functions;' + + if function not in allowed_init_function: + print('Illegal function "%s"', function) + return rc + + tmpcmd = function + ' '.join('%s=%s' % item for item in kwargs.iteritems()) + tmpcmd += ' ' + ' '.join(args) + # Only allow to execute one shell command. + if any(c in tmpcmd for c in (';', '&&', '||')): + print('Illegal function parameters "%s"', tmpcmd) + return rc + + cmd = srccmd + tmpcmd + try: + rc = subprocess.call([cmd], shell=True) + except: + raise Exception('Failed to execute "%s"', cmd, exc_info=True) + + return rc + + +def print_msg(msg, success=True): + """ + Print message followed by a color string "[ OK ]" or "[Failed]". + """ + print msg, + if success: + call_init_function('success') + else: + call_init_function('failure') + print diff --git a/vdsm-tool/needed_service.py.in b/vdsm-tool/needed_service.py.in new file mode 100644 index 0000000..5266408 --- /dev/null +++ b/vdsm-tool/needed_service.py.in @@ -0,0 +1,61 @@ +# Copyright IBM, Corp. 2012 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import os + +from vdsm import constants +from vdsm.tool.base import exec_command, print_msg +from vdsm.tool import expose + +EX_CHKCONFIG = '@CHKCONFIG_PATH@' +EX_SERVICE = constants.EXT_SERVICE + + +@expose('shutdown-conflicting-srv') +def shutdown_conflicting_srv(): + """ + Shutdown conflicting service + """ + srv = 'libvirt-guests' + exec_command([EX_CHKCONFIG, srv, 'off']) + rc = exec_command([EX_SERVICE, srv, 'status'])[0] + if rc == 0: + if os.path.exists('/var/lock/subsys/libvirt-guests'): + os.unlink('/var/lock/subsys/libvirt-guests') + else: + exec_command([EX_SERVICE, srv, 'stop']) + return 0 + + +@expose('start-needed-srv') +def start_needed_srv(): + """ + Start needed service + """ + services = ['iscsid', 'multipathd', 'ntpd', 'wdmd', 'sanlock'] + for srv in services: + rc = exec_command([EX_SERVICE, srv, 'status'])[0] + if rc != 0: + print 'Starting %s...' % srv + rc = exec_command([EX_SERVICE, srv, 'start'])[0] + if rc != 0: + print_msg('vdsm: Dependent %s failed to start' % srv, False) + return rc + exec_command([EX_SERVICE, 'iscsid', 'force-start']) + return 0 diff --git a/vdsm.spec.in b/vdsm.spec.in index f61a36a..dc7ee24 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -771,6 +771,8 @@ %{python_sitearch}/%{vdsm_name}/tool/passwd.py* %{python_sitearch}/%{vdsm_name}/tool/validate_ovirt_certs.py* %{python_sitearch}/%{vdsm_name}/tool/load_needed_modules.py* +%{python_sitearch}/%{vdsm_name}/tool/needed_service.py* +%{python_sitearch}/%{vdsm_name}/tool/base.py* %files tests %doc %{_datadir}/%{vdsm_name}/tests/README diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 2b9ad11..35fb05e 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -32,8 +32,6 @@ CORE_DUMP_PATH=/var/log/core/core.%p.%t.dump DOM_METADATA_BACKUP_DIR=/var/log/vdsm/backup CORE_PATTERN=/proc/sys/kernel/core_pattern -NEEDED_SERVICES="iscsid multipathd ntpd wdmd sanlock" -CONFLICTING_SERVICES="libvirt-guests" LCONF=/etc/libvirt/libvirtd.conf QCONF=/etc/libvirt/qemu.conf @@ -130,49 +128,11 @@ fi } -shutdown_conflicting_srv() { - local srv - - for srv in $CONFLICTING_SERVICES - do - /sbin/chkconfig $srv off - if /sbin/service $srv status > /dev/null 2>&1; - then - if [ "$srv" == "libvirt-guests" ]; then - /bin/rm -f /var/lock/subsys/libvirt-guests - else - /sbin/service $srv stop - fi - fi - done - return 0 -} libvirt_should_use_upstart() { [[ -x /sbin/initctl ]] } -start_needed_srv() { - local srv - local ret_val - - for srv in $NEEDED_SERVICES - do - if ! /sbin/service $srv status > /dev/null 2>&1; - then - echo "Starting $srv..." - /sbin/service $srv start - ret_val=$? - if [ $ret_val -ne 0 ] - then - log_failure_msg "$prog: Dependent $srv failed to start" - return $ret_val - fi - fi - done - - /sbin/service iscsid force-start -} test_lo() { if ! LC_ALL=C /sbin/ifconfig lo | /bin/grep -q UP; @@ -428,7 +388,7 @@ local ret_val python @VDSMDIR@/hooks.pyc before_vdsm_start - shutdown_conflicting_srv && stop_libvirtd_sysv + /usr/bin/vdsm-tool shutdown-conflicting-srv && stop_libvirtd_sysv if ! @LIBEXECDIR@/vdsm-gencerts.sh --check; then echo -n $"Configuring a self-signed VDSM host certificate: " @@ -443,7 +403,7 @@ return $ret_val fi - start_needed_srv && start_libvirtd + /usr/bin/vdsm-tool start-needed-srv && start_libvirtd ret_val=$? if [ $ret_val -ne 0 ] then -- To view, visit http://gerrit.ovirt.org/8759 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I54eca4b2cc3a17e9819a0188e5dc4c8bd28161da Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Wenyi Gao <wenyi(a)linux.vnet.ibm.com>

3 7

Change in vdsm[master]: minimalistic (incomplete) test for mkimage.py with fail for ...
by josef.pacula＠gmail.com 25 Oct '13

25 Oct '13

Josef Pacula has uploaded a new change for review. Change subject: minimalistic (incomplete) test for mkimage.py with fail for non-64base data ...................................................................... minimalistic (incomplete) test for mkimage.py with fail for non-64base data Change-Id: I1a7656359beb5d116ad1919e237fe084a1208d71 Signed-off-by: Josef Pacula <josef.pacula(a)gmail.com> --- M tests/Makefile.am A tests/mkimageTests.py M vdsm/mkimage.py 3 files changed, 20 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/6275/1 -- To view, visit http://gerrit.ovirt.org/6275 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1a7656359beb5d116ad1919e237fe084a1208d71 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Josef Pacula <josef.pacula(a)gmail.com>

4 10

Change in vdsm[master]: Logging: config logging in new process of remoteFileHandler
by zhshzhou＠linux.vnet.ibm.com 13 Oct '13

13 Oct '13

Zhou Zheng Sheng has uploaded a new change for review. Change subject: Logging: config logging in new process of remoteFileHandler ...................................................................... Logging: config logging in new process of remoteFileHandler When new process of Python is started by PoolHandler.__init__() to run remoteFileHandler.py, the logging configurations are lost. This patch sets the configuration for new process of remoteFileHandler.py correctly. This patch deals with the logging problem described in the following bug report in the last paragraph of the "Additional info" section. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=875678 Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/storage/remoteFileHandler.py 1 file changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/9182/1 diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index edf8e68..34de3ef 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -32,6 +32,13 @@ from contextlib import contextmanager from threading import Thread +from vdsm import constants + +if __name__ == "__main__": + from logging import config as lconfig + loggerConfFile = constants.P_VDSM_CONF + 'logger.conf' + lconfig.fileConfig(loggerConfFile) + import misc import fileUtils -- To view, visit http://gerrit.ovirt.org/9182 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1b3b9e2837d632c7532fcd8f7306ed50b0865b5c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

10 45

Change in vdsm[master]: Fix problems with current implementation of forceIscsiRescan
by smizrahi＠redhat.com 29 Sep '13

29 Sep '13

Saggi Mizrahi has uploaded a new change for review. Change subject: Fix problems with current implementation of forceIscsiRescan ...................................................................... Fix problems with current implementation of forceIscsiRescan - Move configuration access out to HSM. - Fix error in logging when reading a bad configuration file. - Clean up slow child processes so we don't have a zombie leak. - Fix performance degradation (minTimeout) for fast HBAs. Change-Id: Ic4e7173086ba15c7706206c5ee1473ed6d334f9e Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M tests/Makefile.am A tests/iscsiTests.py M vdsm/config.py.in M vdsm/storage/hsm.py M vdsm/storage/iscsi.py M vdsm/storage/sdc.py 6 files changed, 67 insertions(+), 37 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/8172/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 52dfda0..98e9112 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -32,6 +32,7 @@ guestIFTests.py \ hooksTests.py \ lsblkTests.py \ + iscsiTests.py \ main.py \ md_utils_tests.py \ miscTests.py \ diff --git a/tests/iscsiTests.py b/tests/iscsiTests.py new file mode 100644 index 0000000..63138fe --- /dev/null +++ b/tests/iscsiTests.py @@ -0,0 +1,14 @@ +from storage import iscsi +from testrunner import VdsmTestCase as TestCaseBase + + +class IscsiForceRescanTests(TestCaseBase): + def testThatRuns(self): + #TODO: It's quite complex actually checking that it did the right + # thing. For now just make sure that it runs. + iscsi.forceIScsiScan(30) + + def testTimeoutFlow(self): + #TODO: It's quite complex actually checking that it did the right + # thing. For now just make sure that it runs. + iscsi.forceIScsiScan(0) diff --git a/vdsm/config.py.in b/vdsm/config.py.in index df85e7e..5dbede3 100644 --- a/vdsm/config.py.in +++ b/vdsm/config.py.in @@ -176,11 +176,8 @@ 'Storage domain validate timeout, the maximum number of seconds ' 'to wait until all the domains will be validated.'), - ('scsi_rescan_minimal_timeout', '2', - 'The minimum number of seconds to wait for scsi scan to return.'), - - ('scsi_rescan_maximal_timeout', '30', - 'The maximal number of seconds to wait for scsi scan to return.'), + ('scsi_rescan_timeout', '30', + 'The number of seconds to wait for scsi scan to return.'), ('sd_health_check_delay', '10', 'Storage domain health check delay, the amount of seconds to ' diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index a89274a..b54ee18 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -293,6 +293,13 @@ def storageRefresh(): lvm._lvminfo.bootstrap() + iscsiRescanTimeout = config.getint('irs', 'scsi_rescan_timeout') + if (iscsiRescanTimeout < 0): + self.log.warning("scsi_rescan_timeout as an invalid value, " + "using the default: %d", + sdCache.iscsiRescanTimeout) + + sdCache.iscsiRescanTimeout = iscsiRescanTimeout sdCache.refreshStorage() self.tasksDir = config.get('irs', 'hsm_tasks') diff --git a/vdsm/storage/iscsi.py b/vdsm/storage/iscsi.py index 578b700..4e8f25a 100644 --- a/vdsm/storage/iscsi.py +++ b/vdsm/storage/iscsi.py @@ -29,10 +29,11 @@ import errno import time from collections import namedtuple +import select +import threading from vdsm import constants import misc -from vdsm.config import config import devicemapper from threading import RLock @@ -349,43 +350,52 @@ except iscsiadm.IscsiError: pass +# FIXME: Because of sampling method semantics, the first caller in each round +# will select the timeout. This doesn't matter as currently the timeout never +# changes. This shows a fundamental problem with this interface. Changing this +# to something more generic will take to much work and is not necessary at the +# moment. @misc.samplingmethod -def forceIScsiScan(): - processes = [] - minTimeout = config.getint('irs', 'scsi_rescan_minimal_timeout') - maxTimeout = config.getint('irs', 'scsi_rescan_maximal_timeout') +def forceIScsiScan(timeout): + poller = select.poll() + procs = {} + for hba in glob.glob(SCAN_PATTERN): cmd = [constants.EXT_DD, 'of=' + hba] p = misc.execCmd(cmd, sudo=False, sync=False) p.stdin.write("- - -") p.stdin.flush() p.stdin.close() - processes.append((hba, p)) - if (minTimeout > maxTimeout or minTimeout < 0): - minTimeout = 2 - maxTimeout = 30 - log.warning("One of the following configuration arguments has an ", - "illegal value: scsi_rescan_minimal_timeout or ", - "scsi_rescan_maximal_timeout. Set to %s and %s seconds ", - "respectively.", minTimeout, maxTimeout) - log.debug("Performing SCSI scan, this will take up to %s seconds", - maxTimeout) - time.sleep(minTimeout) - for i in xrange(maxTimeout - minTimeout): - for p in processes[:]: - (hba, proc) = p - if proc.wait(0): - if proc.returncode != 0: - log.warning('returncode for: %s is: %s', hba, - proc.returncode) - processes.remove(p) - if not processes: - break - else: - time.sleep(1) - else: - log.warning("Still waiting for scsi scan of hbas: %s", - tuple(hba for p in processes)) + fd = p.stdout.fileno() + poller.register(fd, 0) # Only HUP and ERR + procs[fd] = p + + startTime = time.time() + timeLeft = timeout + while (len(procs) > 0) and (timeLeft > 0): + for fd, event in misc.NoIntrPoll(poller.poll, timeout): + poller.unregister(fd) + p = procs.pop(fd) + p.wait() + + timeLeft = timeout - (time.time() - startTime) + + if len(procs) == 0: + return + + log.warn("Not all rescan completed in alotted time") + + # We have to put on a thread that waits for the remaining processes, + # otherwise we will end up with zombie children + + def collectSlowProcs(procs): + for p in procs.itervalues(): + p.wait() + + t = threading.Thread(target=collectSlowProcs, args=(procs,)) + t.setDaemon(True) + t.start() + def devIsiSCSI(dev): hostdir = os.path.realpath(os.path.join("/sys/block", dev, "device/../../..")) diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index 90ccc35..bc73291 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -64,6 +64,7 @@ self.__domainCache = {} self.storage_repo = storage_repo self.storageStale = True + self.iscsiRescanTimeout = 30 def invalidateStorage(self): self.storageStale = True @@ -71,7 +72,7 @@ @misc.samplingmethod def refreshStorage(self): - multipath.rescan() + multipath.rescan(self.iscsiRescanTimeout) lvm.invalidateCache() self.storageStale = False -- To view, visit http://gerrit.ovirt.org/8172 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic4e7173086ba15c7706206c5ee1473ed6d334f9e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

5 13

Change in vdsm[master]: pep8 clean vdsm/storage/securable.py
by xiawenc＠linux.vnet.ibm.com 27 Sep '13

27 Sep '13

Wenchao Xia has uploaded a new change for review. Change subject: pep8 clean vdsm/storage/securable.py ...................................................................... pep8 clean vdsm/storage/securable.py Change-Id: Ie1c5b97b22ffc921e17e4cef2e79244d5ad190c7 Signed-off-by: wenchao xia <xiawenc(a)linux.vnet.ibm.com> --- M Makefile.am M vdsm/storage/securable.py 2 files changed, 10 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/4682/1 -- To view, visit http://gerrit.ovirt.org/4682 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie1c5b97b22ffc921e17e4cef2e79244d5ad190c7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Wenchao Xia <xiawenc(a)linux.vnet.ibm.com>

4 4

Change in vdsm[master]: Qunatum POC changes
by gkotton＠redhat.com 27 Sep '13

27 Sep '13

Gary Kotton has uploaded a new change for review. Change subject: Qunatum POC changes ...................................................................... Qunatum POC changes Change-Id: I992a0dd278daefa27e87b696c584bd7cd590c78e Signed-off-by: Gary Kotton <gkotton(a)redhat.com> --- M .gitignore M vdsm/libvirtvm.py A vdsm/quantum.py 3 files changed, 144 insertions(+), 29 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/43/4043/1 -- To view, visit http://gerrit.ovirt.org/4043 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I992a0dd278daefa27e87b696c584bd7cd590c78e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Gary Kotton <gkotton(a)redhat.com> Gerrit-Reviewer: Federico Simoncelli <fsimonce(a)redhat.com>

4 5

Change in vdsm[master]: Title: fix some code style consist with pep8
by baichm＠linux.vnet.ibm.com 27 Sep '13

27 Sep '13

Bruce Bai has uploaded a new change for review. Change subject: Title: fix some code style consist with pep8 ...................................................................... Title: fix some code style consist with pep8 content: fix some storage code files pep8 style. Change-Id: I50cf61d9b7815cbdd5571930e3f9be59183a83f4 Signed-off-by: Changming Bai <baichm(a)linux.vnet.ibm.com> --- M Makefile.am M vdsm/storage/__init__.py M vdsm/storage/safelease.py M vdsm/storage/storageConstants.py M vdsm/storage/storage_exception.py M vdsm/storage/sync.py 6 files changed, 373 insertions(+), 33 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/4500/1 -- To view, visit http://gerrit.ovirt.org/4500 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I50cf61d9b7815cbdd5571930e3f9be59183a83f4 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Bruce Bai <baichm(a)linux.vnet.ibm.com>

3 5

Change in vdsm[master]: Modify before_vm_migrate_source.py for PEP8 compliance
by qiaoliyong＠gmail.com 27 Sep '13

27 Sep '13

Eli Qiao has uploaded a new change for review. Change subject: Modify before_vm_migrate_source.py for PEP8 compliance ...................................................................... Modify before_vm_migrate_source.py for PEP8 compliance Change-Id: I33b157093350f0c96639836175d2b313fd684984 Signed-off-by: Eli Qiao <qiaoliyong(a)gmail.com> --- M vdsm_hooks/hostusb/before_vm_migrate_source.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/31/4331/1 -- To view, visit http://gerrit.ovirt.org/4331 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I33b157093350f0c96639836175d2b313fd684984 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eli Qiao <qiaoliyong(a)gmail.com>

4 4

Change in vdsm[master]: Modify before_vm_migrate_source.py for PEP8 compliance
by qiaoliyong＠gmail.com 27 Sep '13

27 Sep '13

Eli Qiao has uploaded a new change for review. Change subject: Modify before_vm_migrate_source.py for PEP8 compliance ...................................................................... Modify before_vm_migrate_source.py for PEP8 compliance Change-Id: I0b02d00083eb0390be4708f122c94f6682c03359 Signed-off-by: Eli Qiao <qiaoliyong(a)gmail.com> --- M vdsm_hooks/sriov/before_vm_migrate_source.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/32/4332/1 -- To view, visit http://gerrit.ovirt.org/4332 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0b02d00083eb0390be4708f122c94f6682c03359 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eli Qiao <qiaoliyong(a)gmail.com>

4 4

Change in vdsm[master]: Modify ./vdsm/__init__.py for PEP8 compliance ./vdsm/__init_...
by qiaoliyong＠gmail.com 27 Sep '13

27 Sep '13

Eli Qiao has uploaded a new change for review. Change subject: Modify ./vdsm/__init__.py for PEP8 compliance ./vdsm/__init__.py ...................................................................... Modify ./vdsm/__init__.py for PEP8 compliance ./vdsm/__init__.py Change-Id: Ifca40b37c8a4a01d30589e33d49016975ed0f53e Signed-off-by: Eli Qiao <qiaoliyong(a)gmail.com> --- M vdsm/__init__.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/33/4333/1 -- To view, visit http://gerrit.ovirt.org/4333 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifca40b37c8a4a01d30589e33d49016975ed0f53e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eli Qiao <qiaoliyong(a)gmail.com>

5 5

Change in vdsm[master]: pep8 clean vdsm/storage/hba.py
by xiawenc＠linux.vnet.ibm.com 27 Sep '13

27 Sep '13

Wenchao Xia has uploaded a new change for review. Change subject: pep8 clean vdsm/storage/hba.py ...................................................................... pep8 clean vdsm/storage/hba.py Change-Id: I5afe01d5cf3a6018f6d513a04b3b0e37335c5675 Signed-off-by: wenchao xia <xiawenc(a)linux.vnet.ibm.com> --- M Makefile.am M vdsm/storage/hba.py 2 files changed, 8 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/4382/1 -- To view, visit http://gerrit.ovirt.org/4382 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5afe01d5cf3a6018f6d513a04b3b0e37335c5675 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Wenchao Xia <xiawenc(a)linux.vnet.ibm.com>

6 11

Change in vdsm[master]: hypervisor autodetection for the vdsm capabilities
by fromani＠gmail.com 24 Sep '13

24 Sep '13

Francesco Romani has uploaded a new change for review. Change subject: hypervisor autodetection for the vdsm capabilities ...................................................................... hypervisor autodetection for the vdsm capabilities This is a follow up and an extension of the Change I79f4ab08: add and use hypervisor autodetection in bootstrap. (see http://gerrit.ovirt.org/#/c/7657 ) The vdsm capability reporting code is enhanced to autonomously autodetect and report if the vdsm is running under an hypervisor, thus if the virtualization support has to be emulated. The hypervisor detection code is the same as the patch mentioned above, updated following the comment received so far. The autodetection code is moved in a separate little module (`cpu_utils.py') which can be shared both by the vdsm and the vds_bootstrap code. So far, the two patches are independent and so they use independent (duplicated) detection code, but a further patch (or an extension of the current one, if preferred) is planned in order to share the same module, minimizing the duplication to the point which I believe is the minimum possible. With this change and the one mentioned above both applied, there is no longer need of the `fake_kvm_support' configuration variable. In the proposed patch it is left as last resort/fallback option, but it can be removed with a further patch, planned as well. Change-Id: I565ef639e1f74bdbaef93060f4f25c75085d361b Signed-off-by: Francesco Romani <fromani(a)gmail.com> --- M vdsm.spec.in M vdsm/Makefile.am M vdsm/caps.py A vdsm/cpu_utils.py 4 files changed, 82 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/84/7884/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index bb6730e..fb8bb01 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -559,6 +559,7 @@ %{_datadir}/%{vdsm_name}/blkid.py* %{_datadir}/%{vdsm_name}/caps.py* %{_datadir}/%{vdsm_name}/clientIF.py* +%{_datadir}/%{vdsm_name}/cpu_utils.py* %{_datadir}/%{vdsm_name}/API.py* %{_datadir}/%{vdsm_name}/hooking.py* %{_datadir}/%{vdsm_name}/hooks.py* diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 96c8a56..662190d 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -31,6 +31,7 @@ caps.py \ clientIF.py \ configNetwork.py \ + cpu_utils.py \ debugPluginClient.py \ guestIF.py \ hooking.py \ diff --git a/vdsm/caps.py b/vdsm/caps.py index f1641ff..afc88dd 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -41,6 +41,7 @@ from vdsm import utils from vdsm import constants import storage.hba +import cpu_utils # For debian systems we can use python-apt if available try: @@ -228,11 +229,18 @@ return __osversion +(a)utils.memoized +def _getHypervisorName(): + return cpu_utils.findHypervisor() + + def get(): caps = {} + fake_kvm = bool(_getHypervisorName()) or \ + config.getboolean('vars', 'fake_kvm_support') caps['kvmEnabled'] = \ - str(config.getboolean('vars', 'fake_kvm_support') or + (str(fake_kvm) or os.path.exists('/dev/kvm')).lower() cpuInfo = CpuInfo() @@ -243,7 +251,7 @@ caps['cpuSockets'] = str(cpuInfo.sockets()) caps['cpuSpeed'] = cpuInfo.mhz() - if config.getboolean('vars', 'fake_kvm_support'): + if fake_kvm: caps['cpuModel'] = 'Intel(Fake) CPU' flags = set(cpuInfo.flags() + ['vmx', 'sse2', 'nx']) caps['cpuFlags'] = ','.join(flags) + 'model_486,model_pentium,' \ @@ -276,6 +284,7 @@ config.getint('vars', 'host_mem_reserve') + config.getint('vars', 'extra_mem_reserve')) caps['guestOverhead'] = config.get('vars', 'guest_ram_overhead') + caps['hypervisorName'] = _getHypervisorName() return caps diff --git a/vdsm/cpu_utils.py b/vdsm/cpu_utils.py new file mode 100644 index 0000000..c3f402e --- /dev/null +++ b/vdsm/cpu_utils.py @@ -0,0 +1,69 @@ +# +# Copyright 2008-2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +""" +A module containing miscellaneous cpu detection functions +used by various vdsm components. +""" + +import struct +import logging +import traceback + + +def cpuHypervisorID(): + # we cannot (yet) use _cpuid because of the different + # unpack format. + HYPERVISOR_CPUID_LEAF = 0x40000000 + with open('/dev/cpu/0/cpuid') as f: + f.seek(HYPERVISOR_CPUID_LEAF) + c = struct.unpack('I12s', f.read(16)) + return c[1].strip('\x00') + return '' + +def cpuModelName(): + with open('/proc/cpuinfo') as f: + for line in f: + if ':' in line: + k, v = line.split(':', 1) + k = k.strip() + if k == 'model name': + return v.strip() + return '' + +def findHypervisor(): + name = '' + try: + hid = cpuHypervisorID() + if hid == 'VMwareVMware': + name = 'vmware' + elif hid == 'Microsoft Hv': + name = 'hyperv' + elif hid == 'XenVMMXenVMM': + name = 'xen' + elif hid == 'KVMKVMKVM': + name = 'kvm' + elif 'QEMU' in cpuModelName(): + name = 'qemu' + logging.debug('detected hypervisor: %s', name) + except: + logging.error(traceback.format_exc()) + return name + -- To view, visit http://gerrit.ovirt.org/7884 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I565ef639e1f74bdbaef93060f4f25c75085d361b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)gmail.com>

3 2

Change in vdsm[master]: Integrate deathSignal with execCmd
by smizrahi＠redhat.com 19 Sep '13

19 Sep '13

Saggi Mizrahi has uploaded a new change for review. Change subject: Integrate deathSignal with execCmd ...................................................................... Integrate deathSignal with execCmd Change-Id: Ice79d10ae6bfb0dbb61633fe8a3fce1e26943f14 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/storage/misc.py 1 file changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/42/6242/1 -- To view, visit http://gerrit.ovirt.org/6242 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ice79d10ae6bfb0dbb61633fe8a3fce1e26943f14 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

7 11

Change in vdsm[master]: Avoid race condition with vdsm-reg during firstboot
by fgranha＠linux.vnet.ibm.com 15 Sep '13

15 Sep '13

Fernando Granha Jeronimo has uploaded a new change for review. Change subject: Avoid race condition with vdsm-reg during firstboot ...................................................................... Avoid race condition with vdsm-reg during firstboot In the vdsm-config script, the vdsm-reg service is disabled to avoid race conditions, once it is intended to be executed just before the node is rebooted and the registration process should take place only after reboot. Change-Id: I2a215bbe6c467d10cf414b09cdaee5340092198e Signed-off-by: Fernando Granha Jeronimo <fgranha(a)linux.vnet.ibm.com> --- M vdsm_reg/vdsm-config 1 file changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/8200/1 diff --git a/vdsm_reg/vdsm-config b/vdsm_reg/vdsm-config index 968d78c..0a030d1 100755 --- a/vdsm_reg/vdsm-config +++ b/vdsm_reg/vdsm-config @@ -14,6 +14,16 @@ echo "vdsm-config: starting" | tee $LOG + # Disable vdsm-reg service to avoid race condition + # when parameters are written to config file and the + # service is running. This is necessary, because the + # registration process can take place only after reboot. + if [ -x /bin/systemctl ] ; then + /bin/systemctl stop vdsm-reg.service > /dev/null 2>&1 + elif [ -x /sbin/service ] ; then + /sbin/service vdsm-reg stop > /dev/null 2>&1 + fi + set_vars() { echo "[vars]" >> $VDSM_CONFIG #Adding ts for the coming scripts. echo "trust_store_path = " `$GETCONFITEM $VDSM_CONFIG vars trust_store_path /etc/pki/vdsm` >> $VDSM_CONFIG -- To view, visit http://gerrit.ovirt.org/8200 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2a215bbe6c467d10cf414b09cdaee5340092198e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Fernando Granha Jeronimo <fgranha(a)linux.vnet.ibm.com>

5 4

Change in vdsm[master]: [WIP]change startup process for vdsm and supervdsm
by lvroyce＠linux.vnet.ibm.com 15 Sep '13

15 Sep '13

Royce Lv has uploaded a new change for review. Change subject: [WIP]change startup process for vdsm and supervdsm ...................................................................... [WIP]change startup process for vdsm and supervdsm Using vdsmService to take care of start up and respawn process of vdsm and supervdsm. Remove the function of vdsm to start previlege supervdsm process. Change-Id: I69aae6b0b9529c80291d90c6ad14ff82b21aea53 Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> --- M vdsm.spec.in M vdsm/Makefile.am M vdsm/storage/misc.py M vdsm/supervdsm.py M vdsm/supervdsmServer.py M vdsm/vdsm A vdsm/vdsmService M vdsm/vdsmd.init.in M vdsm_reg/Makefile.am 9 files changed, 208 insertions(+), 63 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/45/4145/1 -- To view, visit http://gerrit.ovirt.org/4145 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I69aae6b0b9529c80291d90c6ad14ff82b21aea53 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

10 38

Change in vdsm[master]: Modify vdsm/netinfo.py for PEP8 compliance
by qiaoliyong＠gmail.com 15 Sep '13

15 Sep '13

Eli Qiao has uploaded a new change for review. Change subject: Modify vdsm/netinfo.py for PEP8 compliance ...................................................................... Modify vdsm/netinfo.py for PEP8 compliance Change-Id: I5a8370da88235ff0edce7cf7b1a350fb0402e7d9 Signed-off-by: Eli Qiao <qiaoliyong(a)gmail.com> --- M vdsm/netinfo.py 1 file changed, 62 insertions(+), 28 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/34/4334/1 -- To view, visit http://gerrit.ovirt.org/4334 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5a8370da88235ff0edce7cf7b1a350fb0402e7d9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eli Qiao <qiaoliyong(a)gmail.com>

6 10

Change in vdsm[master]: Related to BZ# - One shot prepare.
by ewarszaw＠redhat.com 09 Sep '13

09 Sep '13

Eduardo has uploaded a new change for review. Change subject: Related to BZ# - One shot prepare. ...................................................................... Related to BZ# - One shot prepare. The number of storage accesses is reduced to 1 or none instead of the recursive volume images produces. Change-Id: Id47db9c53199385dd5d08586e4782ea23885eb72 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 7 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/20/4220/1 -- To view, visit http://gerrit.ovirt.org/4220 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id47db9c53199385dd5d08586e4782ea23885eb72 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

7 47

Change in vdsm[master]: Adding more qemuImg commands
by Federico Simoncelli 06 Sep '13

06 Sep '13

Federico Simoncelli has uploaded a new change for review. Change subject: Adding more qemuImg commands ...................................................................... Adding more qemuImg commands Change-Id: I1fbe822da8544ece4fcce9cb5b15a39a9bb35a64 --- M vdsm/qemuImg.py 1 file changed, 48 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/2612/1 -- To view, visit http://gerrit.ovirt.org/2612 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1fbe822da8544ece4fcce9cb5b15a39a9bb35a64 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>

3 6

Change in vdsm[master]: Related to BZ#769502 - One shot prepare.
by ewarszaw＠redhat.com 31 Jul '13

31 Jul '13

Eduardo has uploaded a new change for review. Change subject: Related to BZ#769502 - One shot prepare. ...................................................................... Related to BZ#769502 - One shot prepare. The number of storage accesses is reduced to 1 or none instead of the recursive volume images produces. Change-Id: I4b6a4e99cf3a657affb4f5e96aa1ac1978a297da Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/hsm.py M vdsm/storage/image.py 2 files changed, 5 insertions(+), 13 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/34/4234/1 -- To view, visit http://gerrit.ovirt.org/4234 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4b6a4e99cf3a657affb4f5e96aa1ac1978a297da Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

6 18

Change in vdsm[master]: deployUtil: Add logging.error() to CPU virt check
by Douglas Schilling Landgraf 25 Jul '13

25 Jul '13

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: deployUtil: Add logging.error() to CPU virt check ...................................................................... deployUtil: Add logging.error() to CPU virt check Adding logging.error() msg to help users in case CPU virt support is disable in BIOS. Change-Id: I3baff7594ad2cfcf4844a30586a2e4a4313a1972 Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M vdsm_reg/deployUtil.py.in 1 file changed, 3 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/49/3549/1 -- To view, visit http://gerrit.ovirt.org/3549 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3baff7594ad2cfcf4844a30586a2e4a4313a1972 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

4 10

Change in vdsm[master]: Fix MountError exception handling in MountConnection.connect...
by wudxw＠linux.vnet.ibm.com 08 Jul '13

08 Jul '13

Mark Wu has uploaded a new change for review. Change subject: Fix MountError exception handling in MountConnection.connect() ...................................................................... Fix MountError exception handling in MountConnection.connect() The old code swallowed the exception of MountError if dir is removed successfully. The patches re-raise the exception, which could be MountError or OSError. In any case, it will stop executing the following code. Change-Id: Ic6d9ab419d717345c81fa8e8787e089ccd5a890f Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/storage/storageServer.py 1 file changed, 3 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/03/9503/1 diff --git a/vdsm/storage/storageServer.py b/vdsm/storage/storageServer.py index 383e04c..6136777 100644 --- a/vdsm/storage/storageServer.py +++ b/vdsm/storage/storageServer.py @@ -202,7 +202,9 @@ os.rmdir(self._getLocalPath()) except OSError as e: if e.errno != errno.ENOENT: - raise + self.log.warn("Error while removing dir %s", + self._getLocalPath(), exc_info=True) + raise try: fileSD.validateDirAccess(self.getMountObj().getRecord().fs_file) -- To view, visit http://gerrit.ovirt.org/9503 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic6d9ab419d717345c81fa8e8787e089ccd5a890f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

5 8

Change in vdsm[master]: Enable logging facility for test cases
by wudxw＠linux.vnet.ibm.com 25 Jun '13

25 Jun '13

Mark Wu has uploaded a new change for review. Change subject: Enable logging facility for test cases ...................................................................... Enable logging facility for test cases In current code, there's no log handler configured. So make use of the logging configured by nose.config for test cases. Change-Id: Iaf727a2dfe38206737e0711466978277cd95ad34 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M tests/testrunner.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/3712/1 -- To view, visit http://gerrit.ovirt.org/3712 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iaf727a2dfe38206737e0711466978277cd95ad34 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

5 7

Change in vdsm[master]: BZ#836161 - Clean up.
by ewarszaw＠redhat.com 16 Jun '13

16 Jun '13

Eduardo has uploaded a new change for review. Change subject: BZ#836161 - Clean up. ...................................................................... BZ#836161 - Clean up. Change-Id: Icf01de7dc8bf0a903ebf94d9e33ffad9a195b7de Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 0 insertions(+), 71 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/07/8507/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 78fbe66..0ade40d 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1766,73 +1766,6 @@ image.Image(repoPath).multiMove(srcDomUUID, dstDomUUID, imgDict, vmUUID, force) - def deleteImage(self, sdUUID, imgUUID, postZero, force): - """ - Deletes an Image folder with all its volumes. - - This function assumes that imgUUID is locked. - In addition to removing image, this function also does the following: - If removing a template from a backup SD which has dependent images: - - creates a fake template. - If removing the last image which depends on a fake template: - - removes the fake template as well - :param sdUUID: The UUID of the storage domain that contains the images. - :type sdUUID: UUID - :param imgUUID: The UUID of the image you want to delete. - :type imgUUID: UUID - :param postZero: bool - :param force: Should the operation be forced. - :type force: bool - """ - # TODO: This function works on domains. No relation with pools. - # Therefore move this to the relevant *sd module - repoPath = os.path.join(self.storage_repository, self.spUUID) - img = image.Image(repoPath) - dom = sdCache.produce(sdUUID) - allVols = dom.getAllVolumes() - # Filter volumes related to this image - imgsByVol = sd.getVolsOfImage(allVols, imgUUID) - if all(len(v.imgs) == 1 for k, v in imgsByVol.iteritems()): - # This is a self contained regular image, i.e. it's either an image - # which is not based on a template or a template which has no - # derived images, e.g. not derived from a template - img.delete(sdUUID=sdUUID, imgUUID=imgUUID, postZero=postZero, force=force) - else: - # This is either a template with derived images or a derived image - # so needs further scrutiny - ts = tuple((volName, vol.imgs) for volName, vol in - imgsByVol.iteritems() if len(vol.imgs) > 1) - if len(ts) != 1: - raise se.ImageValidationError("Image points to multiple" - "templates %s in %s from %s" % \ - (ts, imgsByVol, allVols)) - # TODO: Lock the template, reload allVols. - # template = ts[0] = [(tName, tImgs)] - tName, tImgs = ts[0] - # getAllVolumes makes the template self img the 1st one in tImgs - templateImage = tImgs[0] - numOfDependentImgs = len(tImgs) - 1 - if templateImage != imgUUID: - # Removing an image based on a template - img.delete(sdUUID=sdUUID, imgUUID=imgUUID, postZero=postZero, force=force) - if numOfDependentImgs == 1 and dom.produceVolume(templateImage, tName).isFake(): - # Remove the fake template since last consumer was removed - img.delete(sdUUID=sdUUID, imgUUID=templateImage, postZero=False, force=True) - - # Removing a template with dependencies - elif force: - img.delete(sdUUID=sdUUID, imgUUID=templateImage, postZero=postZero, - force=force) - elif not dom.isBackup(): - raise se.ImagesActionError("Can't remove template with children %s", - allVols) - else: - # Removing a template with dependencies in backup domain - # A fake template will be created - img.delete(sdUUID=sdUUID, imgUUID=imgUUID, postZero=postZero, force=True) - tParams = dom.produceVolume(imgUUID, tName).getVolumeParams() - img.createFakeTemplate(sdUUID=sdUUID, volParams=tParams) - def mergeSnapshots(self, sdUUID, vmUUID, imgUUID, ancestor, successor, postZero): """ Merges the source volume to the destination volume. @@ -2012,10 +1945,6 @@ def preDeleteRename(self, sdUUID, imgUUID): repoPath = os.path.join(self.storage_repository, self.spUUID) return image.Image(repoPath).preDeleteRename(sdUUID, imgUUID) - - def validateDelete(self, sdUUID, imgUUID): - repoPath = os.path.join(self.storage_repository, self.spUUID) - image.Image(repoPath).validateDelete(sdUUID, imgUUID) def validateVolumeChain(self, sdUUID, imgUUID): repoPath = os.path.join(self.storage_repository, self.spUUID) -- To view, visit http://gerrit.ovirt.org/8507 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icf01de7dc8bf0a903ebf94d9e33ffad9a195b7de Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

8 25

Change in vdsm[master]: packaging: move /rhev/data-center to /var/lib/vdsm/data-cent...
by Alon Bar-Lev 25 May '13

25 May '13

Alon Bar-Lev has uploaded a new change for review. Change subject: packaging: move /rhev/data-center to /var/lib/vdsm/data-center ...................................................................... packaging: move /rhev/data-center to /var/lib/vdsm/data-center rhev is a trademark should not be in ovirt. It is not FHS compliant to add /whatever to filesystem. Change-Id: I4ed316fedbf2e4d7e5bf741b7c77a39704994ae7 Signed-off-by: Alon Bar-Lev <alonbl(a)redhat.com> --- M configure.ac M tests/getAllVolumesTests.py M vdsm.spec.in M vdsm/storage/fileSD.py M vdsm/storage/fileVolume.py 5 files changed, 12 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/8500/1 diff --git a/configure.ac b/configure.ac index 450f714..665f1c6 100644 --- a/configure.ac +++ b/configure.ac @@ -75,7 +75,7 @@ AC_SUBST([vdsmexecdir], ['${libexecdir}/vdsm']) AC_SUBST([vdsmhooksdir], ['${vdsmexecdir}/hooks']) AC_SUBST([vdsmtsdir], ['${sysconfdir}/pki/vdsm']) -AC_SUBST([vdsmrepo], ['/rhev/data-center']) +AC_SUBST([vdsmrepo], ['${vdsmlibdir}/data-center']) AC_SUBST([vdsmpylibdir], ['${pyexecdir}/vdsm']) AC_SUBST([vdsmtooldir], ['${vdsmpylibdir}/tool']) AC_SUBST([vdsmtestsdir], ['${datarootdir}/vdsm/tests']) diff --git a/tests/getAllVolumesTests.py b/tests/getAllVolumesTests.py index ffc12c5..fe4c7b9 100644 --- a/tests/getAllVolumesTests.py +++ b/tests/getAllVolumesTests.py @@ -62,7 +62,7 @@ Input file name: 'glob_<sdName>.out Input file format: str(glob.glob(<imgsDir>)) When - <imgsDir> = /rhev/data-center/mnt/<mntPoint>/<sdName>/images/*.meta + <imgsDir> = @vdsmlibdir@/data-center/mnt/<mntPoint>/<sdName>/images/*.meta """ sdPath, globExp = volMetaPattern.split('/images') sdHead, sdName = os.path.split(sdPath) diff --git a/vdsm.spec.in b/vdsm.spec.in index 716489a..8e9fbba 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -460,6 +460,14 @@ fi %post +# upgrade old locations +if [ -d /rhev/data-center ]; then + mv /rhev/data-center %{_localstatedir}/lib/vdsm/data-center.new && \ + rm -rf %{_localstatedir}/lib/vdsm/data-center && + mv %{_localstatedir}/lib/vdsm/data-center.new %{_localstatedir}/lib/vdsm/data-center && + rm -rf /rhev +fi + # set the vdsm "secret" password for libvirt %{_bindir}/vdsm-tool set-saslpasswd diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 054fadb..bb0f0ba 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -64,7 +64,7 @@ def getDomUuidFromMetafilePath(metafile): # Metafile path has pattern: - # /rhev/data-center/mnt/export-path/sdUUID/dom_md/metadata + # @vdsmlibdir@/data-center/mnt/export-path/sdUUID/dom_md/metadata metaList = os.path.normpath(metafile).split('/') return metaList[-3] diff --git a/vdsm/storage/fileVolume.py b/vdsm/storage/fileVolume.py index 0aa9043..08fc8b8 100644 --- a/vdsm/storage/fileVolume.py +++ b/vdsm/storage/fileVolume.py @@ -40,7 +40,7 @@ def getDomUuidFromVolumePath(volPath): # Volume path has pattern: - # /rhev/data-center/spUUID/sdUUID/images/imgUUID/volUUID + # @vdsmlibdir@/data-center/spUUID/sdUUID/images/imgUUID/volUUID # sdUUID position after data-center sdUUIDPos = 3 -- To view, visit http://gerrit.ovirt.org/8500 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4ed316fedbf2e4d7e5bf741b7c77a39704994ae7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Alon Bar-Lev <alonbl(a)redhat.com>

5 24

Change in vdsm[master]: build: use architecture independent location of site-package...
by Alon Bar-Lev 25 May '13

25 May '13

Alon Bar-Lev has uploaded a new change for review. Change subject: build: use architecture independent location of site-packages if possible ...................................................................... build: use architecture independent location of site-packages if possible Currently all vdsm python modules go into pyexecdir, which is the architecture specific location. Python architecture independent modules should go into pythondir. In x86_64 system locations are: pythondir = /usr/lib/python2.7/site-packages pyexecdir = /usr/lib64/python2.7/site-packages Both are in default search path of python: --- print(sys.path)" ['', '/root', '/usr/lib64/python27.zip', '/usr/lib64/python2.7', '/usr/lib64/python2.7/plat-linux2', '/usr/lib64/python2.7/lib-tk', '/usr/lib64/python2.7/lib-old', '/usr/lib64/python2.7/lib-dynload', '/usr/lib64/python2.7/site-packages', '/usr/lib64/python2.7/site-packages/gst-0.10', '/usr/lib64/python2.7/site-packages/gtk-2.0', '/usr/lib/python2.7/site-packages', '/usr/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info'] --- In vdsm project only the betterPopen package is architecture specific, it should go into pyexecdir while all other packages should go into pythondir. Change-Id: I975cab5c885697d3c43d083e243f8004587fd2bc Signed-off-by: Alon Bar-Lev <alonbl(a)redhat.com> --- M configure.ac M vdsm/betterPopen/Makefile.am 2 files changed, 3 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/98/6098/1 -- To view, visit http://gerrit.ovirt.org/6098 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I975cab5c885697d3c43d083e243f8004587fd2bc Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Alon Bar-Lev <alonbl(a)redhat.com>

1 11

Change in vdsm[master]: vdsmd: remove bashisms found with checkbashism script on ubu...
by Ryan Harper 17 May '13

17 May '13

Ryan Harper has uploaded a new change for review. Change subject: vdsmd: remove bashisms found with checkbashism script on ubuntu ...................................................................... vdsmd: remove bashisms found with checkbashism script on ubuntu Ubuntu uses dash for its /bin/sh and some of the shell scripting includes bashisms that come through even with bash --posix (aka /bin/sh -> /bin/bash. Most of the fixes fall into the following changes: - Using '=' instead of '==' for comparison - Using '[ ]' instead of '[[ ]]' for test - replacing {..} with seq The remaining bashisms are around emitting messages, updating these would require moving to using gettext's shell helper for producing intl. lang equivalents. To find files in the repository that need scanning, I looked for files with '/bin/sh' in the file, excluding non-shell script files. For each of the files that filtered through, I ran checkbashisms to see if the file needed a closer look. % find . -type f | egrep -v "(configure|build|aclocal|.py|Makefile|README|.git|autom4te.cache)" | xargs -i grep -l "/bin/sh" {} ./scripts/make_device_config.sh ./scripts/create_config ./scripts/feature_to_c.sh ./scripts/hxtool ./scripts/signrom.sh ./scripts/tracetool ./scripts/qemu-binfmt-conf.sh ./scripts/update-linux-headers.sh ./scripts/check-qerror.sh ./vdsm/sudoers.vdsm.in ./vdsm/vdsm-logrotate ./vdsm/vdsmd.init.in ./autogen.sh ./vdsm_cli/vdsClient.in ./tests/run_tests.sh.in ./tests/run_tests_local.sh.in ./vdsm_reg/vdsm-reg-logrotate (platechiller) vdsm % find . -type f | egrep -v "(configure|build|aclocal|.py|Makefile|README|.git|autom4te.cache)" | xargs -i grep -l "/bin/sh" {} | xargs -i bash -c 'P={}; checkbashisms $P &>/dev/null || echo $P' ./vdsm/vdsmd.init.in Only vdsmd.init.in needed fixing. Change-Id: Iece592e9cc3dfa361c99812461d783bff7f780a0 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/vdsmd.init.in 1 file changed, 11 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/8336/1 diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 0012157..2c0e894 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -112,13 +112,13 @@ local listen_tcp auth_tcp ssl ssl=`$GETCONFITEM $CONF_FILE vars ssl true | tr A-Z a-z` - [ "$ssl" == true ] && return 0 + [ "$ssl" = "true" ] && return 0 listen_tcp="`get_libvirt_conf_item $LCONF listen_tcp`" auth_tcp="`get_libvirt_conf_item $LCONF auth_tcp`" spice_tls="`get_libvirt_conf_item $QCONF spice_tls`" - if [ "$listen_tcp" == 1 -a "$auth_tcp" == '"none"' -a "$spice_tls" == 0 ]; + if [ "$listen_tcp" = "1" -a "$auth_tcp" = '"none"' -a "$spice_tls" = "0" ]; then return 0 else @@ -138,7 +138,7 @@ /sbin/chkconfig $srv off if /sbin/service $srv status > /dev/null 2>&1; then - if [ "$srv" == "libvirt-guests" ]; then + if [ "$srv" = "libvirt-guests" ]; then /bin/rm -f /var/lock/subsys/libvirt-guests else /sbin/service $srv stop @@ -149,7 +149,7 @@ } libvirt_should_use_upstart() { - [[ -x /sbin/initctl ]] + [ -x /sbin/initctl ] } start_needed_srv() { @@ -297,7 +297,7 @@ set_if_default $lconf host_uuid \"$(uuidgen)\" set_if_default $qconf dynamic_ownership 0 - if [[ "$ssl" == true ]]; then + if [ "$ssl" = "true" ]; then set_if_default $qconf spice_tls 1 else set_if_default $qconf spice_tls 0 @@ -314,7 +314,7 @@ if [ -f $ts/certs/cacert.pem -a \ -f $ts/certs/vdsmcert.pem -a \ -f $ts/keys/vdsmkey.pem -a \ - "$ssl" == true ]; + "$ssl" = "true" ]; then set_if_default $lconf ca_file \"$ts/certs/cacert.pem\" set_if_default $lconf cert_file \"$ts/certs/vdsmcert.pem\" @@ -389,7 +389,7 @@ stop_libvirtd_sysv() { # stop libvirt SysV service if we intend to configure upstart - if libvirt_should_use_upstart && ! [[ -f /etc/init/libvirtd.conf ]]; then + if libvirt_should_use_upstart && ! [ -f /etc/init/libvirtd.conf ]; then /sbin/chkconfig libvirtd off /sbin/service libvirtd stop fi @@ -400,7 +400,7 @@ # its socket is not yet ready. Once issue fixed on libvirt, # this workaround should be removed. - for i in {1..50} + for i in $(seq 1 50) do if /usr/bin/virsh -r version > /dev/null 2>&1; then return 0 @@ -426,14 +426,14 @@ /bin/grep libvirtd.upstart | /usr/bin/tail -1` target=/etc/init/libvirtd.conf - if [[ -f "$packaged" ]] && ! diff -q "$packaged" "$target" >/dev/null; + if [ -f "$packaged" ] && ! diff -q "$packaged" "$target" >/dev/null; then /bin/cp -p "$packaged" "$target" || return 1 /sbin/initctl reload-configuration fi startout=`/sbin/initctl start libvirtd 2>&1` - if [[ "$?" -eq 0 || "$startout" =~ .*already\ running.* ]]; + if [ "$?" -eq 0 || "$startout" =~ .*already\ running.* ]; then await_libvirt_start_workaround return 0 @@ -480,7 +480,7 @@ mk_dom_backup mk_run_path /bin/chmod 1777 /dev/shm - if [ $is_coredump == true ]; then + if [ $is_coredump = "true" ]; then export DAEMON_COREFILE_LIMIT=unlimited echo $CORE_DUMP_PATH > $CORE_PATTERN fi -- To view, visit http://gerrit.ovirt.org/8336 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iece592e9cc3dfa361c99812461d783bff7f780a0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

2 5

Change in vdsm[master]: SUDO change, move fuser call to super vdsm
by xiawenc＠linux.vnet.ibm.com 07 May '13

07 May '13

Wenchao Xia has uploaded a new change for review. Change subject: SUDO change, move fuser call to super vdsm ...................................................................... SUDO change, move fuser call to super vdsm supervdsm.fuser() will return out result from executing, and fuser.py will parse it. Change-Id: I0bb881e6bcad213db1d186e70f41b22d78ee3ecf Signed-off-by: wenchao xia <xiawenc(a)linux.vnet.ibm.com> --- M vdsm/storage/fuser.py M vdsm/supervdsmServer.py 2 files changed, 20 insertions(+), 13 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/14/5314/1 -- To view, visit http://gerrit.ovirt.org/5314 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0bb881e6bcad213db1d186e70f41b22d78ee3ecf Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Wenchao Xia <xiawenc(a)linux.vnet.ibm.com>

4 6

Change in vdsm[master]: Fix 2nd order exception msg in lvm._initpvs().
by ewarszaw＠redhat.com 23 Apr '13

23 Apr '13

Eduardo has uploaded a new change for review. Change subject: Fix 2nd order exception msg in lvm._initpvs(). ...................................................................... Fix 2nd order exception msg in lvm._initpvs(). Change-Id: I48766ec89b53230974048c9c8144926d1c24a82d Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/lvm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/5319/1 -- To view, visit http://gerrit.ovirt.org/5319 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I48766ec89b53230974048c9c8144926d1c24a82d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

5 5

Change in vdsm[master]: CannotRemoveLogicalVolume is raised with arguments.
by ewarszaw＠redhat.com 31 Mar '13

31 Mar '13

Eduardo has uploaded a new change for review. Change subject: CannotRemoveLogicalVolume is raised with arguments. ...................................................................... CannotRemoveLogicalVolume is raised with arguments. Change-Id: Ied29c1dddf4e9d6cf890cb66d362acf9eed99ba6 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/lvm.py M vdsm/storage/storage_exception.py 2 files changed, 7 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/08/8808/1 diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index 78c846a..88d818b 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -1015,7 +1015,7 @@ if _isLVActive(vgName, lvName): #Fix me #Should not remove active LVs - #raise se.CannotRemoveLogicalVolume(vgName, lvName) + #raise se.CannotRemoveLogicalVolume(vgName, (lvName,)) log.warning("Removing active volume %s/%s" % (vgName, lvName)) #LV exists or not in cache, attempting to remove it. @@ -1036,7 +1036,7 @@ else: # Otherwise LV info needs to be refreshed _lvminfo._invalidatelvs(vgName, lvName) - raise se.CannotRemoveLogicalVolume(vgName, str(lvNames)) + raise se.CannotRemoveLogicalVolume(vgName, lvNames) def extendLV(vgName, lvName, size): diff --git a/vdsm/storage/storage_exception.py b/vdsm/storage/storage_exception.py index 54e64e6..b5f63da 100644 --- a/vdsm/storage/storage_exception.py +++ b/vdsm/storage/storage_exception.py @@ -1305,8 +1305,12 @@ class CannotRemoveLogicalVolume(StorageException): + def __init__(self, vgName, lvNames): + self.value("vg: %s lvs: %s", vgName, ", ".join(lvNames)) + self.vg = vgName + self.lvs = lvNames code = 551 - message = "Cannot remove Logical Volume" + message = "Failed to remove some LVs." class CannotDeactivateLogicalVolume(StorageException): -- To view, visit http://gerrit.ovirt.org/8808 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ied29c1dddf4e9d6cf890cb66d362acf9eed99ba6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

3 5

Change in vdsm[master]: 2 of 2 Add network logs for VDSM network commands and file o...
by asegurap＠redhat.com 24 Mar '13

24 Mar '13

Antoni Segura Puimedon has uploaded a new change for review. Change subject: 2 of 2 Add network logs for VDSM network commands and file operations. ...................................................................... 2 of 2 Add network logs for VDSM network commands and file operations. Second patch in the series that creates a module-specific logger and uses it with the utility makeExecCmd to increase the detail level of the debugging information provided by the two modules when calling processes and modifying files. Bug-Id: https://bugzilla.redhat.com/851839 Change-Id: Iea3738f5ce66d2537c078c452def30fb3feb5390 Signed-off-by: Antoni S. Puimedon <asegurap(a)redhat.com> --- M vdsm/configNetwork.py M vdsm/tc.py 2 files changed, 38 insertions(+), 30 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/30/7730/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 74d19c7..ceb846d 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -17,7 +17,7 @@ # Refer to the README and COPYING files for full details of the license # -import sys, subprocess, os, re, traceback +import sys, os, re, traceback import pipes import pwd import time @@ -32,6 +32,7 @@ from vdsm import constants from vdsm import utils +from storage.misc import enableLogSkip import neterrors as ne from vdsm import define from vdsm import netinfo @@ -42,6 +43,11 @@ MAX_BRIDGE_NAME_LEN = 15 ILLEGAL_BRIDGE_CHARS = frozenset(':. \t') +configNetworkLogger = enableLogSkip( + logging.getLogger('configNetwork'), ignoreSourceFiles=[__file__], + logSkipName='configNetwork') +execCmd = utils.makeExecCmd(log=configNetworkLogger) + class ConfigNetworkError(Exception): def __init__(self, errCode, message): self.errCode = errCode @@ -50,25 +56,22 @@ def ifdown(iface): "Bring down an interface" - p = subprocess.Popen([constants.EXT_IFDOWN, iface], stdout=subprocess.PIPE, - stderr=subprocess.PIPE, close_fds=True) - out, err = p.communicate() + rc, out, err = execCmd([constants.EXT_IFDOWN, iface], raw=True) if out.strip(): logging.info(out) if err.strip(): logging.warn('\n'.join([line for line in err.splitlines() if not line.endswith(' does not exist!')])) - return p.returncode + return rc def ifup(iface): "Bring up an interface" - p = subprocess.Popen([constants.EXT_IFUP, iface], stdout=subprocess.PIPE, - stderr=subprocess.PIPE, close_fds=True) - out, err = p.communicate() + rc, out, err = execCmd([constants.EXT_IFUP, iface], raw=True) if out.strip(): logging.info(out) if err.strip(): logging.warn(err) + return rc def ifaceUsers(iface): "Returns a list of entities using the interface" @@ -157,8 +160,9 @@ mounts = open('/proc/mounts').read() if ' /config ext3' in mounts and ' %s ext3' % filename in mounts: - subprocess.call([constants.EXT_UMOUNT, '-n', filename]) + execCmd([constants.EXT_UMOUNT, '-n', filename]) utils.rmFile(filename) + configNetworkLogger.debug("Removed file %s", filename) def _createNetwork(self, netXml): conn = libvirtconnection.get() @@ -291,6 +295,8 @@ for confFile, content in self._backups.iteritems(): if content is None: utils.rmFile(confFile) + configNetworkLogger.debug( + 'Removing empty configuration backup %s', confFile) else: open(confFile, 'w').write(content) logging.info('Restored %s', confFile) @@ -299,9 +305,9 @@ def _persistentBackup(cls, filename): """ Persistently backup ifcfg-* config files """ if os.path.exists('/usr/libexec/ovirt-functions'): - subprocess.call([constants.EXT_SH, '/usr/libexec/ovirt-functions', + execCmd([constants.EXT_SH, '/usr/libexec/ovirt-functions', 'unmount_config', filename]) - logging.debug("unmounted %s using ovirt", filename) + configNetworkLogger.debug("unmounted %s using ovirt", filename) (dummy, basename) = os.path.split(filename) if os.path.exists(filename): @@ -309,7 +315,7 @@ else: # For non-exists ifcfg-* file use predefined header content = cls.DELETED_HEADER + '\n' - logging.debug("backing up %s: %s", basename, content) + configNetworkLogger.debug("backing up %s: %s", basename, content) cls.writeBackupFile(netinfo.NET_CONF_BACK_DIR, basename, content) @@ -349,16 +355,12 @@ if not self._backups and not self._networksBackups: return - subprocess.Popen(['/etc/init.d/network', 'stop'], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE).communicate() + execCmd(['/etc/init.d/network', 'stop']) self.restoreAtomicNetworkBackup() self.restoreAtomicBackup() - subprocess.Popen(['/etc/init.d/network', 'start'], - stdout=subprocess.PIPE, - stderr=subprocess.PIPE).communicate() + execCmd(['/etc/init.d/network', 'start']) @classmethod def clearBackups(cls): @@ -374,8 +376,8 @@ try: selinux.restorecon(fileName) except: - logging.debug('ignoring restorecon error in case SElinux is ' - 'disabled', exc_info=True) + configNetworkLogger.debug('ignoring restorecon error in case ' + 'SElinux is disabled', exc_info=True) def _createConfFile(self, conf, name, ipaddr=None, netmask=None, gateway=None, bootproto=None, mtu=None, onboot='yes', **kwargs): @@ -405,7 +407,7 @@ if re.match('^[a-zA-Z_]\w*$', k): cfg += '%s=%s\n' % (k.upper(), pipes.quote(kwargs[k])) else: - logging.debug('ignoring variable %s', k) + configNetworkLogger.debug('ignoring variable %s', k) self.writeConfFile(self.NET_CONF_PREF + name, cfg) @@ -499,8 +501,7 @@ def removeVlan(self, vlan, iface): vlandev = iface + '.' + vlan ifdown(vlandev) - subprocess.call([constants.EXT_IPROUTE, 'link', 'del', vlandev], - stderr=subprocess.PIPE) + execCmd([constants.EXT_IPROUTE, 'link', 'del', vlandev]) self._backup(self.NET_CONF_PREF + iface + '.' + vlan) self._removeFile(self.NET_CONF_PREF + iface + '.' + vlan) @@ -510,7 +511,7 @@ def removeBridge(self, bridge): ifdown(bridge) - subprocess.call([constants.EXT_BRCTL, 'delbr', bridge]) + execCmd([constants.EXT_BRCTL, 'delbr', bridge]) self._backup(self.NET_CONF_PREF + bridge) self._removeFile(self.NET_CONF_PREF + bridge) @@ -829,7 +830,7 @@ # Validation if not utils.tobool(force): - logging.debug('validating network...') + configNetworkLogger.debug('validating network...') _addNetworkValidation(_netinfo, network=network, vlan=vlan, bonding=bonding, nics=nics, ipaddr=ipaddr, netmask=netmask, gateway=gateway, bondingOptions=bondingOptions, @@ -1125,8 +1126,8 @@ _netinfo = netinfo.NetInfo() for bond, bondAttrs in bondings.iteritems(): - logger.debug("Creating/Editing bond %s with attributes %s", - bond, bondAttrs) + logger.debug("Creating/Editing bond %s with attributes %s", bond, + bondAttrs) brNets = list(_netinfo.getBridgedNetworksForIface(bond)) # Only one bridged-non-VLANed network allowed on same nic/bond @@ -1315,7 +1316,7 @@ def setSafeNetworkConfig(): """Declare current network configuration as 'safe'""" - subprocess.Popen([constants.EXT_VDSM_STORE_NET_CONFIG]) + execCmd([constants.EXT_VDSM_STORE_NET_CONFIG]) def usage(): print """Usage: diff --git a/vdsm/tc.py b/vdsm/tc.py index a0a612f..44c5bb1 100644 --- a/vdsm/tc.py +++ b/vdsm/tc.py @@ -18,16 +18,23 @@ # Refer to the README and COPYING files for full details of the license # +import logging from collections import namedtuple import ethtool -import storage.misc +import utils +from storage.misc import enableLogSkip from vdsm.constants import EXT_TC, EXT_IFCONFIG ERR_DEV_NOEXIST = 2 QDISC_INGRESS = 'ffff:' + +tcLogger = enableLogSkip( + logging.getLogger('trafficControl'), ignoreSourceFiles=[__file__], + logSkipName='trafficControl') +execCmd = utils.makeExecCmd(log=tcLogger) class TrafficControlException(Exception): @@ -95,7 +102,7 @@ def _process_request(command): - retcode, out, err = storage.misc.execCmd(command, raw=True, sudo=False) + retcode, out, err = execCmd(command, raw=True, sudo=False) if retcode != 0: raise TrafficControlException(retcode, err, command) return out -- To view, visit http://gerrit.ovirt.org/7730 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iea3738f5ce66d2537c078c452def30fb3feb5390 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com>

3 5

Change in vdsm[master]: move ifaceUsers, bondingOtherUsers and nicOtherUsers to neti...
by wudxw＠linux.vnet.ibm.com 20 Mar '13

20 Mar '13

Mark Wu has uploaded a new change for review. Change subject: move ifaceUsers, bondingOtherUsers and nicOtherUsers to netinfo ...................................................................... move ifaceUsers, bondingOtherUsers and nicOtherUsers to netinfo Change-Id: Ib4426551dd5bc6b4889c5cae40e7145df51b52db Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/nativeNetConfig.py M vdsm/netinfo.py 2 files changed, 51 insertions(+), 53 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/18/9318/1 diff --git a/vdsm/nativeNetConfig.py b/vdsm/nativeNetConfig.py index a0ad7a3..aca4a33 100644 --- a/vdsm/nativeNetConfig.py +++ b/vdsm/nativeNetConfig.py @@ -62,55 +62,6 @@ return rc -def ifaceUsers(iface): - "Returns a list of entities using the interface" - _netinfo = netinfo.NetInfo() - users = set() - for n, ndict in _netinfo.networks.iteritems(): - if ndict['bridged'] and iface in ndict['ports']: - users.add(n) - elif not ndict['bridged'] and iface == ndict['iface']: - users.add(n) - for b, bdict in _netinfo.bondings.iteritems(): - if iface in bdict['slaves']: - users.add(b) - for v, vdict in _netinfo.vlans.iteritems(): - if iface == vdict['iface']: - users.add(v) - return users - - -def nicOtherUsers(bridge, vlan, bonding, nic): - """ - Returns a list of interfaces using a nic, - other than the specified one (used for validation) - """ - if bonding: - owner = bonding - elif vlan: - owner = nic + '.' + vlan - else: - owner = bridge - users = ifaceUsers(nic) - if bonding: - users.update(bondingOtherUsers(bridge, vlan, bonding)) - users.discard(owner) - return users - - -def bondingOtherUsers(bridge, vlan, bonding): - """ - Return a list of nics/interfaces using a bonding, - other than the specified one (used for validation) - """ - if vlan: - owner = bonding + '.' + vlan - else: - owner = bridge - users = ifaceUsers(bonding) - users.discard(owner) - return users - # This function must respect the order used in: # # /etc/rc.d/init.d/network @@ -446,7 +397,7 @@ if bridge: conf += 'BRIDGE=%s\n' % pipes.quote(bridge) - if ifaceUsers(bonding): + if netinfo.NetInfo().ifaceUsers(bonding): confParams = netinfo.getIfaceCfg(bonding) if not ipaddr: ipaddr = confParams.get('IPADDR', None) @@ -479,7 +430,7 @@ if bonding: conf += 'MASTER=%s\nSLAVE=yes\n' % pipes.quote(bonding) - if ifaceUsers(nic): + if _netinfo.ifaceUsers(nic): confParams = netinfo.getIfaceCfg(nic) if not ipaddr: ipaddr = confParams.get('IPADDR', None) @@ -1029,13 +980,14 @@ # The (relatively) new setupNetwork verb allows to remove a network # defined on top of an bonding device without break the bond itself. if implicitBonding: - if bonding and not bondingOtherUsers(network, vlan, bonding): + if bonding and not _netinfo.bondingOtherUsers(network, + vlan, bonding): ifdown(bonding) configWriter.removeBonding(bonding) iface = None if bonding == iface else iface for nic in nics: - if not nicOtherUsers(network, vlan, bonding, nic): + if not _netinfo.nicOtherUsers(network, vlan, bonding, nic): ifdown(nic) configWriter.removeNic(nic) ifup(nic) diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index b6d1a1f..83e23ac 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -502,3 +502,49 @@ """ return [netname for (netname, net) in networks().iteritems() if not 'bridge' in net] + + def ifaceUsers(self, iface): + "Returns a list of entities using the interface" + users = set() + for n, ndict in self.networks.iteritems(): + if ndict['bridged'] and iface in ndict['ports']: + users.add(n) + elif not ndict['bridged'] and iface == ndict['iface']: + users.add(n) + for b, bdict in self.bondings.iteritems(): + if iface in bdict['slaves']: + users.add(b) + for v, vdict in self.vlans.iteritems(): + if iface == vdict['iface']: + users.add(v) + return users + + def bondingOtherUsers(self, bridge, vlan, bonding): + """ + Return a list of nics/interfaces using a bonding, + other than the specified one (used for validation) + """ + if vlan: + owner = bonding + '.' + vlan + else: + owner = bridge + users = self.ifaceUsers(bonding) + users.discard(owner) + return users + + def nicOtherUsers(self, bridge, vlan, bonding, nic): + """ + Returns a list of interfaces using a nic, + other than the specified one (used for validation) + """ + if bonding: + owner = bonding + elif vlan: + owner = nic + '.' + vlan + else: + owner = bridge + users = self.ifaceUsers(nic) + if bonding: + users.update(self.bondingOtherUsers(bridge, vlan, bonding)) + users.discard(owner) + return users -- To view, visit http://gerrit.ovirt.org/9318 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib4426551dd5bc6b4889c5cae40e7145df51b52db Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

4 7

Change in vdsm[master]: Delete useless withRelaunch argument of onReboot in vm.py
by Jarod.w 14 Mar '13

14 Mar '13

Jarod.w has uploaded a new change for review. Change subject: Delete useless withRelaunch argument of onReboot in vm.py ...................................................................... Delete useless withRelaunch argument of onReboot in vm.py If we set the withRelaunch argument to True, it can't work normally because function relaunch() on clientIF class isn't defined.Here, we delete the useless argument directly to avoid potential error. Change-Id: I53b372feca865327e9e81f1b89e6d7566038c031 Signed-off-by: jarod.w <work.iec23801(a)gmail.com> --- M vdsm/libvirtconnection.py M vdsm/vm.py 2 files changed, 2 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/96/9096/1 diff --git a/vdsm/libvirtconnection.py b/vdsm/libvirtconnection.py index 824923d..d0f9452 100644 --- a/vdsm/libvirtconnection.py +++ b/vdsm/libvirtconnection.py @@ -45,7 +45,7 @@ event, detail = args[:-1] v._onLibvirtLifecycleEvent(event, detail, None) elif eventid == libvirt.VIR_DOMAIN_EVENT_ID_REBOOT: - v.onReboot(False) + v.onReboot() elif eventid == libvirt.VIR_DOMAIN_EVENT_ID_RTC_CHANGE: utcoffset, = args[:-1] v._rtcUpdate(utcoffset) diff --git a/vdsm/vm.py b/vdsm/vm.py index 2229882..48b1164 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -762,7 +762,7 @@ os.rename(f.name, self._recoveryFile) - def onReboot(self, withRelaunch): + def onReboot(self): try: self.log.debug('reboot event') self._startTime = time.time() @@ -773,8 +773,6 @@ if self.conf.get('volatileFloppy'): self._ejectFloppy() self.log.debug('ejected volatileFloppy') - if withRelaunch: - self.cif.relaunch(self.status()) except: self.log.error("Reboot event failed", exc_info=True) -- To view, visit http://gerrit.ovirt.org/9096 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I53b372feca865327e9e81f1b89e6d7566038c031 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Jarod.w <work.iec23801(a)gmail.com>

3 4

Change in vdsm[master]: vdsm_hooks: Add support for checkimages VDSM hook
by Jan Lieskovsky 14 Mar '13

14 Mar '13

Jan Lieskovsky has uploaded a new change for review. Change subject: vdsm_hooks: Add support for checkimages VDSM hook ...................................................................... vdsm_hooks: Add support for checkimages VDSM hook Introduce the checkimages VDSM hook - perform consistency check on a qcow2 format disk image using the QEMU disk image utility. VDSM hook alternative to the validateImage SPM command. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=611183 Change-Id: Ib4d88fb46a725d2c525d10ce2fcc808bba5b0ab4 Signed-off-by: Jan Lieskovsky <jlieskov(a)redhat.com> --- M configure.ac M vdsm.spec.in M vdsm_hooks/Makefile.am A vdsm_hooks/checkimages/Makefile.am A vdsm_hooks/checkimages/README A vdsm_hooks/checkimages/before_vm_start.py 6 files changed, 155 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/54/9154/1 diff --git a/configure.ac b/configure.ac index 7cb12ef..1498e3b 100644 --- a/configure.ac +++ b/configure.ac @@ -192,6 +192,7 @@ tests/functional/Makefile vds_bootstrap/Makefile vdsm_cli/Makefile + vdsm_hooks/checkimages/Makefile vdsm_hooks/directlun/Makefile vdsm_hooks/faqemu/Makefile vdsm_hooks/fileinject/Makefile diff --git a/vdsm.spec.in b/vdsm.spec.in index c117ee5..3bd0865 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -200,6 +200,15 @@ %description tests A test suite for verifying the functionality of a running vdsm instance +%package hook-checkimages +Summary: Qcow2 disk image format check hook for VDSM +BuildArch: noarch +Requires: vdsm + +%description hook-checkimages +VDSM hook used to perform consistency check on a qcow2 format disk image +using the QEMU disk image utility. + %package hook-vhostmd Summary: VDSM hook set for interaction with vhostmd Requires: vhostmd @@ -802,6 +811,10 @@ %{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/50_qemucmdline %if 0%{?with_hooks} +%files hook-checkimages +%defattr(-, root, root, -) +%{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/60_checkimages + %files hook-directlun %defattr(-, root, root, -) %{_sysconfdir}/sudoers.d/50_vdsm_hook_directlun diff --git a/vdsm_hooks/Makefile.am b/vdsm_hooks/Makefile.am index e6a8280..e4d4986 100644 --- a/vdsm_hooks/Makefile.am +++ b/vdsm_hooks/Makefile.am @@ -25,6 +25,7 @@ # Additional hooks if HOOKS SUBDIRS += \ + checkimages \ directlun \ fileinject \ floppy \ diff --git a/vdsm_hooks/checkimages/Makefile.am b/vdsm_hooks/checkimages/Makefile.am new file mode 100644 index 0000000..ad28f25 --- /dev/null +++ b/vdsm_hooks/checkimages/Makefile.am @@ -0,0 +1,31 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +# + +EXTRA_DIST = \ + before_vm_start.py + +install-data-local: + $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_start + $(INSTALL_SCRIPT) $(srcdir)/before_vm_start.py \ + $(DESTDIR)$(vdsmhooksdir)/before_vm_start/60_checkimages + +uninstall-local: + $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_start/60_checkimages diff --git a/vdsm_hooks/checkimages/README b/vdsm_hooks/checkimages/README new file mode 100644 index 0000000..8959955 --- /dev/null +++ b/vdsm_hooks/checkimages/README @@ -0,0 +1,10 @@ +checkimages hook: +================ + +VDSM hook used to perform consistency check on a qcow2 format disk image +using the QEMU disk image utility. + +example: + checkimages=true,timeout:1.12 + +Note: Timeout value is taken in seconds. Check of 1GB image takes ~0.02 s. diff --git a/vdsm_hooks/checkimages/before_vm_start.py b/vdsm_hooks/checkimages/before_vm_start.py new file mode 100755 index 0000000..a80a042 --- /dev/null +++ b/vdsm_hooks/checkimages/before_vm_start.py @@ -0,0 +1,99 @@ +#!/usr/bin/python + +import os +import sys +import traceback + +import hooking + +''' +checkimages vdsm hook +===================== +Hook performs consistency check on all qcow2 format disk images of a +particular VM using the QEMU disk image utility. + +syntax: + checkimages=^true,timeout:(none|\d+\.{1}\d+)$ + +example: + checkimages=true,timeout:1.12 + +Note: Timeout value is taken in seconds. Check of 1GB image takes ~0.02 s. + +''' + +def checkImage(path, timeout): + ''' + Check qcow2 image using qemu-img QEMU utility + ''' + + cmd = ['/usr/bin/qemu-img', 'check', '-f', 'qcow2', path] + + # No timeout was provided. Run the check without limitation + # till the qemu-img command finished + if timeout is None: + rc, out, err = hooking.execCmd(cmd, sudo=False, raw=True) + + # Timeout was provided. Enforce check termination on timeout + # expiration + else: + p = hooking.execCmd(cmd, sudo=False, raw=True, sync=False) + + if not p.wait(timeout): + p.kill() + sys.stderr.write('checkimages: Image check operation timed out.\n') + sys.exit(2) + + out, err = p.communicate() + rc = p.returncode + + if rc == 0: + sys.stderr.write('checkimages: Checking image %s returned: %s\n' % \ + (path, out)) + else: + sys.stderr.write('checkimages: Error running %s command: %s\n' % + (' '.join(cmd), err)) + sys.exit(2) + + +if os.environ.has_key('checkimages'): + + timeout = None + + try: + checkimages = os.environ['checkimages'] + # Might not be necessary. But for any case check we succeeded: + # 1) Obtaining provided timeout value + # 2) Converting provided value to float + try: + req_timeout = checkimages.split(',')[1].split(':')[1] + if req_timeout != 'none': + timeout = float(req_timeout) + except: + sys.stderr.write('checkimages: Error obtaining timeout value. ' \ + 'Continuing with no timeout.\n') + + domxml = hooking.read_domxml() + disks = domxml.getElementsByTagName('disk') + + for disk in disks: + sources = disk.getElementsByTagName('source') + drivers = disk.getElementsByTagName('driver') + + if (len(sources) > 0 and len(drivers) > 0): + disk_path = sources[0].getAttribute('file') + disk_type = drivers[0].getAttribute('type') + + if ((disk_path and disk_type) and (disk_type == 'qcow2')): + sys.stderr.write('checkimages: Checking disk = %s\n' % + disk_path) + checkImage(disk_path, timeout) + elif disk_type != 'qcow2': + sys.stderr.write('checkimages: Disk = %s is not qcow2 ' \ + 'disk image format one. Ignoring.\n' % + disk_path) + + except: + sys.stderr.write('checkimages [unexpected error]: %s\n' % + traceback.format_exc()) + sys.exit(2) -- To view, visit http://gerrit.ovirt.org/9154 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib4d88fb46a725d2c525d10ce2fcc808bba5b0ab4 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Jan Lieskovsky <jlieskov(a)redhat.com>

4 17

Change in vdsm[master]: Fix mbx Exception type.
by ewarszaw＠redhat.com 13 Mar '13

13 Mar '13

Eduardo has uploaded a new change for review. Change subject: Fix mbx Exception type. ...................................................................... Fix mbx Exception type. Change-Id: Id8baf1121aaf351b029945972d5e87dbfa6a0bd5 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/storage_mailbox.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/6617/1 -- To view, visit http://gerrit.ovirt.org/6617 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id8baf1121aaf351b029945972d5e87dbfa6a0bd5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

5 36

Change in vdsm[master]: Removed glusterfs packages from bootstrap
by barumuga＠redhat.com 12 Mar '13

12 Mar '13

Hello Ayal Baron, Timothy Asir, Saggi Mizrahi, Federico Simoncelli, Dan Kenigsberg, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9076 to review the following change. Change subject: Removed glusterfs packages from bootstrap ...................................................................... Removed glusterfs packages from bootstrap As glusterfs packages are dependencies for vdsm-gluster package, having these in bootstrap is redundant Change-Id: I5bdde338599155df43af3a0fd0d14a02e3bddda8 Signed-off-by: Bala.FA <barumuga(a)redhat.com> --- M vds_bootstrap/vds_bootstrap.py M vdsm.spec.in 2 files changed, 2 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/76/9076/1 diff --git a/vds_bootstrap/vds_bootstrap.py b/vds_bootstrap/vds_bootstrap.py index 867d6f4..95f255d 100755 --- a/vds_bootstrap/vds_bootstrap.py +++ b/vds_bootstrap/vds_bootstrap.py @@ -159,8 +159,7 @@ 'seabios', 'qemu-img', 'fence-agents', 'libselinux-python', 'sanlock', 'sanlock-python') # Gluster packages - GLUSTER_PACK = ('vdsm-gluster', 'glusterfs-server', 'glusterfs-rdma', - 'glusterfs-geo-replication') + GLUSTER_PACK = (VDSM_NAME+'-gluster') else: # Devel packages DEVEL_PACK = ('gdb','tcpdump','strace','ltrace','sysstat','ntp', diff --git a/vdsm.spec.in b/vdsm.spec.in index c117ee5..8070cbe 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -356,7 +356,7 @@ BuildArch: noarch Requires: %{name} = %{version}-%{release} -Requires: glusterfs glusterfs-server glusterfs-fuse +Requires: glusterfs, glusterfs-server, glusterfs-fuse, glusterfs-rdma %description gluster Gluster plugin enables VDSM to serve Gluster functionalities. -- To view, visit http://gerrit.ovirt.org/9076 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5bdde338599155df43af3a0fd0d14a02e3bddda8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Bala.FA <barumuga(a)redhat.com> Gerrit-Reviewer: Ayal Baron <abaron(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Federico Simoncelli <fsimonce(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Timothy Asir <tjeyasin(a)redhat.com>

4 16

Change in vdsm[master]: Improve the validation of netmask and gateway
by wudxw＠linux.vnet.ibm.com 09 Mar '13

09 Mar '13

Mark Wu has uploaded a new change for review. Change subject: Improve the validation of netmask and gateway ...................................................................... Improve the validation of netmask and gateway Change-Id: I3c0a39f3f681a9e587f9118246e42a29e18d604b Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M tests/configNetworkTests.py M vdsm/configNetwork.py 2 files changed, 34 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/7713/1 diff --git a/tests/configNetworkTests.py b/tests/configNetworkTests.py index cc88184..58d1ebe 100644 --- a/tests/configNetworkTests.py +++ b/tests/configNetworkTests.py @@ -126,10 +126,10 @@ self.assertEqual(configNetwork.validateIpAddress(address), None) def testIsNetmaskValid(self): - masks = ('10.18.1.254', '10.50.25.177', '250.0.0.1', - '20.20.25.25') - badMasks = ('192.168.1.256', '10.50.25.1777', '256.0.0.1', - '20.20.25.25.25') + masks = ('254.0.0.0', '255.255.255.0', '255.255.255.128', + '255.255.255.224') + badMasks = ('192.168.1.0', '10.50.25.17', '255.0.255.0', + '253.0.0.0') for mask in badMasks: with self.assertRaises(configNetwork.ConfigNetworkError) \ @@ -141,6 +141,22 @@ for mask in masks: self.assertEqual(configNetwork.validateNetmask(mask), None) + def testIsGatewayValid(self): + gateways = (('192.168.122.1', '192.168.122.234', '255.255.255.0'), + ('192.168.122.4', '192.168.122.29', '255.255.255.224')) + badGateways = (('192.168.122.1', '192.168.123.4', '255.255.255.0'), + ('192.168.122.1', '192.168.122.39', '255.255.255.224')) + + for gateway in badGateways: + with self.assertRaises(configNetwork.ConfigNetworkError) \ + as cneContext: + configNetwork.validateGateway(*gateway) + self.assertEqual(cneContext.exception.errCode, + configNetwork.ne.ERR_BAD_ADDR) + + for gateway in gateways: + self.assertEqual(configNetwork.validateGateway(*gateway), None) + @memoized def _bondingModuleOptions(self): p = subprocess.Popen(['/sbin/modinfo', 'bonding'], diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index b8fbd9f..cfa8c40 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -656,6 +656,9 @@ return False return True +def _dottedQuadToNum(ipaddr): + return int(''.join('%02x' % int(i) for i in ipaddr.split('.')), 16) + def validateIpAddress(ipAddr): if not _validateIpAddress(ipAddr): raise ConfigNetworkError(ne.ERR_BAD_ADDR, "Bad IP address: %r"%ipAddr) @@ -663,10 +666,19 @@ def validateNetmask(netmask): if not _validateIpAddress(netmask): raise ConfigNetworkError(ne.ERR_BAD_ADDR, "Bad netmask: %r"%netmask) + num = _dottedQuadToNum(netmask) + if (num & (num - 1)) != (num << 1) & 0xffffffff: + raise ConfigNetworkError(ne.ERR_BAD_ADDR, "Bad netmask: %r"%netmask) -def validateGateway(gateway): +def validateGateway(gateway, ipaddr, netmask): if not _validateIpAddress(gateway): raise ConfigNetworkError(ne.ERR_BAD_ADDR, "Bad gateway: %r"%gateway) + else: + (gw, ip, mask) = [_dottedQuadToNum(addr) for addr + in (gateway, ipaddr, netmask)] + if ip & mask != gw & mask: + raise ConfigNetworkError(ne.ERR_BAD_ADDR, + "Bad gateway: %r"%gateway) def validateBondingName(bonding): if not re.match('^bond[0-9]+$', bonding): @@ -763,7 +775,7 @@ validateIpAddress(ipaddr) validateNetmask(netmask) if gateway: - validateGateway(gateway) + validateGateway(gateway, ipaddr, netmask) else: if netmask or gateway: raise ConfigNetworkError(ne.ERR_BAD_ADDR, -- To view, visit http://gerrit.ovirt.org/7713 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3c0a39f3f681a9e587f9118246e42a29e18d604b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

5 34

Change in vdsm[master]: make compression type configurable, support gzip and xz
by shaohef＠linux.vnet.ibm.com 07 Mar '13

07 Mar '13

ShaoHe Feng has uploaded a new change for review. Change subject: make compression type configurable, support gzip and xz ...................................................................... make compression type configurable, support gzip and xz autoconf check for presense of xz, and if not found fallback on gzip Change-Id: I4d88e9d40aefc87564c9c7e23f0d28aaad867aca Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M Makefile.am M autobuild.sh M configure.ac M vdsm.spec.in 4 files changed, 32 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/74/4174/1 -- To view, visit http://gerrit.ovirt.org/4174 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4d88e9d40aefc87564c9c7e23f0d28aaad867aca Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

9 29

Change in vdsm[master]: [WIP] Add copyVolumes command to the SPM
by Federico Simoncelli 01 Mar '13

01 Mar '13

Federico Simoncelli has uploaded a new change for review. Change subject: [WIP] Add copyVolumes command to the SPM ...................................................................... [WIP] Add copyVolumes command to the SPM Early implementation (the API might change). Change-Id: Ia19dd9b779a06ff88efed2eef52c1d152cdc797c --- M vdsm/API.py M vdsm/BindingXMLRPC.py M vdsm/storage/hsm.py M vdsm/storage/image.py M vdsm/storage/sp.py 5 files changed, 92 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/85/2885/1 -- To view, visit http://gerrit.ovirt.org/2885 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia19dd9b779a06ff88efed2eef52c1d152cdc797c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

1 1

Change in vdsm[master]: Don't rotate core dump logs on Fedora
by Federico Simoncelli 01 Mar '13

01 Mar '13

Federico Simoncelli has uploaded a new change for review. Change subject: Don't rotate core dump logs on Fedora ...................................................................... Don't rotate core dump logs on Fedora Core dumps are a system-level configuration and VDSM shouldn't try to override what the sysadmin decides. Change-Id: I0a2e86614475fd13de3888b956b5512a24016281 --- M configure.ac M vdsm.spec.in M vdsm/Makefile.am A vdsm/cdumps-logrotate.conf A vdsm/vdsm-logrotate.conf D vdsm/vdsm-logrotate.conf.in 6 files changed, 27 insertions(+), 22 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/91/991/1 -- To view, visit http://gerrit.ovirt.org/991 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0a2e86614475fd13de3888b956b5512a24016281 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

2 3

Change in vdsm[master]: DomainMonitor should use use real domains (no proxy)
by Federico Simoncelli 01 Mar '13

01 Mar '13

Federico Simoncelli has uploaded a new change for review. Change subject: DomainMonitor should use use real domains (no proxy) ...................................................................... DomainMonitor should use use real domains (no proxy) In this patch: * produce the domain when refreshing * remove the remaining getRealDomain (DomainProxy) calls * fix manuallyRemoveDomain for non-existent domains Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> Change-Id: Ibbf67fc050658e3418aa666e8fcef1e1244571e9 --- M vdsm/storage/domainMonitor.py M vdsm/storage/sdc.py M vdsm/storage/sp.py 3 files changed, 32 insertions(+), 26 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/94/7294/1 diff --git a/vdsm/storage/domainMonitor.py b/vdsm/storage/domainMonitor.py index 80b6ed3..0272cb1 100644 --- a/vdsm/storage/domainMonitor.py +++ b/vdsm/storage/domainMonitor.py @@ -75,13 +75,14 @@ def monitoredDomains(self): return self._domains.keys() - def startMonitoring(self, domain, hostId): - if domain.sdUUID in self._domains: + def startMonitoring(self, sdUUID, hostId): + if sdUUID in self._domains: return - self._domains[domain.sdUUID] = DomainMonitorThread( - domain, hostId, self._interval) - self._domains[domain.sdUUID].start() + domainThread = DomainMonitorThread(sdUUID, hostId, self._interval) + domainThread.start() + # The domain should be added only after it succesfully started + self._domains[sdUUID] = domainThread def stopMonitoring(self, sdUUID): # The domain monitor issues events that might become raceful if @@ -107,22 +108,25 @@ class DomainMonitorThread(object): log = logging.getLogger('Storage.DomainMonitorThread') - def __init__(self, domain, hostId, interval): + def __init__(self, sdUUID, hostId, interval): self.thread = Thread(target=self._monitorLoop) self.thread.setDaemon(True) self.stopEvent = Event() - self.domain = domain + self.domain = None + self.sdUUID = sdUUID self.hostId = hostId self.interval = interval self.status = DomainMonitorStatus() self.nextStatus = DomainMonitorStatus() - self.isIsoDomain = domain.isISO() + self.isIsoDomain = None self.lastRefresh = time() self.refreshTime = \ config.getint("irs", "repo_stats_cache_refresh_timeout") def start(self): + self.domain = sdCache.produce(self.sdUUID) + self.isIsoDomain = self.domain.isISO() self.thread.start() def stop(self, wait=True): @@ -134,17 +138,17 @@ return self.status.copy() def _monitorLoop(self): - self.log.debug("Starting domain monitor for %s", self.domain.sdUUID) + self.log.debug("Starting domain monitor for %s", self.sdUUID) while not self.stopEvent.is_set(): try: self._monitorDomain() except: self.log.error("The domain monitor for %s failed unexpectedly", - self.domain.sdUUID, exc_info=True) + self.sdUUID, exc_info=True) self.stopEvent.wait(self.interval) - self.log.debug("Stopping domain monitor for %s", self.domain.sdUUID) + self.log.debug("Stopping domain monitor for %s", self.sdUUID) # If this is an ISO domain we didn't acquire the host id and releasing # it is superfluous. @@ -153,16 +157,17 @@ self.domain.releaseHostId(self.hostId, unused=True) except: self.log.debug("Unable to release the host id %s for domain " - "%s", self.hostId, self.domain.sdUUID, exc_info=True) + "%s", self.hostId, self.sdUUID, exc_info=True) def _monitorDomain(self): self.nextStatus.clear() - # Refreshing the domain object in order to pick up changes as, - # for example, the domain upgrade. if time() - self.lastRefresh > self.refreshTime: - self.log.debug("Refreshing domain %s", self.domain.sdUUID) - sdCache.manuallyRemoveDomain(self.domain.sdUUID) + # Refreshing the domain object in order to pick up changes as, + # for example, the domain upgrade. + self.log.debug("Refreshing domain %s", self.sdUUID) + sdCache.manuallyRemoveDomain(self.sdUUID) + self.domain = sdCache.produce(self.sdUUID) self.lastRefresh = time() try: @@ -188,20 +193,19 @@ except Exception, e: self.log.error("Error while collecting domain %s monitoring " - "information", self.domain.sdUUID, exc_info=True) + "information", self.sdUUID, exc_info=True) self.nextStatus.error = e self.nextStatus.lastCheck = time() self.nextStatus.valid = (self.nextStatus.error is None) if self.status.valid != self.nextStatus.valid: - self.log.debug("Domain %s changed its status to %s", - self.domain.sdUUID, + self.log.debug("Domain %s changed its status to %s", self.sdUUID, "Valid" if self.nextStatus.valid else "Invalid") try: self.onDomainConnectivityStateChange.emit( - self.domain.sdUUID, self.nextStatus.valid) + self.sdUUID, self.nextStatus.valid) except: self.log.warn("Could not emit domain state change event", exc_info=True) @@ -213,7 +217,7 @@ self.domain.acquireHostId(self.hostId, async=True) except: self.log.debug("Unable to issue the acquire host id %s " - "request for domain %s", self.hostId, self.domain.sdUUID, + "request for domain %s", self.hostId, self.sdUUID, exc_info=True) self.status.update(self.nextStatus) diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index 9a7d09d..eee8d73 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -137,7 +137,10 @@ def manuallyRemoveDomain(self, sdUUID): with self._syncroot: - del self.__cache[sdUUID] + try: + del self.__cache[sdUUID] + except KeyError: + pass storage_repository = config.get('irs', 'repository') diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index ed2b893..da403d9 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1022,8 +1022,8 @@ if targetFormat is None: targetFormat = self.getFormat() - self._formatConverter.convert(repoPath, self.id, domain.getRealDomain(), - isMsd, targetFormat) + self._formatConverter.convert(repoPath, self.id, domain, isMsd, + targetFormat) sdCache.manuallyRemoveDomain(sdUUID) @unsecured @@ -1557,8 +1557,7 @@ for sdUUID in activeDomains: if sdUUID not in monitoredDomains: try: - self.domainMonitor \ - .startMonitoring(sdCache.produce(sdUUID), self.id) + self.domainMonitor.startMonitoring(sdUUID, self.id) self.log.debug("Storage Pool `%s` started monitoring " "domain `%s`", self.spUUID, sdUUID) except se.StorageException: -- To view, visit http://gerrit.ovirt.org/7294 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibbf67fc050658e3418aa666e8fcef1e1244571e9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

3 4

Change in vdsm[master]: Revert "BZ#842631 Use domain proxies instead of actual domai...
by Federico Simoncelli 01 Mar '13

01 Mar '13

Federico Simoncelli has uploaded a new change for review. Change subject: Revert "BZ#842631 Use domain proxies instead of actual domain references" ...................................................................... Revert "BZ#842631 Use domain proxies instead of actual domain references" This reverts commit 942c2dc8d317b6180529ed7df4eee98fbf1b9836. Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> Change-Id: Id4b1e0600948ab24e144df94f8d02a101a87627a --- M vdsm/storage/sdc.py 1 file changed, 11 insertions(+), 40 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/93/7293/1 diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index 2e8a6ff..9a7d09d 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -36,21 +36,6 @@ DEFAULT_REFRESH_INTERVAL = 300 -class DomainProxy(object): - """Keeps domain references valid even when underlying domain object changes - (due to format conversion for example""" - def __init__(self, cache, sdUUID): - self._sdUUID = sdUUID - self._cache = cache - - def __getattr__(self, attrName): - dom = self.getRealDomain() - return getattr(dom, attrName) - - def getRealDomain(self): - return self._cache._realProduce(self._sdUUID) - - class StorageDomainCache: """ Storage Domain List keeps track of all the storage domains accessible by @@ -59,9 +44,9 @@ log = logging.getLogger('Storage.StorageDomainCache') def __init__(self, storage_repo): - self._syncroot = threading.RLock() - self.__proxyCache = {} - self.__domainCache = {} + self._syncroot = threading.Lock() + self.__cache = {} + self.__weakCache = {} self.storage_repo = storage_repo self.storageStale = True @@ -79,14 +64,14 @@ if self.storageStale == True: return None try: - return self.__proxyCache[sdUUID]() + return self.__weakCache[sdUUID]() except KeyError: return None def _cleanStaleWeakrefs(self): - for sdUUID, ref in self.__proxyCache.items(): + for sdUUID, ref in self.__weakCache.items(): if ref() is None: - del self.__proxyCache[sdUUID] + del self.__weakCache[sdUUID] def produce(self, sdUUID): dom = self._getDomainFromCache(sdUUID) @@ -103,25 +88,11 @@ self._cleanStaleWeakrefs() - dom = DomainProxy(self, sdUUID) - # This is needed to preserve the semantic where if the domain was - # absent from the cache and the domain cannot be found the - # operation would fail - dom.getRealDomain() - self.__proxyCache[sdUUID] = weakref.ref(dom) - return dom - - def _realProduce(self, sdUUID): - with self._syncroot: - try: - return self.__domainCache[sdUUID] - except KeyError: - pass - # _findDomain will raise StorageDomainDoesNotExist if sdUUID is not # found in storage. dom = self._findDomain(sdUUID) - self.__domainCache[sdUUID] = dom + self.__cache[sdUUID] = dom + self.__weakCache[sdUUID] = weakref.ref(dom) return dom def _findDomain(self, sdUUID): @@ -158,15 +129,15 @@ def refresh(self): self.invalidateStorage() - self.__domainCache.clear() + self.__cache.clear() def manuallyAddDomain(self, dom): with self._syncroot: - self.__domainCache[dom.sdUUID] = dom + self.__cache[dom.sdUUID] = dom def manuallyRemoveDomain(self, sdUUID): with self._syncroot: - del self.__domainCache[sdUUID] + del self.__cache[sdUUID] storage_repository = config.get('irs', 'repository') -- To view, visit http://gerrit.ovirt.org/7293 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id4b1e0600948ab24e144df94f8d02a101a87627a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

3 4

Change in vdsm[master]: Rename volumeCopy API to volumeCollapse
by Federico Simoncelli 01 Mar '13

01 Mar '13

Federico Simoncelli has uploaded a new change for review. Change subject: Rename volumeCopy API to volumeCollapse ...................................................................... Rename volumeCopy API to volumeCollapse copyImage (the internal call used by the former volumeCopy) creates a new template/volume collapsing the whole chain (base->volUUID). This is not just a nomenclature problem since we need to implement the real volumeCopy. Change-Id: Ide39bb54d2a97b3ba8d4712750ff10a67c5764f2 --- M vdsm/API.py M vdsm/BindingXMLRPC.py 2 files changed, 11 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/68/2968/1 -- To view, visit http://gerrit.ovirt.org/2968 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ide39bb54d2a97b3ba8d4712750ff10a67c5764f2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

3 5

Change in vdsm[master]: Extend the range of Vm's niceness to allow increasing cpu sh...
by wudxw＠linux.vnet.ibm.com 21 Feb '13

21 Feb '13

Mark Wu has uploaded a new change for review. Change subject: Extend the range of Vm's niceness to allow increasing cpu shares ...................................................................... Extend the range of Vm's niceness to allow increasing cpu shares The current range of Vm's niceness is [0, 19], and the default value 0 represnts the biggest cpu shares. So if we want to assign more cpu shares to one vm, we have to give a higher niceness to other vms.This patch extends the range to [-19, 19], which makes the default value 0 be the middle of it. It's more flexible for client to set Vm's niceness. Change-Id: I5ae55daa5655dbd10c200dfd81ba80d8e71abfb5 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/89/6289/1 -- To view, visit http://gerrit.ovirt.org/6289 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5ae55daa5655dbd10c200dfd81ba80d8e71abfb5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

4 7

Change in vdsm[master]: Replace calling setSchedulerParameters() with filling its XM...
by wudxw＠linux.vnet.ibm.com 21 Feb '13

21 Feb '13

Mark Wu has uploaded a new change for review. Change subject: Replace calling setSchedulerParameters() with filling its XML description. ...................................................................... Replace calling setSchedulerParameters() with filling its XML description. This patch changes to fill cgroup cpu.share into its XML description to set Vm's niceness instead of calling setSchedulerParameters(). It can save us a libvirt call on VM statup. Change-Id: I211e191022f5a18fa7d97d5a8fb42e10729ddd06 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 10 insertions(+), 10 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/90/6290/1 -- To view, visit http://gerrit.ovirt.org/6290 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I211e191022f5a18fa7d97d5a8fb42e10729ddd06 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

4 11

Change in vdsm[master]: Allow tobool() convert int into bool.
by wudxw＠linux.vnet.ibm.com 21 Feb '13

21 Feb '13

Mark Wu has uploaded a new change for review. Change subject: Allow tobool() convert int into bool. ...................................................................... Allow tobool() convert int into bool. The original code return False for any number. This patch makes it return the result of bool(number). That means for any nonzero nubmer it returns True, and returns False for 0. This patch also adds a test case for tobool(). Change-Id: I6ae878b1171fe6827441751d6f02167e12e8bc03 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M tests/Makefile.am A tests/utilsTests.py M vdsm/utils.py 3 files changed, 39 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/80/8480/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index a5e4c64..eb21f25 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -42,12 +42,13 @@ parted_utils_tests.py \ permutationTests.py \ persistentDictTests.py \ - restTests.py \ - restData.py \ - tcTests.py \ - vdsClientTests.py \ remoteFileHandlerTests.py \ - resourceManagerTests.py + resourceManagerTests.py \ + restData.py \ + restTests.py \ + tcTests.py \ + utilsTests.py \ + vdsClientTests.py dist_noinst_DATA = \ run_tests_local.sh diff --git a/tests/utilsTests.py b/tests/utilsTests.py new file mode 100644 index 0000000..01d8de0 --- /dev/null +++ b/tests/utilsTests.py @@ -0,0 +1,32 @@ +# +# Copyright IBM Corp. 2012 +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +from vdsm import utils +from testrunner import VdsmTestCase as TestCaseBase + + +class TestUtils(TestCaseBase): + + def test_tobool(self): + self.assertEqual(utils.tobool('true'), True) + self.assertEqual(utils.tobool('false'), False) + self.assertEqual(utils.tobool('foo'), False) + self.assertEqual(utils.tobool(None), False) + self.assertEqual(utils.tobool(0), False) + self.assertEqual(utils.tobool(1), True) diff --git a/vdsm/utils.py b/vdsm/utils.py index 0dbb342..96aa928 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -657,7 +657,7 @@ return False if type(s) == bool: return s - if s.lower() == 'true': + if type(s) == str and s.lower() == 'true': return True return bool(int(s)) except: -- To view, visit http://gerrit.ovirt.org/8480 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6ae878b1171fe6827441751d6f02167e12e8bc03 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

1 2

Change in vdsm[master]: Add simple text-based console in vdsClient
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: Add simple text-based console in vdsClient ...................................................................... Add simple text-based console in vdsClient Add new command 'openConsole' in vdsClient. This command invoke 'virsh console' directly. This is easy implement for text-based console. It means we can have a vdsm text console right now. We can improve text console in futrue. Change-Id: I368dbcbc2e180161f256f3fb450c344acb6a6c8a Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm_cli/vdsClient.py 1 file changed, 13 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/41/8041/1 diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index fe2f083..e49e66a 100644 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -24,6 +24,7 @@ import re import socket import pprint as pp +import subprocess from vdsm import vdscli try: @@ -122,8 +123,10 @@ def __init__(self): self.useSSL = False self.truststore = None + self.server = "0" def do_connect(self, server, port): + self.server = server self.s = vdscli.connect(server + ':' + port, self.useSSL, self.truststore) @@ -1607,6 +1610,13 @@ return status['status']['code'], status['status']['message'] + def openVmConsole(self, args): + protocol = "tls" if self.useSSL else "tcp" + conn = "qemu+" + protocol + "://" + self.server + "/system" + p = subprocess.Popen(["virsh", "-c", conn, "console", args[0]]) + p.wait() + + if __name__ == '__main__': if _glusterEnabled: serv = ge.GlusterService() @@ -2265,6 +2275,9 @@ ('<vmId> <sdUUID> <imgUUID> <baseVolUUID> <volUUID>', "Take a live snapshot" )), + 'openConsole': (serv.openVmConsole, + ("<vmName>", + "open pty console for vm")), } if _glusterEnabled: commands.update(ge.getGlusterCmdDict(serv)) -- To view, visit http://gerrit.ovirt.org/8041 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I368dbcbc2e180161f256f3fb450c344acb6a6c8a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

5 16

Change in vdsm[master]: Add synchronous mode for udevTrigger
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: Add synchronous mode for udevTrigger ...................................................................... Add synchronous mode for udevTrigger udevadm trigger was asynchronous. Execute udevadm settle after udevadm trigger, then it can works as synchronous. Change-Id: I731c3ef6fe945c7c0278d638bf4cc8164345fd01 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm/storage/hsm.py M vdsm/supervdsmServer.py 2 files changed, 6 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/16/6916/1 -- To view, visit http://gerrit.ovirt.org/6916 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I731c3ef6fe945c7c0278d638bf4cc8164345fd01 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

4 7

Change in vdsm[master]: iscsi: fix update discoverydb with wrong key
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: iscsi: fix update discoverydb with wrong key ...................................................................... iscsi: fix update discoverydb with wrong key When execute discoverSendTargets will get errror message: Thread-15::DEBUG::2012-07-10 10:44:32,384::__init__::1249::Storage.Misc.excCmd::(_log) '/usr/bin/sudo -n /usr/sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 192.168.122.89:3260 -n node.startup -v manual --op=update' (cwd None) Thread-15::DEBUG::2012-07-10 10:44:32,404::__init__::1249::Storage.Misc.excCmd::(_log) FAILED: <err> = 'iscsiadm: Cannot modify node.startup. Invalid param name.\n'; <rc> = 7 We should set discovery.startup when discover targets. Change-Id: Ic561c3a69d2f3ec8b5f83b1837e0d126cb5a314a Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm/storage/iscsi.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/6092/1 -- To view, visit http://gerrit.ovirt.org/6092 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic561c3a69d2f3ec8b5f83b1837e0d126cb5a314a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

3 5

Change in vdsm[master]: BZ#809497: connectStroragServer will fail if iscsi session a...
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: BZ#809497: connectStroragServer will fail if iscsi session already exists ...................................................................... BZ#809497: connectStroragServer will fail if iscsi session already exists Add check for connection. If iscsi session existed, will not add iscsi node. Change-Id: I3e012e69b5b27841e6f29cce0e1864b8acf332c2 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm/storage/storageServer.py 1 file changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/57/5957/1 -- To view, visit http://gerrit.ovirt.org/5957 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3e012e69b5b27841e6f29cce0e1864b8acf332c2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

2 2

Change in vdsm[master]: add hack for regenerate vdsm.spec with configure param from ...
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: add hack for regenerate vdsm.spec with configure param from vdsm.spec ...................................................................... add hack for regenerate vdsm.spec with configure param from vdsm.spec Because vdsm.spec was generated before rpm build, @marco@ wasn't generated with configure params from vdsm.spec. This hack will fix this problem. Change-Id: I2fdf228f79f6f626f4faa9c6a7f342a2c60d4f54 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M Makefile.am M autobuild.sh 2 files changed, 15 insertions(+), 10 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/90/5790/1 -- To view, visit http://gerrit.ovirt.org/5790 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2fdf228f79f6f626f4faa9c6a7f342a2c60d4f54 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

2 4

Change in vdsm[master]: add configure variable vdsmcoredumpdir for /var/log/core
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: add configure variable vdsmcoredumpdir for /var/log/core ...................................................................... add configure variable vdsmcoredumpdir for /var/log/core 6106db0db4832fd0e9a9f08c288d524d99fb0799 was reverted. Because vdsm.spec was generated before rpm build, so we won't use @VDSMCOREDUMPDIR@ in vdsm.spec.in. Change-Id: I993c5d550dd828800eb6ac68e74072489caa996a Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M build-aux/Makefile.subs M configure.ac M vdsm/Makefile.am M vdsm/sos/vdsm.py.in M vdsm/vdsm-logrotate.conf.in R vdsm/vdsm-logrotate.in 6 files changed, 10 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/23/5723/1 -- To view, visit http://gerrit.ovirt.org/5723 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I993c5d550dd828800eb6ac68e74072489caa996a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

3 6

Change in vdsm[master]: remove su directive in logrotate for el6
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: remove su directive in logrotate for el6 ...................................................................... remove su directive in logrotate for el6 In el6 didn't support su directive, so remove it and add limition in spec file. Change-Id: Ie01d9b020b0d70ae76ab1e4d81626202fd8d12c6 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm.spec.in M vdsm/vdsm-logrotate-coredump.conf.in 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/5155/1 -- To view, visit http://gerrit.ovirt.org/5155 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie01d9b020b0d70ae76ab1e4d81626202fd8d12c6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

3 6

Change in vdsm[master]: add vdsm-tool verb configure-core-dump
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: add vdsm-tool verb configure-core-dump ...................................................................... add vdsm-tool verb configure-core-dump Change-Id: Ifed530a35339e1e2a366a61f1bce1a8c9e71db53 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm-tool/Makefile.am A vdsm-tool/configureCoreDump.py M vdsm.spec.in M vdsm/constants.py.in M vdsm/vdsmd.init.in 5 files changed, 48 insertions(+), 17 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/54/5154/1 -- To view, visit http://gerrit.ovirt.org/5154 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifed530a35339e1e2a366a61f1bce1a8c9e71db53 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

2 5

Change in vdsm[master]: move monitored_paths to configure.ac
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: move monitored_paths to configure.ac ...................................................................... move monitored_paths to configure.ac move monitored_paths to configure.ac, then we can make it configurable Change-Id: I5d2c468c56f0c97eadacd1905c7e7def5a207963 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M build-aux/Makefile.subs M configure.ac M vdsm/constants.py.in M vdsm/utils.py 4 files changed, 10 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/53/5153/1 -- To view, visit http://gerrit.ovirt.org/5153 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5d2c468c56f0c97eadacd1905c7e7def5a207963 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

2 5

Change in vdsm[master]: split logrotate configure into two file for coredump and log
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: split logrotate configure into two file for coredump and log ...................................................................... split logrotate configure into two file for coredump and log We will only collected core dump for rl6 in default, so split it into two file will easy for config. Change-Id: Ic22b4ac89583aa8a73863a7c3e1574895159c978 Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M vdsm.spec.in M vdsm/Makefile.am R vdsm/vdsm-logrotate-coredump A vdsm/vdsm-logrotate-coredump.conf.in A vdsm/vdsm-logrotate-log A vdsm/vdsm-logrotate-log.conf.in D vdsm/vdsm-logrotate.conf.in 7 files changed, 50 insertions(+), 30 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/52/5152/1 -- To view, visit http://gerrit.ovirt.org/5152 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic22b4ac89583aa8a73863a7c3e1574895159c978 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

3 14

Change in vdsm[master]: Remove '/var/log/core' for fedora
by Xu He Jie 21 Feb '13

21 Feb '13

Xu He Jie has uploaded a new change for review. Change subject: Remove '/var/log/core' for fedora ...................................................................... Remove '/var/log/core' for fedora Remove '/var/log/core' for fedora, this directory only be created for RHEL Change-Id: I594b5aa2ecaad1b1a268fa231a5769ef437ec75e Signed-off-by: Xu He Jie <xuhj(a)linux.vnet.ibm.com> --- M build-aux/Makefile.subs M configure.ac M vdsm/Makefile.am M vdsm/constants.py.in M vdsm/sos/vdsm.py.in M vdsm/utils.py M vdsm/vdsm-logrotate.conf.in A vdsm/vdsm-logrotate.conf.rhel.in M vdsm/vdsmd.init.in 9 files changed, 56 insertions(+), 27 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/48/4348/1 -- To view, visit http://gerrit.ovirt.org/4348 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I594b5aa2ecaad1b1a268fa231a5769ef437ec75e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Xu He Jie <xuhj(a)linux.vnet.ibm.com>

4 12

Change in vdsm[master]: Refactor configNetwork.py: add a new class NativeConfigurato...
by wudxw＠linux.vnet.ibm.com 21 Feb '13

21 Feb '13

Mark Wu has uploaded a new change for review. Change subject: Refactor configNetwork.py: add a new class NativeConfigurator ...................................................................... Refactor configNetwork.py: add a new class NativeConfigurator Basically, this patch just adds a new class NativeConfigurator and moves the functions of add, delete, edit and setup network into that class. The purpose of this patch is to make it easy to integrate netcf support by adding a wrapper to the actual network configurator. Change-Id: I052955a8e2b7b92b123a5179afd15b79798e6c50 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/configNetwork.py 1 file changed, 440 insertions(+), 414 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/15/7915/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 12e5e5b..803ba3c 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -855,293 +855,6 @@ _validateInterNetworkCompatibility(_netinfo, vlan, nic, bridged) -def addNetwork(network, vlan=None, bonding=None, nics=None, ipaddr=None, - netmask=None, mtu=None, gateway=None, force=False, - configWriter=None, bondingOptions=None, bridged=True, - **options): - nics = nics or () - _netinfo = netinfo.NetInfo() - bridged = utils.tobool(bridged) - - if mtu: - mtu = int(mtu) - - # Validation - if not utils.tobool(force): - logging.debug('validating network...') - _addNetworkValidation(_netinfo, network=network, - vlan=vlan, bonding=bonding, nics=nics, ipaddr=ipaddr, - netmask=netmask, gateway=gateway, bondingOptions=bondingOptions, - bridged=bridged, **options) - - logging.info("Adding network %s with vlan=%s, bonding=%s, nics=%s," - " bondingOptions=%s, mtu=%s, bridged=%s, options=%s", - network, vlan, bonding, nics, bondingOptions, - mtu, bridged, options) - - if configWriter is None: - configWriter = ConfigWriter() - - prevmtu = None - if mtu: - prevmtu = configWriter.getMaxMtu(nics, mtu) - - nic = nics[0] if nics else None - iface = bonding or nic - - # take down nics that need to be changed - vlanedIfaces = [v['iface'] for v in _netinfo.vlans.values()] - if bonding not in vlanedIfaces: - for nic in nics: - if nic not in vlanedIfaces: - ifdown(nic) - - if bridged: - configWriter.addBridge(network, ipaddr=ipaddr, netmask=netmask, - mtu=mtu, gateway=gateway, **options) - ifdown(network) - # We need to define (if requested) ip, mask & gateway on ifcfg-* - # only on most top device according to following order: - # bridge -> vlan -> bond -> nic - # For lower level devices we should ignore it. - # reset ip, netmask, gateway for lower level devices - ipaddr = netmask = gateway = None - - # For VLAN we should attach bridge only to the VLAN device - # rather than to underlying NICs or bond - brName = network if bridged else None - bridgeForNic = None if vlan else brName - - # We want to create config files (ifcfg-*) in top-down order - # (bridge->vlan->bond->nic) to be able to handle IP/NETMASK - # correctly for bridgeless networks - if vlan: - # don't ifup VLAN interface here, it should be done last, - # after the bond and nic up - configWriter.addVlan(vlan, iface, network=brName, - mtu=mtu, bridged=bridged, - ipaddr=ipaddr, netmask=netmask, - gateway=gateway, **options) - iface += '.' + vlan - # reset ip, netmask, gateway for lower level devices - ipaddr = netmask = gateway = None - - # First we need to prepare all conf files - if bonding: - configWriter.addBonding(bonding, bridge=bridgeForNic, - bondingOptions=bondingOptions, - mtu=max(prevmtu, mtu), - ipaddr=ipaddr, netmask=netmask, - gateway=gateway, **options) - # reset ip, netmask, gateway for lower level devices - ipaddr = netmask = gateway = None - - for nic in nics: - configWriter.addNic(nic, bonding=bonding, - bridge=bridgeForNic if not bonding else None, - mtu=max(prevmtu, mtu), - ipaddr=ipaddr, netmask=netmask, - gateway=gateway, **options) - - # Now we can run ifup for all interfaces - if bonding: - ifup(bonding) - - # NICs must be activated in the same order of boot time - # to expose the correct MAC address. - for nic in nicSort(nics): - ifup(nic) - - # Now we can ifup VLAN interface, because bond and nic already up - if vlan: - ifup(iface) - - if bridged: - if options.get('bootproto') == 'dhcp' and \ - not utils.tobool(options.get('blockingdhcp')): - # wait for dhcp in another thread, - # so vdsm won't get stuck (BZ#498940) - t = threading.Thread(target=ifup, name='ifup-waiting-on-dhcp', - args=(network,)) - t.daemon = True - t.start() - else: - ifup(network) - - # add libvirt network - configWriter.createLibvirtNetwork(network, bridged, iface) - - -def assertBridgeClean(bridge, vlan, bonding, nics): - brifs = os.listdir('/sys/class/net/%s/brif/' % bridge) - for nic in nics: - try: - brifs.remove(nic) - except: - pass - if vlan: - brif = (bonding or nics[0]) + '.' + vlan - else: - brif = bonding - try: - brifs.remove(brif) - except: - pass - - if brifs: - raise ConfigNetworkError(ne.ERR_USED_BRIDGE, - 'bridge %s has interfaces %s connected' % (bridge, brifs)) - - -def showNetwork(network): - _netinfo = netinfo.NetInfo() - if network not in _netinfo.networks: - print "Network %r doesn't exist" % network - return - - bridged = _netinfo.networks[network]['bridged'] - print "Network %s(Bridged: %s):" % (network, bridged) - - nics, vlan, bonding = _netinfo.getNicsVlanAndBondingForNetwork(network) - - if bridged: - ipaddr = _netinfo.networks[network]['addr'] - netmask = _netinfo.networks[network]['netmask'] - gateway = _netinfo.networks[network]['gateway'] - print "ipaddr=%s, netmask=%s, gateway=%s" % (ipaddr, netmask, gateway) - else: - iface = _netinfo.networks[network]['iface'] - ipaddr = _netinfo.nics[iface]['addr'] - netmask = _netinfo.nics[iface]['netmask'] - print "ipaddr=%s, netmask=%s" % (ipaddr, netmask) - - print "vlan=%s, bonding=%s, nics=%s" % (vlan, bonding, nics) - - -def listNetworks(): - _netinfo = netinfo.NetInfo() - print "Networks:", _netinfo.networks.keys() - print "Vlans:", _netinfo.vlans.keys() - print "Nics:", _netinfo.nics.keys() - print "Bondings:", _netinfo.bondings.keys() - - -def delNetwork(network, vlan=None, bonding=None, nics=None, force=False, - configWriter=None, implicitBonding=True, **options): - _netinfo = netinfo.NetInfo() - - if configWriter is None: - configWriter = ConfigWriter() - - if network not in _netinfo.networks: - logging.info("Network %r: doesn't exist in libvirt database", network) - if network in netinfo.bridges(): - configWriter.removeBridge(network) - else: - raise ConfigNetworkError(ne.ERR_BAD_BRIDGE, - "Cannot delete network %r: It doesn't exist " - "in the system" % network) - - if vlan: - configWriter.removeVlan(vlan, bonding or nics[0]) - - return - - nics, vlan, bonding = _netinfo.getNicsVlanAndBondingForNetwork(network) - bridged = _netinfo.networks[network]['bridged'] - - logging.info("Removing network %s with vlan=%s, bonding=%s, nics=%s," - "options=%s" % (network, vlan, bonding, nics, options)) - - if not utils.tobool(force): - if bonding: - validateBondingName(bonding) - if set(nics) != set(_netinfo.bondings[bonding]["slaves"]): - raise ConfigNetworkError(ne.ERR_BAD_NIC, - "delNetwork: %s are not all nics enslaved to %s" % - (nics, bonding)) - if vlan: - validateVlanId(vlan) - if bridged: - assertBridgeClean(network, vlan, bonding, nics) - - configWriter.setNewMtu(network=network, bridged=bridged) - configWriter.removeLibvirtNetwork(network) - - # We need to gather NetInfo again to refresh networks info from libvirt. - # The deleted bridge should never be up at this stage. - if network in netinfo.NetInfo().networks: - raise ConfigNetworkError(ne.ERR_USED_BRIDGE, - "delNetwork: bridge %s still exists" % network) - - if network and bridged: - configWriter.removeBridge(network) - - nic = nics[0] if nics else None - iface = bonding if bonding else nic - if iface: - if vlan: - configWriter.removeVlan(vlan, iface) - else: - cf = configWriter.NET_CONF_PREF + iface - if not bridged: - # When removing bridgeless non-VLANed network - # we need to remove IP/NETMASK from the cfg file - for key in ('IPADDR', 'NETMASK', 'GATEWAY', 'BOOTPROTO'): - configWriter._updateConfigValue(cf, key, '', True) - else: - # When removing bridged non-VLANed network - # we need to remove BRIDGE from the cfg file - configWriter._updateConfigValue(cf, 'BRIDGE', '', True) - - # The (relatively) new setupNetwork verb allows to remove a network - # defined on top of an bonding device without break the bond itself. - if implicitBonding: - if bonding and not bondingOtherUsers(network, vlan, bonding): - ifdown(bonding) - configWriter.removeBonding(bonding) - iface = None if bonding == iface else iface - - for nic in nics: - if not nicOtherUsers(network, vlan, bonding, nic): - ifdown(nic) - configWriter.removeNic(nic) - iface = None if nic == iface else iface - - # Now we can restart changed interface - if iface: - ifdown(iface) - ifup(iface) - - -def clientSeen(timeout): - start = time.time() - while timeout >= 0: - if os.stat(constants.P_VDSM_CLIENT_LOG).st_mtime > start: - return True - time.sleep(1) - timeout -= 1 - return False - - -def editNetwork(oldBridge, newBridge, vlan=None, bonding=None, nics=None, - **options): - configWriter = ConfigWriter() - try: - delNetwork(oldBridge, configWriter=configWriter, **options) - addNetwork(newBridge, vlan=vlan, bonding=bonding, nics=nics, - configWriter=configWriter, **options) - except: - configWriter.restoreBackups() - raise - if utils.tobool(options.get('connectivityCheck', False)): - if not clientSeen(int(options.get('connectivityTimeout', - CONNECTIVITY_TIMEOUT_DEFAULT))): - delNetwork(newBridge, force=True) - configWriter.restoreBackups() - return define.errCode['noConPeer']['status']['code'] - - def _validateNetworkSetup(networks={}, bondings={}): _netinfo = netinfo.NetInfo() @@ -1171,73 +884,441 @@ "Unknown nics in: %r" % list(nics)) -def _editBondings(bondings, configWriter): - """ Add/Edit bond interface """ - logger = logging.getLogger("_editBondings") +def assertBridgeClean(bridge, vlan, bonding, nics): + brifs = os.listdir('/sys/class/net/%s/brif/' % bridge) + for nic in nics: + try: + brifs.remove(nic) + except: + pass + if vlan: + brif = (bonding or nics[0]) + '.' + vlan + else: + brif = bonding + try: + brifs.remove(brif) + except: + pass - _netinfo = netinfo.NetInfo() + if brifs: + raise ConfigNetworkError(ne.ERR_USED_BRIDGE, + 'bridge %s has interfaces %s connected' % (bridge, brifs)) - for bond, bondAttrs in bondings.iteritems(): - logger.debug("Creating/Editing bond %s with attributes %s", - bond, bondAttrs) - brNets = list(_netinfo.getBridgedNetworksForIface(bond)) - # Only one bridged-non-VLANed network allowed on same nic/bond - bridge = brNets[0] if brNets else None +def clientSeen(timeout): + start = time.time() + while timeout >= 0: + if os.stat(constants.P_VDSM_CLIENT_LOG).st_mtime > start: + return True + time.sleep(1) + timeout -= 1 + return False - mtu = None - if bond in _netinfo.bondings: - # Save MTU for future set on NICs - confParams = netinfo.getIfaceCfg(bond) - mtu = confParams.get('MTU', None) - if mtu: - mtu = int(mtu) - ifdown(bond) - # Take down all bond's NICs. - for nic in _netinfo.getNicsForBonding(bond): - ifdown(nic) - configWriter.removeNic(nic) +class NativeConfigurator(): - # Note! In case we have bridge up and connected to the bond - # we will get error in log: - # (ifdown) bridge XXX is still up; can't delete it - # But, we prefer this behaviour instead of taking bridge down - # Anyway, we will not be able to take it down with connected VMs + def addNetwork(self, network, vlan=None, bonding=None, nics=None, + ipaddr=None, netmask=None, mtu=None, gateway=None, + force=False, configWriter=None, bondingOptions=None, + bridged=True, **options): + nics = nics or () + _netinfo = netinfo.NetInfo() + bridged = utils.tobool(bridged) + + if mtu: + mtu = int(mtu) + + # Validation + if not utils.tobool(force): + logging.debug('validating network...') + _addNetworkValidation(_netinfo, network=network, + vlan=vlan, bonding=bonding, nics=nics, ipaddr=ipaddr, + netmask=netmask, gateway=gateway, + bondingOptions=bondingOptions, bridged=bridged, **options) + + logging.info("Adding network %s with vlan=%s, bonding=%s, nics=%s," + " bondingOptions=%s, mtu=%s, bridged=%s, options=%s", + network, vlan, bonding, nics, bondingOptions, + mtu, bridged, options) + + if configWriter is None: + configWriter = ConfigWriter() + + prevmtu = None + if mtu: + prevmtu = configWriter.getMaxMtu(nics, mtu) + + nic = nics[0] if nics else None + iface = bonding or nic + + # take down nics that need to be changed + vlanedIfaces = [v['iface'] for v in _netinfo.vlans.values()] + if bonding not in vlanedIfaces: + for nic in nics: + if nic not in vlanedIfaces: + ifdown(nic) + + if bridged: + configWriter.addBridge(network, ipaddr=ipaddr, netmask=netmask, + mtu=mtu, gateway=gateway, **options) + ifdown(network) + # We need to define (if requested) ip, mask & gateway on ifcfg-* + # only on most top device according to following order: + # bridge -> vlan -> bond -> nic + # For lower level devices we should ignore it. + # reset ip, netmask, gateway for lower level devices + ipaddr = netmask = gateway = None + + # For VLAN we should attach bridge only to the VLAN device + # rather than to underlying NICs or bond + brName = network if bridged else None + bridgeForNic = None if vlan else brName + + # We want to create config files (ifcfg-*) in top-down order + # (bridge->vlan->bond->nic) to be able to handle IP/NETMASK + # correctly for bridgeless networks + if vlan: + # don't ifup VLAN interface here, it should be done last, + # after the bond and nic up + configWriter.addVlan(vlan, iface, network=brName, + mtu=mtu, bridged=bridged, + ipaddr=ipaddr, netmask=netmask, + gateway=gateway, **options) + iface += '.' + vlan + # reset ip, netmask, gateway for lower level devices + ipaddr = netmask = gateway = None # First we need to prepare all conf files - configWriter.addBonding(bond, bridge=bridge, mtu=mtu, - bondingOptions=bondAttrs.get('options', None)) + if bonding: + configWriter.addBonding(bonding, bridge=bridgeForNic, + bondingOptions=bondingOptions, + mtu=max(prevmtu, mtu), + ipaddr=ipaddr, netmask=netmask, + gateway=gateway, **options) + # reset ip, netmask, gateway for lower level devices + ipaddr = netmask = gateway = None - for nic in bondAttrs['nics']: - configWriter.addNic(nic, bonding=bond, mtu=mtu) + for nic in nics: + configWriter.addNic(nic, bonding=bonding, + bridge=bridgeForNic if not bonding else None, + mtu=max(prevmtu, mtu), + ipaddr=ipaddr, netmask=netmask, + gateway=gateway, **options) # Now we can run ifup for all interfaces - ifup(bond) + if bonding: + ifup(bonding) + # NICs must be activated in the same order of boot time # to expose the correct MAC address. - for nic in nicSort(bondAttrs['nics']): + for nic in nicSort(nics): ifup(nic) + # Now we can ifup VLAN interface, because bond and nic already up + if vlan: + ifup(iface) -def _removeBondings(bondings, configWriter): - """ Remove bond interface """ - logger = logging.getLogger("_removeBondings") + if bridged: + if options.get('bootproto') == 'dhcp' and \ + not utils.tobool(options.get('blockingdhcp')): + # wait for dhcp in another thread, + # so vdsm won't get stuck (BZ#498940) + t = threading.Thread(target=ifup, name='ifup-waiting-on-dhcp', + args=(network,)) + t.daemon = True + t.start() + else: + ifup(network) - _netinfo = netinfo.NetInfo() + # add libvirt network + configWriter.createLibvirtNetwork(network, bridged, iface) - for bond, bondAttrs in bondings.items(): - if 'remove' in bondAttrs: - nics = _netinfo.getNicsForBonding(bond) - logger.debug("Removing bond %r with nics = %s", bond, nics) - ifdown(bond) - configWriter.removeBonding(bond) + def delNetwork(self, network, vlan=None, bonding=None, nics=None, + force=False, configWriter=None, implicitBonding=True, + **options): + _netinfo = netinfo.NetInfo() + + if configWriter is None: + configWriter = ConfigWriter() + + if network not in _netinfo.networks: + logging.info("Network %r: doesn't exist in libvirt database", + network) + if network in netinfo.bridges(): + configWriter.removeBridge(network) + else: + raise ConfigNetworkError(ne.ERR_BAD_BRIDGE, + "Cannot delete network %r: It doesn't exist " + "in the system" % network) + + if vlan: + configWriter.removeVlan(vlan, bonding or nics[0]) + + return + + nics, vlan, bonding = _netinfo.getNicsVlanAndBondingForNetwork(network) + bridged = _netinfo.networks[network]['bridged'] + + logging.info("Removing network %s with vlan=%s, bonding=%s, nics=%s," + "options=%s" % (network, vlan, bonding, nics, options)) + + if not utils.tobool(force): + if bonding: + validateBondingName(bonding) + if set(nics) != set(_netinfo.bondings[bonding]["slaves"]): + raise ConfigNetworkError(ne.ERR_BAD_NIC, + "delNetwork: %s are not all nics enslaved to %s" % + (nics, bonding)) + if vlan: + validateVlanId(vlan) + if bridged: + assertBridgeClean(network, vlan, bonding, nics) + + configWriter.setNewMtu(network=network, bridged=bridged) + configWriter.removeLibvirtNetwork(network) + + # We need to gather NetInfo again to refresh networks info from + # libvirt. The deleted bridge should never be up at this stage. + if network in netinfo.NetInfo().networks: + raise ConfigNetworkError(ne.ERR_USED_BRIDGE, + "delNetwork: bridge %s still exists" % network) + + if network and bridged: + configWriter.removeBridge(network) + + nic = nics[0] if nics else None + iface = bonding if bonding else nic + if iface: + if vlan: + configWriter.removeVlan(vlan, iface) + else: + cf = configWriter.NET_CONF_PREF + iface + if not bridged: + # When removing bridgeless non-VLANed network + # we need to remove IP/NETMASK from the cfg file + for key in ('IPADDR', 'NETMASK', 'GATEWAY', 'BOOTPROTO'): + configWriter._updateConfigValue(cf, key, '', True) + else: + # When removing bridged non-VLANed network + # we need to remove BRIDGE from the cfg file + configWriter._updateConfigValue(cf, 'BRIDGE', '', True) + + # The (relatively) new setupNetwork verb allows to remove a network + # defined on top of an bonding device without break the bond itself. + if implicitBonding: + if bonding and not bondingOtherUsers(network, vlan, bonding): + ifdown(bonding) + configWriter.removeBonding(bonding) + iface = None if bonding == iface else iface for nic in nics: - ifdown(nic) - configWriter.removeNic(nic) + if not nicOtherUsers(network, vlan, bonding, nic): + ifdown(nic) + configWriter.removeNic(nic) + iface = None if nic == iface else iface - del bondings[bond] + # Now we can restart changed interface + if iface: + ifdown(iface) + ifup(iface) + + def editNetwork(self, oldBridge, newBridge, vlan=None, bonding=None, + nics=None, **options): + configWriter = ConfigWriter() + try: + self.delNetwork(oldBridge, configWriter=configWriter, **options) + self.addNetwork(newBridge, vlan=vlan, bonding=bonding, nics=nics, + configWriter=configWriter, **options) + except: + configWriter.restoreBackups() + raise + if utils.tobool(options.get('connectivityCheck', False)): + if not clientSeen(int(options.get('connectivityTimeout', + CONNECTIVITY_TIMEOUT_DEFAULT))): + delNetwork(newBridge, force=True) + configWriter.restoreBackups() + return define.errCode['noConPeer']['status']['code'] + + def _editBondings(self, bondings, configWriter): + """ Add/Edit bond interface """ + logger = logging.getLogger("_editBondings") + + _netinfo = netinfo.NetInfo() + + for bond, bondAttrs in bondings.iteritems(): + logger.debug("Creating/Editing bond %s with attributes %s", + bond, bondAttrs) + + brNets = list(_netinfo.getBridgedNetworksForIface(bond)) + # Only one bridged-non-VLANed network allowed on same nic/bond + bridge = brNets[0] if brNets else None + + mtu = None + if bond in _netinfo.bondings: + # Save MTU for future set on NICs + confParams = netinfo.getIfaceCfg(bond) + mtu = confParams.get('MTU', None) + if mtu: + mtu = int(mtu) + + ifdown(bond) + # Take down all bond's NICs. + for nic in _netinfo.getNicsForBonding(bond): + ifdown(nic) + configWriter.removeNic(nic) + + # Note! In case we have bridge up and connected to the bond + # we will get error in log: + # (ifdown) bridge XXX is still up; can't delete it + # But, we prefer this behaviour instead of taking bridge down + # Anyway, we will not be able to take it down with connected VMs + + # First we need to prepare all conf files + configWriter.addBonding(bond, bridge=bridge, mtu=mtu, + bondingOptions=bondAttrs.get('options', None)) + + for nic in bondAttrs['nics']: + configWriter.addNic(nic, bonding=bond, mtu=mtu) + + # Now we can run ifup for all interfaces + ifup(bond) + # NICs must be activated in the same order of boot time + # to expose the correct MAC address. + for nic in nicSort(bondAttrs['nics']): + ifup(nic) + + def _removeBondings(self, bondings, configWriter): + """ Remove bond interface """ + logger = logging.getLogger("_removeBondings") + + _netinfo = netinfo.NetInfo() + + for bond, bondAttrs in bondings.items(): + if 'remove' in bondAttrs: + nics = _netinfo.getNicsForBonding(bond) + logger.debug("Removing bond %r with nics = %s", bond, nics) + ifdown(bond) + configWriter.removeBonding(bond) + + for nic in nics: + ifdown(nic) + configWriter.removeNic(nic) + + del bondings[bond] + + def setupNetworks(self, networks={}, bondings={}, **options): + + logger = logging.getLogger("setupNetworks") + + try: + _netinfo = netinfo.NetInfo() + configWriter = ConfigWriter() + networksAdded = set() + # keep set netsWithNewBonds to be able remove + # a new added network if connectivity check fail. + # If a new network needs to be created on top of existing bond, + # we will need to keep the bond on rollback flow, + # else we will break the new created bond. + netsWithNewBonds = set() + + logger.debug("Setting up network according to configuration: " + "networks:%r, bondings:%r, options:%r" % (networks, + bondings, options)) + + force = options.get('force', False) + if not utils.tobool(force): + logging.debug("Validating configuration") + _validateNetworkSetup(dict(networks), dict(bondings)) + + logger.debug("Applying...") + try: + # Remove edited networks and networks with 'remove' attribute + for network, networkAttrs in networks.items(): + if network in _netinfo.networks: + logger.debug("Removing network %r" % network) + self.delNetwork(network, configWriter=configWriter, + force=force, implicitBonding=False) + if 'remove' in networkAttrs: + del networks[network] + else: + networksAdded.add(network) + + # Remove bonds with 'remove' attribute + self._removeBondings(bondings, configWriter) + + # Check whether bonds should be resized + self._editBondings(bondings, configWriter) + + # We need to use the newest host info + _ni = netinfo.NetInfo() + for network, networkAttrs in networks.iteritems(): + d = dict(networkAttrs) + if 'bonding' in d: + # we may not receive any information + # about the bonding device if it is unchanged + # In this case check whether this bond exists + # on host and take its parameters + if bondings.get(d['bonding']): + d['nics'] = bondings[d['bonding']]['nics'] + d['bondingOptions'] = \ + bondings[d['bonding']].get('options', None) + # we create a new bond + if network in networksAdded: + netsWithNewBonds.add(network) + elif d['bonding'] in _ni.bondings: + logger.debug("Updating bond %r info", d['bonding']) + d['nics'] = _ni.bondings[d['bonding']]['slaves'] + d['bondingOptions'] = \ + _ni.bondings[d['bonding']]['cfg'].get( + 'BONDING_OPTS', None) + else: + d['nics'] = [d.pop('nic')] + d['force'] = force + + logger.debug("Adding network %r" % network) + self.addNetwork(network, configWriter=configWriter, + implicitBonding=True, **d) + + if utils.tobool(options.get('connectivityCheck', True)): + logger.debug('Checking connectivity...') + if not clientSeen(int(options.get('connectivityTimeout', + CONNECTIVITY_TIMEOUT_DEFAULT))): + logger.info('Connectivity check failed, rolling back') + for network in networksAdded: + self.delNetwork(network, force=True, + implicitBonding=network in netsWithNewBonds) + raise ConfigNetworkError(ne.ERR_LOST_CONNECTION, + 'connectivity check failed') + except: + configWriter.restoreBackups() + raise + + except Exception, e: + # SuperVdsm eats the error, so let's print it ourselves + logger.error(e, exc_info=True) + raise + + +configurator = NativeConfigurator() + + +def addNetwork(network, vlan=None, bonding=None, nics=None, ipaddr=None, + netmask=None, mtu=None, gateway=None, force=False, + bondingOptions=None, bridged=True, **options): + configurator.addNetwork(network, vlan, bonding, nics, ipaddr, netmask, mtu, + gateway, force, bondingOptions, bridged, **options) + + +def delNetwork(network, vlan=None, bonding=None, nics=None, force=False, + implicitBonding=True, **options): + configurator.delNetwork(network, vlan, bonding, nics, force, + implicitBonding, **options) + + +def editNetwork(oldBridge, newBridge, vlan=None, bonding=None, nics=None, + **options): + configurator.editNetwork(oldBridge, newBridge, vlan, bonding, nics, + **options) def setupNetworks(networks={}, bondings={}, **options): @@ -1277,95 +1358,40 @@ the attachment in the network's attributes). Similarly, if you edit a bonding, it's not necessary to specify its networks. """ - logger = logging.getLogger("setupNetworks") + configurator.setupNetworks(networks, bondings, **options) - try: - _netinfo = netinfo.NetInfo() - configWriter = ConfigWriter() - networksAdded = set() - # keep set netsWithNewBonds to be able remove - # a new added network if connectivity check fail. - # If a new network needs to be created on top of existing bond, - # we will need to keep the bond on rollback flow, - # else we will break the new created bond. - netsWithNewBonds = set() - logger.debug("Setting up network according to configuration: " - "networks:%r, bondings:%r, options:%r" % (networks, - bondings, options)) +def showNetwork(network): + _netinfo = netinfo.NetInfo() + if network not in _netinfo.networks: + print "Network %r doesn't exist" % network + return - force = options.get('force', False) - if not utils.tobool(force): - logging.debug("Validating configuration") - _validateNetworkSetup(dict(networks), dict(bondings)) + bridged = _netinfo.networks[network]['bridged'] + print "Network %s(Bridged: %s):" % (network, bridged) - logger.debug("Applying...") - try: - # Remove edited networks and networks with 'remove' attribute - for network, networkAttrs in networks.items(): - if network in _netinfo.networks: - logger.debug("Removing network %r" % network) - delNetwork(network, configWriter=configWriter, force=force, - implicitBonding=False) - if 'remove' in networkAttrs: - del networks[network] - else: - networksAdded.add(network) + nics, vlan, bonding = _netinfo.getNicsVlanAndBondingForNetwork(network) - # Remove bonds with 'remove' attribute - _removeBondings(bondings, configWriter) + if bridged: + ipaddr = _netinfo.networks[network]['addr'] + netmask = _netinfo.networks[network]['netmask'] + gateway = _netinfo.networks[network]['gateway'] + print "ipaddr=%s, netmask=%s, gateway=%s" % (ipaddr, netmask, gateway) + else: + iface = _netinfo.networks[network]['iface'] + ipaddr = _netinfo.nics[iface]['addr'] + netmask = _netinfo.nics[iface]['netmask'] + print "ipaddr=%s, netmask=%s" % (ipaddr, netmask) - # Check whether bonds should be resized - _editBondings(bondings, configWriter) + print "vlan=%s, bonding=%s, nics=%s" % (vlan, bonding, nics) - # We need to use the newest host info - _ni = netinfo.NetInfo() - for network, networkAttrs in networks.iteritems(): - d = dict(networkAttrs) - if 'bonding' in d: - # we may not receive any information - # about the bonding device if it is unchanged - # In this case check whether this bond exists - # on host and take its parameters - if bondings.get(d['bonding']): - d['nics'] = bondings[d['bonding']]['nics'] - d['bondingOptions'] = \ - bondings[d['bonding']].get('options', None) - # we create a new bond - if network in networksAdded: - netsWithNewBonds.add(network) - elif d['bonding'] in _ni.bondings: - logger.debug("Updating bond %r info", d['bonding']) - d['nics'] = _ni.bondings[d['bonding']]['slaves'] - d['bondingOptions'] = \ - _ni.bondings[d['bonding']]['cfg'].get( - 'BONDING_OPTS', None) - else: - d['nics'] = [d.pop('nic')] - d['force'] = force - logger.debug("Adding network %r" % network) - addNetwork(network, configWriter=configWriter, - implicitBonding=True, **d) - - if utils.tobool(options.get('connectivityCheck', True)): - logger.debug('Checking connectivity...') - if not clientSeen(int(options.get('connectivityTimeout', - CONNECTIVITY_TIMEOUT_DEFAULT))): - logger.info('Connectivity check failed, rolling back') - for network in networksAdded: - delNetwork(network, force=True, - implicitBonding=network in netsWithNewBonds) - raise ConfigNetworkError(ne.ERR_LOST_CONNECTION, - 'connectivity check failed') - except: - configWriter.restoreBackups() - raise - - except Exception, e: - # SuperVdsm eats the error, so let's print it ourselves - logger.error(e, exc_info=True) - raise +def listNetworks(): + _netinfo = netinfo.NetInfo() + print "Networks:", _netinfo.networks.keys() + print "Vlans:", _netinfo.vlans.keys() + print "Nics:", _netinfo.nics.keys() + print "Bondings:", _netinfo.bondings.keys() def setSafeNetworkConfig(): -- To view, visit http://gerrit.ovirt.org/7915 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I052955a8e2b7b92b123a5179afd15b79798e6c50 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

3 10

Change in vdsm[master]: Remove external usage of CONNECTIVITY_TIMEOUT_DEFAULT
by wudxw＠linux.vnet.ibm.com 21 Feb '13

21 Feb '13

Mark Wu has uploaded a new change for review. Change subject: Remove external usage of CONNECTIVITY_TIMEOUT_DEFAULT ...................................................................... Remove external usage of CONNECTIVITY_TIMEOUT_DEFAULT This patch makes CONNECTIVITY_TIMEOUT_DEFAULT only used as default timeout in clientSeen internally. Change-Id: I080a765ed22b3ede4bc2d0e8238367760a5f420f Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/nativeNetConfig.py M vdsm/netConfig.py 2 files changed, 6 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/9319/1 diff --git a/vdsm/nativeNetConfig.py b/vdsm/nativeNetConfig.py index aca4a33..73631a2 100644 --- a/vdsm/nativeNetConfig.py +++ b/vdsm/nativeNetConfig.py @@ -1011,8 +1011,7 @@ configWriter.restoreBackups() raise if utils.tobool(options.get('connectivityCheck', False)): - if not netConfig.clientSeen(int(options.get('connectivityTimeout', - netConfig.CONNECTIVITY_TIMEOUT_DEFAULT))): + if not netConfig.clientSeen(options.get('connectivityTimeout')): self.delNetwork(newBridge, force=True) configWriter.restoreBackups() return define.errCode['noConPeer']['status']['code'] @@ -1162,8 +1161,7 @@ if utils.tobool(options.get('connectivityCheck', True)): logger.debug('Checking connectivity...') if not netConfig.clientSeen( - int(options.get('connectivityTimeout', - netConfig.CONNECTIVITY_TIMEOUT_DEFAULT))): + options.get('connectivityTimeout')): logger.info('Connectivity check failed, rolling back') for network in networksAdded: diff --git a/vdsm/netConfig.py b/vdsm/netConfig.py index 8596d44..4a06fac 100644 --- a/vdsm/netConfig.py +++ b/vdsm/netConfig.py @@ -98,6 +98,10 @@ def clientSeen(timeout): + if timeout is None: + timeout = CONNECTIVITY_TIMEOUT_DEFAULT + else: + timeout = int(timeout) start = time.time() while timeout >= 0: if os.stat(constants.P_VDSM_CLIENT_LOG).st_mtime > start: -- To view, visit http://gerrit.ovirt.org/9319 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I080a765ed22b3ede4bc2d0e8238367760a5f420f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

2 4

Change in vdsm[master]: WIP: Add netcf support
by wudxw＠linux.vnet.ibm.com 21 Feb '13

21 Feb '13

Mark Wu has uploaded a new change for review. Change subject: WIP: Add netcf support ...................................................................... WIP: Add netcf support Change-Id: I0070ea45de9d75570168410913a0aef62bcd940a Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/configNetwork.py 1 file changed, 314 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/18/8618/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 0f6de37..abf4e1f 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -26,6 +26,7 @@ import time import logging import threading +import xml.dom.minidom from xml.sax.saxutils import escape import glob import shutil @@ -1299,6 +1300,319 @@ raise +class Netcf(object): + + def __init__(self): + self.doc = xml.dom.minidom.Document() + self.conn = libvirtconnection.get() + + def addIfaceType(self, ifaceType, name): + interface = self.doc.createElement('interface') + interface.setAttribute('type', ifaceType) + interface.setAttribute('name', name) + return interface + + def addTopIfaceAttrs(self, topIface, ipaddr=None, netmask=None, + gateway=None, mtu=None, dhcp=False, onboot=True): + + def toNetmaskBits(netmask): + num = int(''.join('%02x' % int(i) for i in netmask.split('.')), 16) + bits = 32 + while num / 2 != 0: + if num % 2 == 0: + bits -= 1 + num /= 2 + return bits + + firstChild = topIface.firstChild + + if onboot: + start = self.doc.createElement('start') + start.setAttribute('mode', 'onboot') + topIface.insertBefore(start, firstChild) + + if mtu != None: + mtuChild = self.doc.createElement('mtu') + mtuChild.setAttribute('size', str(mtu)) + topIface.insertBefore(mtuChild, firstChild) + + protocol = self.doc.createElement('protocol') + protocol.setAttribute('family', "ipv4") + if ipaddr: + ip = self.doc.createElement('ip') + ip.setAttribute('address', (pipes.quote(ipaddr))) + prefix = str(toNetmaskBits(pipes.quote(netmask))) + ip.setAttribute('prefix', prefix) + protocol.appendChild(ip) + if gateway: + route = self.doc.createElement('route') + route.setAttribute('gateway', (pipes.quote(gateway))) + protocol.appendChild(ip) + elif dhcp: + dhcp = self.doc.createElement('dhcp') + protocol.appendChild(dhcp) + topIface.insertBefore(protocol, firstChild) + return topIface + + def addBridge(self, name, port, ipaddr=None, netmask=None, mtu=None, + gateway=None, bootproto=None, delay='0', onboot='yes', **kwargs): + """ + <interface type="bridge" name="br0"> + <start mode="onboot"/> + <mtu size="1500"/> + <protocol family="ipv4"> + <ip address="192.168.0.5" prefix="24"/> + <route gateway="192.168.0.1"/> + </protocol> + <bridge stp="off" delay="0.01"> + <interface type="ethernet" name="eth0"> + <mac address="ab:bb:cc:dd:ee:ff"/> + </interface> + <interface type="ethernet" name="eth1"/> + </bridge> + </interface> + """ + + brIface = self.addIfaceType('bridge', pipes.quote(name)) + self.addTopIfaceAttrs(brIface, ipaddr, netmask, mtu, gateway) + bridge = self.doc.createElement('bridge') + bridge.setAttribute('delay', pipes.quote(delay)) + # fix me need send a patch to allow set 'NM_CONTROLLED=no\n' + bridge.appendChild(port) + brIface.appendChild(bridge) + return brIface + + """ + <start mode="onboot"/> + <protocol family="ipv4"> + <dhcp peerdns="no"/> + </protocol> + """ + def addBareVlan(self, vlanId, iface): + """ + <interface type="vlan" name="eth0.42"> + <vlan tag="42"> + <interface name="eth0"/> + </vlan> + </interface> + """ + ifaceName = iface.getAttribute('name') + vlanIface = self.addIfaceType("vlan", ifaceName + '.' + vlanId) + vlanDev = self.doc.createElement("vlan") + vlanDev.setAttribute("tag", vlanId) + nicIface = self.doc.createElement("interface") + nicIface.setAttribute("name", ifaceName) + vlanDev.appendChild(nicIface) + vlanIface.appendChild(vlanDev) + return vlanIface + + def addBareBonding(self, bonding, slaves, bondingOptions=None): + """ + <interface type="bond" name="bond0"> + <bond mode="active-backup"> + <arpmon interval="10" target="192.168.122.1"/> + <interface type="ethernet" name="eth0"> + <mac address="ab:bb:cc:dd:ee:ff"/> + </interface> + <interface type="ethernet" name="eth1"/> + </bond> + </interface> + """ + bondIface = self.addIfaceType("bond", bonding) + bond = self.doc.createElement('bond') + # Todo: parse bonding options + # the default options is bondingOptions = 'mode=802.3ad miimon=150' + bond.setAttribute('mode', "802.3ad") + for slave in slaves: + bond.appendChild(slave) + bondIface.appendChild(bond) + return bondIface + + def addBareNic(self, nic): + """ + <interface type="ethernet" name="eth0"> + <mac address="ab:bb:cc:dd:ee:ff"/> + </interface> + """ + _netinfo = netinfo.NetInfo() + hwaddr = _netinfo.nics[nic].get('permhwaddr') or \ + _netinfo.nics[nic]['hwaddr'] + + nicIface = self.addIfaceType('ethernet', nic) + mac = self.doc.createElement('mac') + mac.setAttribute('address', hwaddr) + nicIface.appendChild(mac) + return nicIface + + def netcfDefineIface(self, xmlstr): + try: + self.conn.interfaceDefineXML(xmlstr, 0) + except: + logging.error('Failed to define interface', exc_info=True) + + def netcfCreateIface(self, ifaceName): + iface = self.conn.interfaceLookupByName(ifaceName) + iface.create(0) + + def netcfUndefineIface(self, ifaceName): + iface = self.conn.interfaceLookupByName(ifaceName) + if iface.isActive(): + iface.destroy(0) + #Todo implement configuration rollback + #self.conn.changeBegin(0) + iface.undefine() + + #self.conn.changeCommit(0) + #self.conn.changeRollback(0) + + def _createNetwork(self, netXml): + conn = libvirtconnection.get() + net = conn.networkDefineXML(netXml) + net.create() + net.setAutostart(1) + + def createLibvirtNetwork(self, network, bridged=True, iface=None, + skipBackup=False): + netName = netinfo.LIBVIRT_NET_PREFIX + network + if bridged: + netXml = '''<network><name>%s</name><forward mode='bridge'/> + <bridge name='%s'/></network>''' % (escape(netName), + escape(network)) + else: + netXml = '''<network><name>%s</name><forward mode='passthrough'> + <interface dev='%s'/></forward></network>''' % \ + (escape(netName), escape(iface)) + if not skipBackup: + self._networkBackup(network) + self._createNetwork(netXml) + + def _removeNetwork(self, network): + netName = netinfo.LIBVIRT_NET_PREFIX + network + conn = libvirtconnection.get() + + net = conn.networkLookupByName(netName) + if net.isActive(): + net.destroy() + if net.isPersistent(): + net.undefine() + + def removeLibvirtNetwork(self, network, skipBackup=False): + if not skipBackup: + self._networkBackup(network) + self._removeNetwork(network) + + def _networkBackup(self, network): + """Not impolemented now""" + + pass + + +class NetcfConfigurator(): + + def __init__(self): + self.configWriter = Netcf() + + def addNetwork(self, network, vlan=None, bonding=None, nics=None, + ipaddr=None, netmask=None, mtu=None, gateway=None, + force=False, configWriter=None, bondingOptions=None, + bridged=True, **options): + nics = nics or () + _netinfo = netinfo.NetInfo() + bridged = utils.tobool(bridged) + + if mtu: + mtu = int(mtu) + + # Validation + if not utils.tobool(force): + logging.debug('validating network...') + _addNetworkValidation(_netinfo, network=network, + vlan=vlan, bonding=bonding, nics=nics, ipaddr=ipaddr, + netmask=netmask, gateway=gateway, + bondingOptions=bondingOptions, bridged=bridged, **options) + + logging.info("Adding network %s with vlan=%s, bonding=%s, nics=%s," + " bondingOptions=%s, mtu=%s, bridged=%s, options=%s", + network, vlan, bonding, nics, bondingOptions, + mtu, bridged, options) + + # Todo, consider mtu support for multiple vlans + + nic = nics[0] if nics else None + iface = bonding or nic + + # take down nics that need to be changed + vlanedIfaces = [v['iface'] for v in _netinfo.vlans.values()] + if bonding not in vlanedIfaces: + for nic in nics: + if nic not in vlanedIfaces: + ifdown(nic) + + if bridged: + ifdown(network) + + slaves = [] + for nic in nics: + topIface = self.configWriter.addBareNic(nic) + slaves.append(topIface) + if bonding: + topIface = self.configWriter.addBareBonding(bonding, slaves, + bondingOptions) + if vlan: + # netcf doesn't allow create the underlying device together with + # vlan. So create the underlying device before vlan. + topIface = self.configWriter.addTopIfaceAttrs(topIface) + topIfaceXml = topIface.toprettyxml(encoding='utf-8') + self.configWriter.netcfDefineIface(topIfaceXml) + topIface = self.configWriter.addBareVlan(vlan, topIface) + if bridged: + topIface = self.configWriter.addBridge( + network, port=topIface, ipaddr=ipaddr, netmask=netmask, + mtu=mtu, gateway=gateway, **options) + else: + topIface = self.configWriter.addTopIfaceAttrs( + topIface, ipaddr=ipaddr, netmask=netmask, mtu=mtu, + gateway=gateway) + + self.configWriter.netcfDefineIface( + topIface.toprettyxml(encoding='utf-8')) + + # Todo consider dhcp block issue + ifaceName = topIface.getAttribute('name') + self.configWriter.netcfCreateIface(ifaceName) + # add libvirt network + self.configWriter.createLibvirtNetwork(network, bridged, iface) + + def delNetworkNetcf(self, network, vlan=None, bonding=None, nics=None, + force=False, configWriter=None, implicitBonding=True, + **options): + _netinfo = netinfo.NetInfo() + + if network not in _netinfo.networks: + logging.info("Network %r: doesn't exist in libvirt database", + network) + if network in netinfo.bridges(): + self.configWriter.removeBridge(network) + else: + raise ConfigNetworkError(ne.ERR_BAD_BRIDGE, + "Cannot delete network %r: It doesn't exist " + "in the system" % network) + + if vlan: + self.configWriter.removeVlan(vlan, bonding or nics[0]) + + return + + self.configWriter.removeLibvirtNetwork(network, log=False) + # We need to gather NetInfo again to refresh networks info from + # libvirt. The deleted bridge should never be up at this stage. + if network in netinfo.NetInfo().networks: + raise ConfigNetworkError(ne.ERR_USED_BRIDGE, + "delNetwork: bridge %s still exists" % network) + + self.configWriter.netcfUndefineIface(network) + + configurator = NativeConfigurator() -- To view, visit http://gerrit.ovirt.org/8618 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0070ea45de9d75570168410913a0aef62bcd940a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

2 3

Change in vdsm[master]: Remove unused qemuImg.py.
by ewarszaw＠redhat.com 20 Feb '13

20 Feb '13

Eduardo has uploaded a new change for review. Change subject: Remove unused qemuImg.py. ...................................................................... Remove unused qemuImg.py. Change-Id: Ib3ff2fe876a838306a15b6773aa810c8102e30fa Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm.spec.in M vdsm/Makefile.am D vdsm/qemuImg.py 3 files changed, 0 insertions(+), 97 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/5582/1 -- To view, visit http://gerrit.ovirt.org/5582 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib3ff2fe876a838306a15b6773aa810c8102e30fa Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

4 6

Change in vdsm[master]: Adding threads limitation to misc.tmap
by ybronhei＠redhat.com 14 Feb '13

14 Feb '13

Yaniv Bronhaim has uploaded a new change for review. Change subject: Adding threads limitation to misc.tmap ...................................................................... Adding threads limitation to misc.tmap Using threads queue to keep the threads order and limit threads count I moved the function code next to itmap for readability. Currently noone calls to tmap function, so I don't care to change and fix it now to avoid future bugs. Exceptions that related to the queue usage will be raised as part of the function's exceptions. Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> Change-Id: I07845bfd78b9215e8994ac2ebe46a7ff78c85625 --- M vdsm/storage/misc.py 1 file changed, 59 insertions(+), 36 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/58/8858/1 diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 426d181..f1a4581 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -1085,42 +1085,6 @@ return tuple(res) -def tmap(func, iterable): - resultsDict = {} - error = [None] - - def wrapper(f, arg, index): - try: - resultsDict[index] = f(arg) - except Exception, e: - # We will throw the last error received - # we can only throw one error, and the - # last one is as good as any. This shouldn't - # happen. Wrapped methods should not throw - # exceptions, if this happens it's a bug - log.error("tmap caught an unexpected error", exc_info=True) - error[0] = e - resultsDict[index] = None - - threads = [] - for i, arg in enumerate(iterable): - t = threading.Thread(target=wrapper, args=(func, arg, i)) - threads.append(t) - t.start() - - for t in threads: - t.join() - - results = [None] * len(resultsDict) - for i, result in resultsDict.iteritems(): - results[i] = result - - if error[0] is not None: - raise error[0] - - return tuple(results) - - def getfds(): return [int(fd) for fd in os.listdir("/proc/self/fd")] @@ -1251,6 +1215,65 @@ raise exception +def tmap(func, iterable, maxthreads=UNLIMITED_THREADS): + """ + Make an iterator that computes the function using arguments from the + iterable by running each operation in a different thread in specific + order. + maxthreads stands for maximum threads that we can initiate simultaneosly. + If we reached to max threads the function waits for thread to + finish before initiate the next one. + """ + if maxthreads < 1 and maxthreads != UNLIMITED_THREADS: + raise ValueError("Wrong input passed to function tmap: %s", maxthreads) + + resultsDict = {} + error = [None] + + def wrapper(f, arg, index): + try: + resultsDict[index] = f(arg) + except Exception, e: + # We will throw the last error received + # we can only throw one error, and the + # last one is as good as any. This shouldn't + # happen. Wrapped methods should not throw + # exceptions, if this happens it's a bug + log.error("tmap caught an unexpected error", exc_info=True) + error[0] = e + resultsDict[index] = None + + if maxthreads != UNLIMITED_THREADS: + threadsQueue = Queue.Queue(maxthreads) + else: + threadsQueue = Queue.Queue() + + for i, arg in enumerate(iterable): + if not threadsQueue.full(): + t = threading.Thread(target=wrapper, args=(func, arg, i)) + threadsQueue.put_nowait(t) + t.start() + else: + # waits for the first unfinished thread in list to finish if we + # have already initiate all possible thread's slots (maxthreads) + if threadsQueue.empty(): + raise RuntimeError("No thread initieated") + else: + threadsQueue.get_nowait().join() + + while not threadsQueue.empty(): + threadsQueue.get(False).join() + + results = [None] * len(resultsDict) + for i, result in resultsDict.iteritems(): + results[i] = result + + if error[0] is not None: + raise error[0] + + return tuple(results) + + def itmap(func, iterable, maxthreads=UNLIMITED_THREADS): """ Make an iterator that computes the function using -- To view, visit http://gerrit.ovirt.org/8858 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I07845bfd78b9215e8994ac2ebe46a7ff78c85625 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

5 20

Change in vdsm[master]: testRegeneration of remoteFileHandler fails when running all...
by ybronhei＠redhat.com 14 Feb '13

14 Feb '13

Yaniv Bronhaim has uploaded a new change for review. Change subject: testRegeneration of remoteFileHandler fails when running all tests ...................................................................... testRegeneration of remoteFileHandler fails when running all tests It happens when timeout is raised when calling to testEcho (we expect to normal result), this happens because handlers were full with testTimeout requests. This patch split the calls, and test the same issue. When all file handlers are done working, we release them and allow other handlers to start. Change-Id: I35ae1258d01455ad2fe131cd17bc3dff89224c1b Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M tests/remoteFileHandlerTests.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/9412/1 diff --git a/tests/remoteFileHandlerTests.py b/tests/remoteFileHandlerTests.py index 9508536..6c9233e 100644 --- a/tests/remoteFileHandlerTests.py +++ b/tests/remoteFileHandlerTests.py @@ -51,6 +51,7 @@ the requests""" for i in range(HANDLERS_NUM * 2): self.testTimeout() + for i in range(HANDLERS_NUM): self.testEcho() def tearDown(self): -- To view, visit http://gerrit.ovirt.org/9412 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I35ae1258d01455ad2fe131cd17bc3dff89224c1b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

3 9

Change in vdsm[master]: Alert if vdsm log has wrong user\group permissions
by ybronhei＠redhat.com 14 Feb '13

14 Feb '13

Yaniv Bronhaim has uploaded a new change for review. Change subject: Alert if vdsm log has wrong user\group permissions ...................................................................... Alert if vdsm log has wrong user\group permissions Bug-Id: https://bugzilla.redhat.com/show_bug.cgi?id=870139 Change-Id: I8bd379803b01627d6897992ee8798c6a22195b0f Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/vdsmd.init.in 1 file changed, 17 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/18/9118/1 diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 5b38d55..a3d8471 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -102,6 +102,18 @@ done } +verify_log_permissions() { + VDSM_USER=`id -u vdsm` + VDSM_GROUP=`id -g vdsm` + LOG_OWNER=`stat -c "%u" @VDSMLOGDIR@/vdsm.log` + LOG_GROUP=`stat -c "%g" @VDSMLOGDIR@/vdsm.log` + if [[ $VDSM_USER -ne $LOG_OWNER || $VDSM_GROUP -ne $LOG_GROUP ]]; + then + return 1 + fi + return 0 +} + get_libvirt_conf_item() { local cfile key @@ -496,6 +508,11 @@ return 1 fi + if [[ verify_log_permissions != 0 ]]; + then + log_failure_msg "Permissions error with vdsm log file" + fi + echo $"Starting up vdsm daemon: " local vdsm_nice=`$GETCONFITEM $CONF_FILE vars vdsm_nice -5` -- To view, visit http://gerrit.ovirt.org/9118 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8bd379803b01627d6897992ee8798c6a22195b0f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

6 24

Change in vdsm[master]: Adding VM based locks to ovf modify functions.
by ewarszaw＠redhat.com 14 Feb '13

14 Feb '13

Eduardo has uploaded a new change for review. Change subject: Adding VM based locks to ovf modify functions. ...................................................................... Adding VM based locks to ovf modify functions. Caveat Emptor: Avoids concurrent runs, but not serializes. Simultaneous updates or removes of ovf files (like removing many disks from the same VM) may result in inconsistent ovf's. Bug-url: https://bugzilla.redhat.com/show_bug.cgi?id=878894 Change-Id: I57d9f87a34e1d5659115b09530493824fcda94de Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/hsm.py M vdsm/storage/sp.py 2 files changed, 18 insertions(+), 17 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/9456/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 05d11c2..7e321de 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1215,13 +1215,19 @@ :type sdUUID: UUID :param options: ? """ - #getSharedLock(spUUID...) vars.task.getSharedLock(STORAGE, spUUID) - #getExclusiveLock(vmList...) + pool = self.getPool(spUUID) if sdUUID and sdUUID != sd.BLANK_UUID: pool.validatePoolSD(sdUUID) self.validateSdUUID(sdUUID) + else: + sdUUID = pool.masterDomain.sdUUID + + vmUUIDs = [vmDesc['vm'] for vmDesc in vmList] + vmUUIDs.sort() + for vmUUID in vmUUIDs: + vars.task.getExclusiveLock(STORAGE, "%s_%s" % (vmUUID, sdUUID)) pool.updateVM(vmList=vmList, sdUUID=sdUUID) @public @@ -1240,11 +1246,13 @@ :param options: ? """ vars.task.getSharedLock(STORAGE, spUUID) - #getExclusiveLock(vmList...) pool = self.getPool(spUUID) if sdUUID and sdUUID != sd.BLANK_UUID: pool.validatePoolSD(sdUUID) self.validateSdUUID(sdUUID) + else: + pool.masterDomain.sdUUID + vars.task.getExclusiveLock(STORAGE, "%s_%s" % (vmUUID, sdUUID)) pool.removeVM(vmUUID=vmUUID, sdUUID=sdUUID) @public diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 21ac4bf..816d4c3 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1253,7 +1253,7 @@ self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion) - def updateVM(self, vmList, sdUUID=None): + def updateVM(self, vmList, sdUUID): """ Update VMs. 'vmList' - [{'vm':vmUUID,'ovf','imglist':'imgUUID1,imgUUID2,...'},...] @@ -1262,9 +1262,6 @@ If sdUUID is None, the update is on the pool, and therefore the master domain will be updated. """ - if sdUUID is None: - sdUUID = self.masterDomain.sdUUID - self.log.info("spUUID=%s sdUUID=%s", self.spUUID, sdUUID) vms = self._getVMsPath(sdUUID) # We should exclude 'masterd' link from IMG_METAPATTERN globing @@ -1288,7 +1285,7 @@ raise se.MiscDirCleanupFailure(str(e)) try: - os.mkdir(vmPath) + fileUtils.createdir(vmPath) codecs.open(os.path.join(vmPath, vmUUID + '.ovf'), 'w', encoding='utf8').write(ovf) except OSError, ex: @@ -1298,7 +1295,7 @@ raise - def removeVM(self, vmUUID, sdUUID=None): + def removeVM(self, vmUUID, sdUUID): """ Remove VM. 'vmUUID' - Virtual machine UUID @@ -1635,16 +1632,12 @@ @unsecured def _getVMsPath(self, sdUUID): """ - Return general path of VMs from the pool. - If 'sdUUID' is given then return VMs dir within it. + Return VMs dir within SD with sdUUID. """ - if sdUUID and sdUUID != sd.BLANK_UUID: - if not self.isActive(sdUUID): - raise se.StorageDomainNotActive(sdUUID) - vmPath = sdCache.produce(sdUUID).getVMsDir() + if not self.isActive(sdUUID): + raise se.StorageDomainNotActive(sdUUID) + vmPath = sdCache.produce(sdUUID).getVMsDir() # Get VMs path from the pool (from the master domain) - else: - vmPath = self.masterDomain.getVMsDir() if not os.path.exists(vmPath): raise se.VMPathNotExists(vmPath) -- To view, visit http://gerrit.ovirt.org/9456 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I57d9f87a34e1d5659115b09530493824fcda94de Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

5 21

Change in vdsm[master]: Move hsm.validatePoolSD() -> pool.validatePoolSD().
by ewarszaw＠redhat.com 14 Feb '13

14 Feb '13

Eduardo has uploaded a new change for review. Change subject: Move hsm.validatePoolSD() -> pool.validatePoolSD(). ...................................................................... Move hsm.validatePoolSD() -> pool.validatePoolSD(). Related to BZ#878894. Change-Id: I3ba16c8a427c9478ae81c2693d79b8bd601f1ec3 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/hsm.py M vdsm/storage/sp.py 2 files changed, 25 insertions(+), 26 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/9455/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index e47eee1..05d11c2 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -268,13 +268,6 @@ raise se.StorageDomainTypeNotBackup(sdUUID) @classmethod - def validatePoolSD(cls, spUUID, sdUUID): - pool = cls.getPool(spUUID) - if sdUUID not in pool.getDomains(): - raise se.StorageDomainNotMemberOfPool(spUUID, sdUUID) - return pool - - @classmethod def validateNonDomain(cls, sdUUID): """ Validates that there is no domain with this UUID. @@ -642,12 +635,12 @@ se.VolumeExtendingError( "spUUID=%s, sdUUID=%s, volumeUUID=%s, size=%s" % (spUUID, sdUUID, volumeUUID, size))) - self.validatePoolSD(spUUID, sdUUID) size = misc.validateN(size, "size") # ExtendVolume expects size in MB size = math.ceil(size / 2 ** 20) pool = self.getPool(spUUID) + pool.validatePoolSD(sdUUID) pool.extendVolume(sdUUID, volumeUUID, size, isShuttingDown) @public @@ -671,10 +664,10 @@ se.StorageDomainActionError( "sdUUID=%s, devlist=%s" % (sdUUID, devlist))) - self.validatePoolSD(spUUID, sdUUID) vars.task.getExclusiveLock(STORAGE, sdUUID) # We need to let the domain to extend itself pool = self.getPool(spUUID) + pool.validatePoolSD(sdUUID) pool.extendSD(sdUUID, devlist, force) @public @@ -718,11 +711,10 @@ "sdUUID=%s, spUUID=%s, msdUUID=%s, masterVersion=%s" % (sdUUID, spUUID, msdUUID, masterVersion))) - self.validatePoolSD(spUUID, sdUUID) - vars.task.getExclusiveLock(STORAGE, spUUID) vars.task.getExclusiveLock(STORAGE, sdUUID) pool = self.getPool(spUUID) + pool.validatePoolSD(sdUUID) pool.detachSD(sdUUID) @public @@ -1108,11 +1100,11 @@ (sdUUID, spUUID, msdUUID, masterVersion) ) ) - self.validatePoolSD(spUUID, sdUUID) vars.task.getExclusiveLock(STORAGE, spUUID) vars.task.getExclusiveLock(STORAGE, sdUUID) pool = self.getPool(spUUID) + pool.validatePoolSD(sdUUID) pool.deactivateSD(sdUUID, msdUUID, masterVersion) @public @@ -1174,10 +1166,9 @@ :param description: The new human readable description of the volume. :type description: str """ - self.validatePoolSD(spUUID, sdUUID) - vars.task.getSharedLock(STORAGE, sdUUID) pool = self.getPool(spUUID) + pool.validatePoolSD(sdUUID) pool.setVolumeDescription(sdUUID, imgUUID, volUUID, description) @public @@ -1198,11 +1189,10 @@ :param description: The legality status ot the volume.? :type description: ? """ - self.validatePoolSD(spUUID, sdUUID) - vars.task.getSharedLock(STORAGE, sdUUID) pool = self.getPool(spUUID) + pool.validatePoolSD(sdUUID) pool.setVolumeLegality(sdUUID, imgUUID, volUUID, legality) @public @@ -1225,13 +1215,13 @@ :type sdUUID: UUID :param options: ? """ - if sdUUID and sdUUID != sd.BLANK_UUID: - self.validatePoolSD(spUUID, sdUUID) - self.validateSdUUID(sdUUID) #getSharedLock(spUUID...) vars.task.getSharedLock(STORAGE, spUUID) #getExclusiveLock(vmList...) pool = self.getPool(spUUID) + if sdUUID and sdUUID != sd.BLANK_UUID: + pool.validatePoolSD(sdUUID) + self.validateSdUUID(sdUUID) pool.updateVM(vmList=vmList, sdUUID=sdUUID) @public @@ -1249,13 +1239,12 @@ :type sdUUID: UUID :param options: ? """ - if sdUUID and sdUUID != sd.BLANK_UUID: - self.validatePoolSD(spUUID, sdUUID) - self.validateSdUUID(sdUUID) - #getSharedLock(spUUID...) vars.task.getSharedLock(STORAGE, spUUID) #getExclusiveLock(vmList...) pool = self.getPool(spUUID) + if sdUUID and sdUUID != sd.BLANK_UUID: + pool.validatePoolSD(sdUUID) + self.validateSdUUID(sdUUID) pool.removeVM(vmUUID=vmUUID, sdUUID=sdUUID) @public @@ -1274,7 +1263,7 @@ """ pool = self.getPool(spUUID) if sdUUID and sdUUID != sd.BLANK_UUID: - self.validatePoolSD(spUUID, sdUUID) + pool.validatePoolSD(sdUUID) self.validateSdUUID(sdUUID) else: sdUUID = pool.masterDomain.sdUUID @@ -1302,7 +1291,7 @@ """ pool = self.getPool(spUUID) if sdUUID and sdUUID != sd.BLANK_UUID: - self.validatePoolSD(spUUID, sdUUID) + pool.validatePoolSD(sdUUID) # Only backup domains are allowed in this path self.validateBackupDom(sdUUID) else: diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 25134d5..21ac4bf 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -456,13 +456,23 @@ def getMasterVersion(self): return self.getMetaParam(PMDK_MASTER_VER) + # TODO: Remove or rename this function. + def validatePoolSD(self, sdUUID): + if sdUUID not in self.getDomains(): + raise se.StorageDomainNotMemberOfPool(self.spUUID, sdUUID) + return True + @unsecured def validateAttachedDomain(self, dom, domainStatuses): """ Avoid handling domains if not owned by pool. """ - if dom.sdUUID not in domainStatuses.iterkeys(): + try: + self.validatePoolSD(dom.sdUUID) + except se.StorageDomainNotMemberOfPool: + # TODO: Remove this exception translation. Misleading. raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID) + pools = dom.getPools() if self.spUUID not in pools: raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID) -- To view, visit http://gerrit.ovirt.org/9455 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3ba16c8a427c9478ae81c2693d79b8bd601f1ec3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

7 21

Change in vdsm[master]: BZ#841555 A migration failure with an "AttributeError" excep...
by ghammer＠redhat.com 12 Feb '13

12 Feb '13

Gal Hammer has uploaded a new change for review. Change subject: BZ#841555 A migration failure with an "AttributeError" exception. ...................................................................... BZ#841555 A migration failure with an "AttributeError" exception. Libvirt sometimes send the SUSPENDED/SUSPENDED_PAUSED event after RESUMED/RESUMED_MIGRATED (when VM status is PAUSED when migration completes, see qemuMigrationFinish function). In this case self._dom is None because the function _waitForIncomingMigrationFinish didn't update it yet. Change-Id: I3b7ed13c6dc8d1d590449118cf167290f4630070 Signed-off-by: Gal Hammer <ghammer(a)redhat.com> --- M vdsm/libvirtvm.py 1 file changed, 7 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/01/6901/1 -- To view, visit http://gerrit.ovirt.org/6901 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3b7ed13c6dc8d1d590449118cf167290f4630070 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Gal Hammer <ghammer(a)redhat.com>

6 17

Change in vdsm[master]: BZ#840407 - Create a fake template when moving to backup SD.
by ewarszaw＠redhat.com 10 Feb '13

10 Feb '13

Eduardo has uploaded a new change for review. Change subject: BZ#840407 - Create a fake template when moving to backup SD. ...................................................................... BZ#840407 - Create a fake template when moving to backup SD. This code is repeated twice because the copy in deprecated moveMultipleImages() will be removed when this image will be removed in the next version. All the related fake template code should be removed. Change-Id: Ibbeef2480e03cc075b80880485739139e496d0b6 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 26 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/6519/1 -- To view, visit http://gerrit.ovirt.org/6519 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibbeef2480e03cc075b80880485739139e496d0b6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

6 25

Change in vdsm[master]: netinfo: report ipv6 information
by mhuntxu＠gmail.com 08 Feb '13

08 Feb '13

Hunt Xu has uploaded a new change for review. Change subject: netinfo: report ipv6 information ...................................................................... netinfo: report ipv6 information Change-Id: Iaf195eeeb06ee385099e0d9f149d467e897717b8 Signed-off-by: huntxu <mhuntxu(a)gmail.com> --- M vdsm/netinfo.py 1 file changed, 6 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/9382/1 diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index e0ee606..4a94907 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -341,6 +341,8 @@ 'addr': getaddr(devname), 'netmask': getnetmask(devname), 'gateway': routes.get(devname, '0.0.0.0'), + 'ipv6addrs': getipv6addrs(devname), + 'ipv6gateway': routes6.get(devname, '::'), 'mtu': getMtu(devname), }) except OSError, e: if e.errno == errno.ENOENT: @@ -359,6 +361,7 @@ 'stp': bridge_stp_state(bridge), 'addr': getaddr(bridge), 'netmask': getnetmask(bridge), + 'ipv6addrs': getipv6addrs(bridge), 'mtu': getMtu(bridge), 'cfg': getIfaceCfg(bridge), }) @@ -367,6 +370,7 @@ d['nics'] = dict([(nic, {'speed': speed(nic), 'addr': getaddr(nic), 'netmask': getnetmask(nic), + 'ipv6addrs': getipv6addrs(nic), 'hwaddr': gethwaddr(nic), 'mtu': getMtu(nic), 'cfg': getIfaceCfg(nic), @@ -379,6 +383,7 @@ d['bondings'] = dict([(bond, {'slaves': slaves(bond), 'addr': getaddr(bond), 'netmask': getnetmask(bond), + 'ipv6addrs': getipv6addrs(bond), 'hwaddr': gethwaddr(bond), 'cfg': getIfaceCfg(bond), 'mtu': getMtu(bond)}) @@ -386,6 +391,7 @@ d['vlans'] = dict([(vlan, {'iface': vlan.split('.')[0], 'addr': getaddr(vlan), 'netmask': getnetmask(vlan), + 'ipv6addrs': getipv6addrs(vlan), 'mtu': getMtu(vlan), 'cfg': getIfaceCfg(vlan), }) -- To view, visit http://gerrit.ovirt.org/9382 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iaf195eeeb06ee385099e0d9f149d467e897717b8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Hunt Xu <mhuntxu(a)gmail.com>

5 29

Change in vdsm[master]: migrateStatus() progress report
by peet＠redhat.com 07 Feb '13

07 Feb '13

Peter V. Saveliev has uploaded a new change for review. Change subject: migrateStatus() progress report ...................................................................... migrateStatus() progress report The percentage is based on libvirt dataTotal and dataProcessed fields of the virDomainJobInfo struct Signed-off-by: Peter V. Saveliev <peet(a)redhat.com> Change-Id: I3ff00e85c88e865cd81697d427d6bd5473e0f79e --- M vdsm/vm.py 1 file changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/6824/1 -- To view, visit http://gerrit.ovirt.org/6824 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3ff00e85c88e865cd81697d427d6bd5473e0f79e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Peter V. Saveliev <peet(a)redhat.com>

8 85

Change in vdsm[master]: Rename parent volume in blockSD.getAllVolumes.
by ewarszaw＠redhat.com 07 Feb '13

07 Feb '13

Eduardo has uploaded a new change for review. Change subject: Rename parent volume in blockSD.getAllVolumes. ...................................................................... Rename parent volume in blockSD.getAllVolumes. Change-Id: I62f1385e4af90a923c5d97a71571bea8780aa339 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/blockSD.py 1 file changed, 5 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/08/8608/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index e596fb6..d8f7824 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -159,14 +159,14 @@ for vName in vols.iterkeys(): res[vName] = {'imgs': [], 'parent': None} - for vName, vImg, vPar in vols.itervalues(): - res[vName]['parent'] = vPar + for vName, vImg, parentVol in vols.itervalues(): + res[vName]['parent'] = parentVol if vImg not in res[vName]['imgs']: res[vName]['imgs'].insert(0, vImg) - if vPar != sd.BLANK_UUID and \ + if parentVol != sd.BLANK_UUID and \ not vName.startswith(blockVolume.image.REMOVED_IMAGE_PREFIX) \ - and vImg not in res[vPar]['imgs']: - res[vPar]['imgs'].append(vImg) + and vImg not in res[parentVol]['imgs']: + res[parentVol]['imgs'].append(vImg) return dict((k, sd.ImgsPar(tuple(v['imgs']), v['parent'])) for k, v in res.iteritems()) -- To view, visit http://gerrit.ovirt.org/8608 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I62f1385e4af90a923c5d97a71571bea8780aa339 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

5 18

Change in vdsm[master]: BZ#829110 Remove a "Domain not found" error from stats threa...
by ghammer＠redhat.com 06 Feb '13

06 Feb '13

Gal Hammer has uploaded a new change for review. Change subject: BZ#829110 Remove a "Domain not found" error from stats thread. ...................................................................... BZ#829110 Remove a "Domain not found" error from stats thread. Removed redundant code which changed VM status if error occured on the stats thead. Change-Id: I1f648b9e3a70dca66cbf056c1f143bc6f0455e4a Signed-off-by: Gal Hammer <ghammer(a)redhat.com> --- M vdsm/libvirtvm.py 1 file changed, 0 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/7746/1 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 6323d0c..2f6e544 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -291,17 +291,6 @@ if ex.get_error_code() != libvirt.VIR_ERR_NO_DOMAIN: return False - # If a VM is down, hibernating, migrating, destroyed or in the - # process of being shutdown we were expecting it to disappear - if ((self._vm.lastStatus in ('Down', - 'Saving State', 'Migration Source')) - or self._vm.destroyed - or self._vm._guestEvent == 'Powering down'): - return True - - self._log.debug("VM not found, moving to Down", exc_info=True) - self._vm.setDownStatus(ERROR, str(ex)) - return True -- To view, visit http://gerrit.ovirt.org/7746 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1f648b9e3a70dca66cbf056c1f143bc6f0455e4a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Gal Hammer <ghammer(a)redhat.com>

7 9

Change in vdsm[master]: configNetwork: _updateConfigValue: Add a new line char '\n' ...
by Igor Lvovsky 06 Feb '13

06 Feb '13

Igor Lvovsky has uploaded a new change for review. Change subject: configNetwork: _updateConfigValue: Add a new line char '\n' at the end of the line ...................................................................... configNetwork: _updateConfigValue: Add a new line char '\n' at the end of the line This patch remove an empty line from the middle of iface config file when removing network with custom MTU. Example: --------- [root@zeus02 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth3 DEVICE=eth3 ONBOOT=yes HWADDR=78:ac:c0:11:79:79 NM_CONTROLLED=no STP=no MTU=9000[root@zeus02 ~]# Change-Id: I6030f0c92a2f3f2572b52cc48df6198da5cd2847 Signed-off-by: Igor Lvovsky <ilvovsky(a)redhat.com> --- M vdsm/configNetwork.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/7688/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 74d19c7..20f1136 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -558,7 +558,7 @@ if not line.startswith(entry + '=') ] if not delete: - entries.append('\n' + entry + '=' + value) + entries.append(entry + '=' + value + '\n') self._backup(conffile) with open(conffile, 'w') as f: -- To view, visit http://gerrit.ovirt.org/7688 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6030f0c92a2f3f2572b52cc48df6198da5cd2847 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Igor Lvovsky <ilvovsky(a)redhat.com>

3 6

Change in vdsm[master]: BZ#836161 - Remove preDeleteRename().
by ewarszaw＠redhat.com 06 Feb '13

06 Feb '13

Eduardo has uploaded a new change for review. Change subject: BZ#836161 - Remove preDeleteRename(). ...................................................................... BZ#836161 - Remove preDeleteRename(). Change-Id: I2452594c701eb9c3ccbab7de92e8fca69f033074 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/image.py M vdsm/storage/sp.py 2 files changed, 0 insertions(+), 35 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/08/8508/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 9a22863..5871c3a 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -214,37 +214,6 @@ randomStr = misc.randomStr(RENAME_RANDOM_STRING_LEN) return "%s%s_%s" % (REMOVED_IMAGE_PREFIX, randomStr, uuid) - def preDeleteRename(self, sdUUID, imgUUID): - # Get the list of the volumes - volclass = sdCache.produce(sdUUID).getVolumeClass() - uuidlist = volclass.getImageVolumes(self.repoPath, sdUUID, imgUUID) - imageDir = self.getImageDir(sdUUID, imgUUID) - - # If image directory doesn't exist we are done - if not os.path.exists(imageDir): - return imgUUID - - # Otherwise move it out of the way - newImgUUID = self.deletedVolumeName(imgUUID) - self.log.info("Rename image %s -> %s", imgUUID, newImgUUID) - if not imgUUID.startswith(REMOVED_IMAGE_PREFIX): - removedImage = os.path.join(os.path.dirname(imageDir), newImgUUID) - os.rename(imageDir, removedImage) - else: - self.log.warning("Image %s in domain %s already renamed", imgUUID, sdUUID) - - volumes = [volclass(self.repoPath, sdUUID, newImgUUID, volUUID) for volUUID in uuidlist] - for vol in volumes: - if not vol.volUUID.startswith(REMOVED_IMAGE_PREFIX): - vol.rename(self.deletedVolumeName(vol.volUUID), recovery=False) - else: - self.log.warning("Volume %s of image %s already renamed", vol.volUUID, imgUUID) - # We change image UUID in metadata - # (and IU_ LV tag for block volumes) of all volumes in image - vol.setImage(newImgUUID) - - return newImgUUID - def __chainSizeCalc(self, sdUUID, imgUUID, volUUID, size): """ Compute an estimate of the whole chain size diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 0ade40d..115f575 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1942,10 +1942,6 @@ except: self.log.warning("SP %s SD %s img %s Vol %s - teardown failed") - def preDeleteRename(self, sdUUID, imgUUID): - repoPath = os.path.join(self.storage_repository, self.spUUID) - return image.Image(repoPath).preDeleteRename(sdUUID, imgUUID) - def validateVolumeChain(self, sdUUID, imgUUID): repoPath = os.path.join(self.storage_repository, self.spUUID) image.Image(repoPath).validateVolumeChain(sdUUID, imgUUID) -- To view, visit http://gerrit.ovirt.org/8508 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2452594c701eb9c3ccbab7de92e8fca69f033074 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

4 10

Change in vdsm[master]: netinfo: implement functions gathering ipv6 information
by mhuntxu＠gmail.com 05 Feb '13

05 Feb '13

Hunt Xu has uploaded a new change for review. Change subject: netinfo: implement functions gathering ipv6 information ...................................................................... netinfo: implement functions gathering ipv6 information Change-Id: I2318ffa7d71abe3f57cb5e480bc46565e29f3894 Signed-off-by: huntxu <mhuntxu(a)gmail.com> --- M vdsm/netinfo.py 1 file changed, 34 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/81/9381/1 diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index b6d1a1f..e0ee606 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -22,6 +22,8 @@ import errno import glob import ethtool +import socket +import struct import shlex import logging from fnmatch import fnmatch @@ -178,6 +180,13 @@ return bitmask_to_address(netmask) +def getipv6addrs(dev): + """Return a list of ipv6 addresses in the format of 'address/prefixlen'.""" + dev_info_list = ethtool.get_interfaces_info(dev.encode('utf8')) + ipv6addrs = dev_info_list[0].get_ipv6_addresses() + return ['/'.join([addr.address, str(addr.netmask)]) for addr in ipv6addrs] + + def gethwaddr(dev): return file('/sys/class/net/%s/address' % dev).read().strip() @@ -223,10 +232,15 @@ return '.'.join(ip_address) +def ipv6StrToAddress(ipv6_str): + chr_list = [chr(int(ipv6_str[x:x + 2], 16)) for x in xrange(0, 32, 2)] + return socket.inet_ntop(socket.AF_INET6, struct.pack('16c', *chr_list)) + + def getRoutes(): """Return the interface default gateway or None if not found.""" - gateways = dict() + gateways = ipv6gateways = dict() with open("/proc/net/route") as route_file: route_file.readline() # skip header line @@ -238,7 +252,24 @@ ip_num = int(route_parm[2], 16) gateways[route_parm[0]] = intToAddress(ip_num) - return gateways + try: + with open("/proc/net/ipv6_route") as route_file: + for route_line in route_file.xreadlines(): + route_parm = route_line.rstrip().split(' ') + dest = route_parm[0] + prefix = route_parm[1] + nexthop = route_parm[4] + device = route_parm[-1] + if dest == '0' * 32 and prefix == '00' and nexthop != '0' * 32: + ipv6gateways[device] = ipv6StrToAddress(nexthop) + except IOError as e: + if e.errno == 2: + # ipv6 module not loaded + pass + else: + raise e + + return gateways, ipv6gateways def getIfaceCfg(iface): @@ -273,7 +304,7 @@ def get(): d = {} - routes = getRoutes() + routes, routes6 = getRoutes() # FIXME handle bridge/nic missing from ifconfig d['networks'] = {} nets = networks() -- To view, visit http://gerrit.ovirt.org/9381 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2318ffa7d71abe3f57cb5e480bc46565e29f3894 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Hunt Xu <mhuntxu(a)gmail.com>

6 26

Change in vdsm[master]: Handle spaces in device names using lvm filter
by ykaplan＠redhat.com 31 Jan '13

31 Jan '13

Yeela Kaplan has uploaded a new change for review. Change subject: Handle spaces in device names using lvm filter ...................................................................... Handle spaces in device names using lvm filter Change-Id: I1efb6dfd30c0999f14c413fff778213b247a178f Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=819745 Signed-off-by: Yeela Kaplan <ykaplan(a)redhat.com> --- M vdsm/storage/lvm.py 1 file changed, 8 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/91/9191/1 diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index e846e38..16c2c44 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -130,11 +130,16 @@ def _buildFilter(devList): devList = list(devList) devList.sort() - filt = '|'.join(dev.strip() for dev in devList if dev.strip()) + devs = [] + for dev in devList: + if dev.strip(): + devs.append('\\\\\\\\x20'.join(dev.strip(). + encode("string_escape").split())) + filt = '|'.join(devs) if len(filt) > 0: - filt = '"a%' + filt + '%", ' + filt = "'a%" + filt + "%', " - filt = 'filter = [ ' + filt + '"r%.*%" ]' + filt = "filter = [ " + filt + "'r%.*%' ]" return filt -- To view, visit http://gerrit.ovirt.org/9191 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1efb6dfd30c0999f14c413fff778213b247a178f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yeela Kaplan <ykaplan(a)redhat.com>

7 38

Change in vdsm[master]: enable task tag as list of strings
by barumuga＠redhat.com 29 Jan '13

29 Jan '13

Hello Ayal Baron, Timothy Asir, Saggi Mizrahi, Federico Simoncelli, Dan Kenigsberg, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/8899 to review the following change. Change subject: enable task tag as list of strings ...................................................................... enable task tag as list of strings Multiple tags are stored as space-separated string and search is done an in-string search instead of a proper set membership. This approach enables tag as list of strings and supports backward compatibility Change-Id: Ib3c6f8cba17df7f545049eabe2b8f38318c5db7f Signed-off-by: Bala.FA <barumuga(a)redhat.com> --- M vdsm/storage/task.py 1 file changed, 14 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/99/8899/1 diff --git a/vdsm/storage/task.py b/vdsm/storage/task.py index a596494..6c79c85 100644 --- a/vdsm/storage/task.py +++ b/vdsm/storage/task.py @@ -49,6 +49,7 @@ import logging import threading import types +import ast import storage_exception as se import uuid @@ -411,13 +412,22 @@ high = "high" +class TagList(list): + def __init__(self, s): + if s.startswith('[') and s.endswith(']'): + l = ast.literal_eval(s) + else: + l = s.split(' ') + list.__init__(self, l) + + class Task: # External Task info fields = { # field_name: type "id": str, "name": unicode, - "tag": unicode, + "tag": TagList, "store": unicode, "recoveryPolicy": TaskRecoveryType, "persistPolicy": TaskPersistType, @@ -448,7 +458,7 @@ self.callbackLock = threading.Lock() self.id = str(id) self.name = name - self.tag = tag + self.tag = TagList(tag) self.priority = priority self.recoveryPolicy = recovery self.persistPolicy = TaskPersistType.none @@ -972,7 +982,7 @@ def setTag(self, tag): if KEY_SEPERATOR in tag: raise ValueError("tag cannot include %s character" % KEY_SEPERATOR) - self.tag = unicode(tag) + self.tag = TagList(unicode(tag)) def isDone(self): return self.state.isDone() @@ -1257,7 +1267,7 @@ "code": self.result.code, "message": self.result.message, "result": self.result.result, - "tag": self.tag + "tag": str(self.tag) } def getID(self): -- To view, visit http://gerrit.ovirt.org/8899 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib3c6f8cba17df7f545049eabe2b8f38318c5db7f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Bala.FA <barumuga(a)redhat.com> Gerrit-Reviewer: Ayal Baron <abaron(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Federico Simoncelli <fsimonce(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com> Gerrit-Reviewer: Timothy Asir <tjeyasin(a)redhat.com>

3 5

Change in vdsm[master]: Expose a custom property for tunnelled migration of a VM
by Dan Kenigsberg 28 Jan '13

28 Jan '13

Dan Kenigsberg has uploaded a new change for review. Change subject: Expose a custom property for tunnelled migration of a VM ...................................................................... Expose a custom property for tunnelled migration of a VM Change-Id: Id713b455db4f9bf437720b64c92bba9377cd89ae --- M vdsm/libvirtvm.py 1 file changed, 9 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/51/2551/1 -- To view, visit http://gerrit.ovirt.org/2551 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id713b455db4f9bf437720b64c92bba9377cd89ae Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

8 23

Change in vdsm[master]: Add lvm.lvPhysDev() to get the physical LV device.
by ewarszaw＠redhat.com 27 Jan '13

27 Jan '13

Eduardo has uploaded a new change for review. Change subject: Add lvm.lvPhysDev() to get the physical LV device. ...................................................................... Add lvm.lvPhysDev() to get the physical LV device. Change-Id: I6d3286589ea02025713d10d85a9892024f586b73 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/lvm.py 1 file changed, 10 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/51/8751/1 diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index 78c846a..094569d 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -1156,6 +1156,16 @@ def lvPath(vgName, lvName): return os.path.join("/dev", vgName, lvName) +def lvPhysDev(vgName, lvName): + """Return the LV physical device. + + returns: dm-X + The LV should be active. + """ + + lvp = lvPath(vgName, lvName) + return os.path.basename(os.readlink(lvp)) + def _isLVActive(vgName, lvName): """Active volumes have a mp link. -- To view, visit http://gerrit.ovirt.org/8751 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6d3286589ea02025713d10d85a9892024f586b73 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

4 7

Change in vdsm[master]: [RFC] Support for GLUSTERFS_DOMAIN
by deepakcs＠linux.vnet.ibm.com 24 Jan '13

24 Jan '13

Deepak C Shetty has uploaded a new change for review. Change subject: [RFC] Support for GLUSTERFS_DOMAIN ...................................................................... [RFC] Support for GLUSTERFS_DOMAIN This patch introduces a new storage domain of type GLUSTERFS_DOMAIN, which uses gluster as the storage backend. In GLUSTERFS_DOMAIN, vdsm creates the storage domain by mounting the gluster volume (akin to nfs mounting export path). VMs created using this domain exploit the QEMU's gluster block backend. Instead of accessing the vmdisk as a file path, it accesses the vmdisk as a network disk device, served by gluster server/volume. This patch attempts to re-use nfsSD core logic to support domain of type GLUSTERFS_DOMAIN. Also has some changes to pass pep8 checks Change-Id: I9ac37da88625f20d148beaf53bb6371c15b33ad7 Signed-off-by: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com> --- M vdsm.spec.in M vdsm/API.py M vdsm/BindingXMLRPC.py M vdsm/libvirtvm.py M vdsm/storage/Makefile.am M vdsm/storage/fileSD.py A vdsm/storage/glusterVolume.py M vdsm/storage/hsm.py M vdsm/storage/nfsSD.py M vdsm/storage/sd.py M vdsm/storage/volume.py M vdsm/vm.py 12 files changed, 124 insertions(+), 34 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/6856/1 -- To view, visit http://gerrit.ovirt.org/6856 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9ac37da88625f20d148beaf53bb6371c15b33ad7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com>

10 92

Change in vdsm[master]: [WIP] BZ#748386 - qemuConvert test cases
by lyarwood＠redhat.com 22 Jan '13

22 Jan '13

Lee Yarwood has uploaded a new change for review. Change subject: [WIP] BZ#748386 - qemuConvert test cases ...................................................................... [WIP] BZ#748386 - qemuConvert test cases Adds some simple qemu-img tests against volume.qemuConvert() to ensure volumes remain the correct format after being converted. Change-Id: I1c9f302928c246f7ce195d169f8d5a533df1a344 Signed-off-by: Lee Yarwood <lyarwood(a)redhat.com> --- A tests/qemuImgTests.py A vdsm/storage/fakeVolume.py 2 files changed, 214 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/90/9290/1 diff --git a/tests/qemuImgTests.py b/tests/qemuImgTests.py new file mode 100644 index 0000000..e1cc70c --- /dev/null +++ b/tests/qemuImgTests.py @@ -0,0 +1,148 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +from testrunner import VdsmTestCase as TestCase + +import tempfile +import uuid +import os + + +from storage.threadLocal import vars +from storage.fakeVolume import FakeFileVolume +import storage.task as task +import storage.volume as volume +import storage.misc as misc + +EXT_QEMUIMG = '/bin/qemu-img' +EXT_DD = '/bin/dd' +VOL_SIZE = "10M" +VOL_ALLOC = "20" + +class FileVolumes(): + + def __init__(self): + self.dir = tempfile.mkdtemp() + self.sduuid = str(uuid.uuid4()) + self.imguuid = str(uuid.uuid4()) + self.repo = self.dir + "/" + self.sduuid + "/images/" + self.imguuid + os.makedirs(self.repo) + + def genPathUUID(self): + u = str(uuid.uuid4()) + p = self.repo + "/" + u + return p, u + + + def createCOW(self): + path, voluuid = self.genPathUUID() + misc.execCmd([EXT_QEMUIMG, "create", "-f", "qcow2", path, VOL_SIZE],sync=False) + return FakeFileVolume(repoPath=self.dir, sdUUID=self.sduuid, imgUUID=self.imguuid, volUUID=voluuid, + volFormat=volume.COW_FORMAT, volPath=path) + + def createRAW(self): + path, voluuid = self.genPathUUID() + misc.execCmd([EXT_QEMUIMG, "create", "-f", "raw", path, VOL_SIZE],sync=False) + return FakeFileVolume(repoPath=self.dir, sdUUID=self.sduuid, imgUUID=self.imguuid, volUUID=voluuid, + volFormat=volume.RAW_FORMAT, volPath=path) + + def createCOWPrealloc(self): + v = self.createCOW() + misc.execCmd([EXT_DD, "if=/dev/zero", "of=%s" % v.getVolumePath(), "bs=512"]) + return v + + + def createCOWChild(self,parent): + path, voluuid = self.genPathUUID() + misc.execCmd([EXT_QEMUIMG, "create", "-f", "qcow2", "-o", "backing_file=%s/%s,backing_fmt=raw" % + (self.repo,parent.volUUID), self.repo + "/" + voluuid, VOL_SIZE],sync=False) + fake = FakeFileVolume(repoPath=self.dir, sdUUID=self.sduuid, imgUUID=self.imguuid, volUUID=voluuid, + volFormat=volume.COW_FORMAT, volPath=path) + # Hack to avoid a call to sdc.produce() later when we try to find the parent. + fake.setParent(parent.volUUID) + fake.setParentVolume(parent) + return fake + + def cleanUp(self): + os.rmdir(self.dir) + + +class qemuImgRebaseTestCase(TestCase): + """ + + """ + +class qemuImgCreateTestCase(TestCase): + """ + + """ + +class qemuImgConvertTestCase(TestCase): + + + def runQemuConvert(self, src, dst): + + vars.task = task.Task(id=None, name='qemuImgTests') + try: + (rc, out, err) = volume.qemuConvert(src, dst, vars.task.aborting) + self.assertFalse(rc) + except Exception: + self.fail() + + + def testFileQemuConvert(self): + # create the repo + t = FileVolumes() + + # RAW to RAW + src = t.createRAW() + dst = t.createRAW() + self.runQemuConvert(src, dst) + + # QCOW2 to RAW + #src = t.createCOW() + #dst = t.createRAW() + #self.runQemuConvert(src, dst) + + # RAW to QCOW2 + #src = t.createRAW() + #dst = t.createCOW() + #self.runQemuConvert(src, dst) + + # RAW<-QCOW2 to RAW<-QCOW2 + parent = t.createRAW() + src = t.createCOWChild(parent) + dst = t.createCOW() + self.runQemuConvert(src, dst) + + # Convert a COW parent child to another COW + # RAW<-QCOW2<-QCOW2 to RAW<-QCOW2<-QCOW2 + rawParent = t.createRAW() + src = t.createCOWChild(rawParent) + cowChild = t.createCOWChild(src) + dst = t.createCOW() + self.runQemuConvert(src,dst) + + + + + def testBlockQemuConvert(self): + pass + diff --git a/vdsm/storage/fakeVolume.py b/vdsm/storage/fakeVolume.py new file mode 100644 index 0000000..72516b7 --- /dev/null +++ b/vdsm/storage/fakeVolume.py @@ -0,0 +1,66 @@ +# +# Copyright 2012 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import volume +import misc + +class FakeFileVolume(volume.Volume): + """ + A fake file volume class to aid with testing. + """ + + def __init__(self, repoPath, sdUUID, imgUUID, volUUID, volFormat, volPath): + volume.Volume.__init__(self, repoPath, sdUUID, imgUUID, volUUID) + + self.md = {"SIZE": None,"TYPE":None,"FORMAT":None,"DISKTYPE":None,"VOLTYPE":None, + "PUUID":None,"DOMAIN":None,"CTIME":None,"IMAGE":None,"DESCRIPTION":None, + "LEGALITY":None,"MTIME":None,"ILLEGAL":None,"LEGAL":None,"FAKE":None} + + self.setFormat(volFormat) + self.setVolumePath(volPath) + self.parentVolume = None + + def validateImagePath(self): + pass + + def validateVolumePath(self): + pass + + def setVolumePath(self,volPath): + self.volumePath = volPath + + def setParent(self, puuid): + self.setMetaParam(volume.PUUID, puuid) + + def getParent(self): + return self.getMetaParam(volume.PUUID) + + def setParentVolume(self,parent): + self.parentVolume = parent + + def getParentVolume(self): + return self.parentVolume + + def setMetadata(self, m, mid=None): + for k,v in m.iteritems(): + self.md[k] = v + + def getMetadata(self): + return self.md -- To view, visit http://gerrit.ovirt.org/9290 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1c9f302928c246f7ce195d169f8d5a533df1a344 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Lee Yarwood <lyarwood(a)redhat.com>

2 8

Change in vdsm[master]: [WIP] BZ#748386 - refactor qemuConvert to only use qemu-img ...
by lyarwood＠redhat.com 22 Jan '13

22 Jan '13

Lee Yarwood has uploaded a new change for review. Change subject: [WIP] BZ#748386 - refactor qemuConvert to only use qemu-img convert. ...................................................................... [WIP] BZ#748386 - refactor qemuConvert to only use qemu-img convert. This change refactors the qemuConvert method, dropping the use of dd with RAW volumes in favour of qemu-img convert thus preserving sparseness. The change also introduces the use of the backing_file and backing_fmt options when converting COW volumes. Change-Id: I7adc41e8a9f5d4b9faa58a8cf5bec685b66e303e Signed-off-by: Lee Yarwood <lyarwood(a)redhat.com> --- M vdsm/storage/image.py M vdsm/storage/volume.py 2 files changed, 25 insertions(+), 38 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/28/8728/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index e86d94c..69ab6b6 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -788,19 +788,9 @@ try: # Start the actual copy image procedure - srcVol.prepare(rw=False) + srcVol.prepare(rw=True, chainrw=True, setrw=True) dstVol.prepare(rw=True, setrw=True) - - try: - (rc, out, err) = volume.qemuConvert(volParams['path'], dstPath, - volParams['volFormat'], dstVolFormat, vars.task.aborting, - size=srcVol.getVolumeSize(bs=1), dstvolType=dstVol.getType()) - if rc: - raise se.StorageException("rc: %s, err: %s" % (rc, err)) - except se.ActionStopped, e: - raise e - except se.StorageException, e: - raise se.CopyImageError(str(e)) + (rc, out, err) = volume.qemuConvert(srcVol, dstVol, self.idle, vars.task.aborting) # Mark volume as SHARED if volType == volume.SHARED_VOL: @@ -997,12 +987,12 @@ # Step 2: Convert successor to new volume # qemu-img convert -f qcow2 successor -O raw newUUID - (rc, out, err) = volume.qemuConvert(srcVolParams['path'], newVol.getVolumePath(), - srcVolParams['volFormat'], volParams['volFormat'], vars.task.aborting, - size=volParams['apparentsize'], dstvolType=newVol.getType()) - if rc: - self.log.error("qemu-img convert failed: rc=%s, out=%s, err=%s", - rc, out, err) + try: + (rc, out, err) = volume.qemuConvert(srcVol, newVol, self.idle, vars.task.aborting) + except se.ActionStopped: + raise + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) raise se.MergeSnapshotsError(newUUID) newVol.teardown(sdUUID=newVol.sdUUID, volUUID=newVol.volUUID) diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py index b9eb356..875b59a 100644 --- a/vdsm/storage/volume.py +++ b/vdsm/storage/volume.py @@ -960,28 +960,25 @@ log.debug('(qemuRebase): REBASE %s DONE' % (src)) return (rc, out, err) +def qemuConvert(src, dst, idle, stop): + """ + Copy the src image into the new dst. + """ + cmd = constants.CMD_LOWPRIO + [constants.EXT_QEMUIMG, "convert", "-t", "none", + "-f", fmt2str(src.getFormat()), "-O", fmt2str(dst.getFormat())] -def qemuConvert(src, dst, src_fmt, dst_fmt, stop, size, dstvolType): - """ - Convert the 'src' image (or chain of images) into a new single 'dst' - """ - src_fmt = fmt2str(src_fmt) - dst_fmt = fmt2str(dst_fmt) - log.debug('(qemuConvert): COPY %s (%s) to %s (%s) START' % - (src, src_fmt, dst, dst_fmt)) + # backing_file only used when src has a parent and both volumes are COW. + if src.getParentVolume() and src.getFormat() == COW_FORMAT and dst.getFormat() == COW_FORMAT : + cmd += ["-o", "backing_file=%s,backing_fmt=%s" % (src.getParentVolume().getVolumePath(), + fmt2str(src.getParentVolume().getFormat()))] - if (src_fmt == "raw" and dst_fmt == "raw" and - dstvolType == PREALLOCATED_VOL): - (rc, out, err) = misc.ddWatchCopy( - src=src, dst=dst, - stop=stop, size=size, - recoveryCallback=baseAsyncTasksRollback) - else: - cmd = constants.CMD_LOWPRIO + [constants.EXT_QEMUIMG, "convert", - "-t", "none", "-f", src_fmt, src, - "-O", dst_fmt, dst] - (rc, out, err) = misc.watchCmd(cmd, stop=stop, + cmd += [src.getVolumePath(), dst.getVolumePath()] + + (rc, out, err) = misc.watchCmd(cmd, stop=stop, idle=idle, sudo=False, recoveryCallback=baseAsyncTasksRollback) - log.debug('(qemuConvert): COPY %s to %s DONE' % (src, dst)) + if rc: + raise se.StorageException + return (rc, out, err) + -- To view, visit http://gerrit.ovirt.org/8728 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7adc41e8a9f5d4b9faa58a8cf5bec685b66e303e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Lee Yarwood <lyarwood(a)redhat.com>

5 14

Change in vdsm[master]: Extract bonding options building into a separate function.
by wudxw＠linux.vnet.ibm.com 20 Jan '13

20 Jan '13

Mark Wu has uploaded a new change for review. Change subject: Extract bonding options building into a separate function. ...................................................................... Extract bonding options building into a separate function. To make setupNetworks shorter, move the bonding options building code into a separate function. Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> Change-Id: I77fefefcefa05f5bd0d7fa2755357d88b7aa615e --- M vdsm/configNetwork.py 1 file changed, 30 insertions(+), 27 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/85/8885/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 3940446..6de38e3 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -1249,6 +1249,30 @@ del bondings[bond] +def _buildBondOptions(bondName, bondings): + logger = logging.getLogger("_buildBondOptions") + + # We need to use the newest host info + _ni = netinfo.NetInfo() + + bond = {} + if bondings.get(bondName): + bond['nics'] = bondings[bondName]['nics'] + bond['bondingOptions'] = bondings[bondName].get('options', None) + elif bondName in _ni.bondings: + # We may not receive any information about the bonding device if it is + # unchanged. In this case check whether this bond exists on host and + # take its parameters. + logger.debug("Fetching bond %r info", bondName) + existingBond = _ni.bondings[bondName] + bond['nics'] = existingBond['slaves'] + bond['bondingOptions'] = existingBond['cfg'].get('BONDING_OPTS', None) + else: + raise ConfigNetworkError(ne.ERR_BAD_PARAMS, "No given bonding option, \ + nor existing bond %s found." % bondName) + return bond + + def setupNetworks(networks={}, bondings={}, **options): """Add/Edit/Remove configuration for networks and bondings. @@ -1292,12 +1316,6 @@ _netinfo = netinfo.NetInfo() configWriter = ConfigWriter() networksAdded = set() - # keep set netsWithNewBonds to be able remove - # a new added network if connectivity check fail. - # If a new network needs to be created on top of existing bond, - # we will need to keep the bond on rollback flow, - # else we will break the new created bond. - netsWithNewBonds = set() logger.debug("Setting up network according to configuration: " "networks:%r, bondings:%r, options:%r" % (networks, @@ -1327,28 +1345,10 @@ # Check whether bonds should be resized _editBondings(bondings, configWriter) - # We need to use the newest host info - _ni = netinfo.NetInfo() for network, networkAttrs in networks.iteritems(): d = dict(networkAttrs) if 'bonding' in d: - # we may not receive any information - # about the bonding device if it is unchanged - # In this case check whether this bond exists - # on host and take its parameters - if bondings.get(d['bonding']): - d['nics'] = bondings[d['bonding']]['nics'] - d['bondingOptions'] = \ - bondings[d['bonding']].get('options', None) - # we create a new bond - if network in networksAdded: - netsWithNewBonds.add(network) - elif d['bonding'] in _ni.bondings: - logger.debug("Updating bond %r info", d['bonding']) - d['nics'] = _ni.bondings[d['bonding']]['slaves'] - d['bondingOptions'] = \ - _ni.bondings[d['bonding']]['cfg'].get( - 'BONDING_OPTS', None) + d.update(_buildBondOptions(d['bonding'], bondings)) else: d['nics'] = [d.pop('nic')] d['force'] = force @@ -1363,8 +1363,11 @@ CONNECTIVITY_TIMEOUT_DEFAULT))): logger.info('Connectivity check failed, rolling back') for network in networksAdded: - delNetwork(network, force=True, - implicitBonding=network in netsWithNewBonds) + # If the new added network was created on top of + # existing bond, we need to keep the bond on rollback + # flow, else we will break the new created bond. + isNew = networkAttrs['bonding'] in bondings + delNetwork(network, force=True, implicitBonding=isNew) raise ConfigNetworkError(ne.ERR_LOST_CONNECTION, 'connectivity check failed') except: -- To view, visit http://gerrit.ovirt.org/8885 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I77fefefcefa05f5bd0d7fa2755357d88b7aa615e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

4 47

Change in vdsm[master]: Add a flag to libvirtvm's status to report watchdog event
by shaohef＠linux.vnet.ibm.com 17 Jan '13

17 Jan '13

ShaoHe Feng has uploaded a new change for review. Change subject: Add a flag to libvirtvm's status to report watchdog event ...................................................................... Add a flag to libvirtvm's status to report watchdog event This flag will be set once watchdog event is triggered. And thie flag will be not cleanup after it is set unless the client restart or shutdown VM. So add a timestamp in the message of this flag. The timestamp is the time in seconds since the Epoch. When the client poll this flag, it should record and check the timestamp to make sure that the event is already triggered. Change-Id: I7aa676299c748d761e216469569156140a79af65 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/9429/1 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 3439dc3..15d3187 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -2225,6 +2225,8 @@ except IndexError: return "Received unknown watchdog action(%s)" % action + self.watchdogEvent = "Time: %s, Action: %s" % (time.time(), + actionToString(action)) self.log.debug("Watchdog event comes from guest %s. " "Action: %s", self.conf['vmName'], actionToString(action)) @@ -2449,6 +2451,8 @@ def getStats(self): stats = vm.Vm.getStats(self) stats['hash'] = self._devXmlHash + if hasattr(self, "watchdogEvent"): + stats["watchdogEvent"] = self.watchdogEvent return stats def _getBalloonInfo(self): -- To view, visit http://gerrit.ovirt.org/9429 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7aa676299c748d761e216469569156140a79af65 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

5 30

Change in vdsm[master]: fix two parameters of VmParameters in vdsmapi schema
by shaohef＠linux.vnet.ibm.com 17 Jan '13

17 Jan '13

ShaoHe Feng has uploaded a new change for review. Change subject: fix two parameters of VmParameters in vdsmapi schema ...................................................................... fix two parameters of VmParameters in vdsmapi schema from the comments of VmParameters, the two parameters are #optional Change-Id: Id9bce789c1d33357e079ff47300a99a59025834e Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/9112/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index d210c7d..6fa45de 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -2382,8 +2382,8 @@ 'data': {'acpiEnable': 'bool', '*custom': 'StringMap', '*devices': ['VmDevice'], 'display': 'VmDisplayType', 'kvmEnable': 'bool', 'memSize': 'uint', - 'nice': 'int', 'smp': 'uint', 'smpCoresPerSocket': 'uint', - 'smpThreadsPerCore': 'uint', 'timeOffset': 'uint', + 'nice': 'int', 'smp': 'uint', '*smpCoresPerSocket': 'uint', + '*smpThreadsPerCore': 'uint', 'timeOffset': 'uint', 'transparentHugePages': 'bool', 'vmId': 'UUID', 'vmName': 'str', 'vmType': 'VmType'}} -- To view, visit http://gerrit.ovirt.org/9112 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id9bce789c1d33357e079ff47300a99a59025834e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

5 11

Change in vdsm[master]: Changing threadName when using multiprocessing package
by ybronhei＠redhat.com 17 Jan '13

17 Jan '13

Yaniv Bronhaim has uploaded a new change for review. Change subject: Changing threadName when using multiprocessing package ...................................................................... Changing threadName when using multiprocessing package multiprocessing.manager adds current process name in front of current threadName. This patch changes the name back to Thread-[No]. Change-Id: I541beb63fd5e1a2f7d6eed7f66f4970bf37a6705 Bug-Id: https://bugzilla.redhat.com/show_bug.cgi?id=868663 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/supervdsmServer.py 1 file changed, 3 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/37/9037/1 diff --git a/vdsm/supervdsmServer.py b/vdsm/supervdsmServer.py index a1bf8e6..a5bfc68 100755 --- a/vdsm/supervdsmServer.py +++ b/vdsm/supervdsmServer.py @@ -74,6 +74,9 @@ def wrapper(*args, **kwargs): try: + threadName = threading.current_thread().name + if threadName.find('|') != -1: + _, threading.current_thread().name = threadName.split('|') return func(*args, **kwargs) except: callbackLogger.error("Error in %s", func.__name__, exc_info=True) -- To view, visit http://gerrit.ovirt.org/9037 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I541beb63fd5e1a2f7d6eed7f66f4970bf37a6705 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

5 13

Change in vdsm[master]: Adding new resourceFactory that only opens fd
by ybronhei＠redhat.com 17 Jan '13

17 Jan '13

Yaniv Bronhaim has uploaded a new change for review. Change subject: Adding new resourceFactory that only opens fd ...................................................................... Adding new resourceFactory that only opens fd Avoiding activate every time we try to access lv and only open fd instead. This replace the refCounter and the call to lvchange anytime we prepare volume. Change-Id: Ic936c8959562a262de5ceba8ca4f68bc53b5cae2 Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/storage/blockVolume.py M vdsm/storage/resourceFactories.py 2 files changed, 51 insertions(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/84/8084/1 diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index 38285c1..2324ee7 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -35,7 +35,7 @@ import resourceManager as rm from threadLocal import vars from sdc import sdCache -from resourceFactories import LVM_ACTIVATION_NAMESPACE +import resourceFactories as rf import fileUtils TAG_PREFIX_MD = "MD_" @@ -75,8 +75,22 @@ def __init__(self, repoPath, sdUUID, imgUUID, volUUID): self.metaoff = None volume.Volume.__init__(self, repoPath, sdUUID, imgUUID, volUUID) - self.lvmActivationNamespace = sd.getNamespace(self.sdUUID, - LVM_ACTIVATION_NAMESPACE) + self._registerResourceNamespaces(sdUUID,volUUID) + + def _registerResourceNamespaces(self, sdUUID, volUUID): + """ + Register resources namespaces and create + factories for it. + """ + rmanager = rm.ResourceManager.getInstance() + # Register lvm activation resource namespace for the underlying VG + volWatcherFactory = rf.VolumeWatcherFactory(sdUUID, volUUID) + self.volWatcherNamespace = rf.VOLUME_WATCHER_NAMESPACE + try: + rmanager.registerNamespace(self.volWatcherNamespace, volWatcherFactory) + except Exception: + self.log.warn("Resource namespace %s already registered", self.volWatcherNamespace) + def validate(self): try: @@ -356,8 +370,14 @@ if setrw: self.setrw(rw=rw) access = rm.LockType.exclusive if rw else rm.LockType.shared - activation = rmanager.acquireResource(self.lvmActivationNamespace, + + # active volume + self.activateVolumes(self.volUUID) + + # acquireResource - open file + activation = rmanager.acquireResource(self.volWatcherNamespace, self.volUUID, access) + activation.autoRelease = False @classmethod @@ -369,9 +389,10 @@ """ cls.log.info("Tearing down volume %s/%s justme %s" % (sdUUID, volUUID, justme)) - lvmActivationNamespace = sd.getNamespace(sdUUID, - LVM_ACTIVATION_NAMESPACE) - rmanager.releaseResource(lvmActivationNamespace, volUUID) + + cls.deactivateVolumes(volUUID) + + rmanager.releaseResource(cls.volWatcherNamespace, volUUID) if not justme: try: pvolUUID = _getVolumeTag(sdUUID, volUUID, TAG_PREFIX_PARENT) diff --git a/vdsm/storage/resourceFactories.py b/vdsm/storage/resourceFactories.py index 02aba0f..8893f70 100644 --- a/vdsm/storage/resourceFactories.py +++ b/vdsm/storage/resourceFactories.py @@ -31,11 +31,34 @@ LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' IMAGE_NAMESPACE = 'imageNS' VOLUME_NAMESPACE = 'volumeNS' +VOLUME_WATCHER_NAMESPACE = 'volWatchNS' rmanager = rm.ResourceManager.getInstance() log = logging.getLogger('Storage.ResourcesFactories') +class VolumeWatcher(object): + def __init__(self, vgName, lvName): + self._path = lvm.lvPath(vgName, lvName) + try: + self._f = open(self._path) + except IOError: + log.error("Cannot open device %s", self._path) + + def close(self): + try: + self._f.close() + except: + log.error("error closing device %s", self._path) + +class VolumeWatcherFactory(rm.SimpleResourceFactory): + def __init__(self, vgName, lvName): + rm.SimpleResourceFactory.__init__(self) + self._lv = lvName + self._vg = vgName + + def createResource(self): + return VolumeWatcher(self._lv, self._vg) class LvmActivation(object): """ -- To view, visit http://gerrit.ovirt.org/8084 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic936c8959562a262de5ceba8ca4f68bc53b5cae2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

4 8

Change in vdsm[master]: prepare volume instead of prepare blockSD
by ybronhei＠redhat.com 17 Jan '13

17 Jan '13

Yaniv Bronhaim has uploaded a new change for review. Change subject: prepare volume instead of prepare blockSD ...................................................................... prepare volume instead of prepare blockSD Bug-Id: https://bugzilla.redhat.com/show_bug.cgi?id=851837 change the call of activateVolume to llprepare volume, and teardown instead of deactiveVolume, which sperates lv syscall operation to only verifies that the lv is in used as part of the volume. If the sd is in use by the checked volume, we want to keep ref counter to sign this volume, and when we teardown it, we verify that there are no references to the same sd before deactivate it. Change-Id: I25b9c260a7de5f883420a5322f90195b3379e80e Signed-off-by: Yaniv Bronhaim <ybronhei(a)redhat.com> --- M vdsm/storage/image.py 1 file changed, 2 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/50/8050/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index d004046..97112dc 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -383,8 +383,7 @@ chain.insert(0, tmplVolume) # Activating the volumes - sdCache.produce(sdUUID).activateVolumes( - volUUIDs=[vol.volUUID for vol in chain]) + sdCache.produce(sdUUID).llPrepare() return chain @@ -392,11 +391,8 @@ chain = self.getChain(sdUUID, imgUUID, volUUID) # Deactivating the volumes - sdCache.produce(sdUUID).deactivateVolumes( - volUUIDs=[vol.volUUID for vol in chain]) - # Do not deactivate the template yet (might be in use by an other vm) - # TODO: reference counting to deactivate when unused + sdCache.produce(sdUUID).teardown(sdUUID, volUUIDs=[vol.volUUID for vol in chain]) def __templateRelink(self, destDom, imgUUID, volUUID): """ -- To view, visit http://gerrit.ovirt.org/8050 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I25b9c260a7de5f883420a5322f90195b3379e80e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Yaniv Bronhaim <ybronhei(a)redhat.com>

5 7

Change in vdsm[master]: Unify produceVolume
by deepakcs＠linux.vnet.ibm.com 13 Jan '13

13 Jan '13

Deepak C Shetty has uploaded a new change for review. Change subject: Unify produceVolume ...................................................................... Unify produceVolume Unify produceVolume function and make it use getVolumeClass so that subclasses can override getVolumeClass to specify their own Volume and ensure that produceVolume works properly with subclass supplying their own volumeclass. Change-Id: I367582b2741dc0276ec64d8d13fa1e4953f0314b Signed-off-by: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com> --- M vdsm/storage/blockSD.py M vdsm/storage/fileSD.py M vdsm/storage/sd.py 3 files changed, 7 insertions(+), 15 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/15/8115/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index e596fb6..2da277d 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -477,14 +477,6 @@ return time.time() - t - def produceVolume(self, imgUUID, volUUID): - """ - Produce a type specific volume object - """ - repoPath = self._getRepoPath() - return blockVolume.BlockVolume(repoPath, self.sdUUID, imgUUID, volUUID) - - def getVolumeClass(self): """ Return a type specific volume generator object diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 054fadb..ae06c90 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -215,13 +215,6 @@ filesDict[fileName] = stats return filesDict - def produceVolume(self, imgUUID, volUUID): - """ - Produce a type specific volume object - """ - repoPath = self._getRepoPath() - return fileVolume.FileVolume(repoPath, self.sdUUID, imgUUID, volUUID) - def getVolumeClass(self): """ Return a type specific volume generator object diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 55c5b12..4e7485f 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -358,6 +358,13 @@ self.log.warn("Resource namespace %s already registered", volumeResourcesNamespace) + def produceVolume(self, imgUUID, volUUID): + """ + Produce a type specific volume object + """ + repoPath = self._getRepoPath() + return self.getVolumeClass()(repoPath, self.sdUUID, imgUUID, volUUID) + def getVolumeClass(self): """ Return a type specific volume generator object -- To view, visit http://gerrit.ovirt.org/8115 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I367582b2741dc0276ec64d8d13fa1e4953f0314b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com>

5 18

Change in vdsm[master]: Introduce getMountPoint for NfsStorageDomain
by deepakcs＠linux.vnet.ibm.com 13 Jan '13

13 Jan '13

Deepak C Shetty has uploaded a new change for review. Change subject: Introduce getMountPoint for NfsStorageDomain ...................................................................... Introduce getMountPoint for NfsStorageDomain Move the code to generate the mount point into a seperate function, so that classes reusing NfsStorageDomain can override the function to present their own mount point path. Change-Id: I80f98eddcc31c1f01f42af838ecc483abb955f41 Signed-off-by: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com> --- M vdsm/storage/nfsSD.py 1 file changed, 6 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/85/8085/1 diff --git a/vdsm/storage/nfsSD.py b/vdsm/storage/nfsSD.py index d84fee6..6b53208 100644 --- a/vdsm/storage/nfsSD.py +++ b/vdsm/storage/nfsSD.py @@ -71,8 +71,7 @@ # Create local path mntPath = fileUtils.transformPath(remotePath) - mntPoint = os.path.join(cls.storage_repository, - sd.DOMAIN_MNT_POINT, mntPath) + mntPoint = cls.getMountPoint(mntPath) cls._preCreateValidation(sdUUID, mntPoint, remotePath, version) @@ -94,6 +93,11 @@ return fsd + @classmethod + def getMountPoint(cls, mountPath): + return os.path.join(cls.storage_repository, + sd.DOMAIN_MNT_POINT, mountPath) + def selftest(self): """ Run internal self test -- To view, visit http://gerrit.ovirt.org/8085 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I80f98eddcc31c1f01f42af838ecc483abb955f41 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com>

7 26

Change in vdsm[master]: [WIP] vdsm API and libvdsm
by agl＠us.ibm.com 11 Jan '13

11 Jan '13

Adam Litke has uploaded a new change for review. Change subject: [WIP] vdsm API and libvdsm ...................................................................... [WIP] vdsm API and libvdsm In this release you can actually execute real API calls against a remote server. See the test programs for examples in C and Python. Known issues: - Unions are untested and probably broken - Optional value parameters (int, bool, enum, etc) do not work from python - Incorrect GIR generation for functions returning arrays (see gir-fixes.patch) - Need to convert parameters/return values in Bridge when the schema types differ from the internal types (ie. IntStr) Changes since V1: - API is generated for the full schema - Server-side support - RPM packaging (vdsm-jsonrpc (for server) and vdsm-api (for client)) This work-in-progress patch demonstrates the mechanics of generating a working C API and remote transport from a schema definition. The "C" API is written in Vala to more easily take advantage of GObjects and reduce the amount of boilerplate code that we need to generate and maintain. The API has a static part and a dynamically generated part. I generate the dynamic part by using the same technique employed by qemu for generating the qapi bindings (python format strings). The build system produces libvdsm and a C test program. An included python program demonstrates how the automatic python bindings work. A test server is included to enable the test programs to function. We use JSON for internal object storage and also as the wire format. Thank you Saggi for sharing some great ideas (many of which are incorporated into this patch). TODO: - Fix issues in the main schema * Return values must be anonymous, singular types * Remove 'IntStr' and friends in favor of native representation - Support generation for the entire schema * Mapping types * Enums with non-C string values * Finish array support * Generate the command sub-classes (VM, Volume, StorageDomain, etc) - Connect the server to the real vdsm API * Functions must map from the new schema to the old implementation - Validate argument and return types on the server side - Probably a few more things I haven't yet thought of Signed-off-by: Adam Litke <agl(a)us.ibm.com> Change-Id: If6bd34700b86aa84c7e289f02c0e9f2ac6fcba63 --- M Makefile.am M build-aux/gitlog-to-changelog M configure.ac M vdsm.spec.in M vdsm/clientIF.py M vdsm/config.py.in A vdsm_api/BindingJsonRpc.py A vdsm_api/Bridge.py A vdsm_api/Makefile.am A vdsm_api/generate.py A vdsm_api/gir-fixes.patch A vdsm_api/libvdsm-base.vala A vdsm_api/schema.json A vdsm_api/test.c A vdsm_api/test.py A vdsm_api/test_serv.py M vdsm_api/vdsmapi-schema.json 17 files changed, 4,308 insertions(+), 59 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/16/7516/1 diff --git a/Makefile.am b/Makefile.am index 17b5a85..d52964b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -18,10 +18,12 @@ # Refer to the README and COPYING files for full details of the license # -SUBDIRS = vdsm vdsm_cli vds_bootstrap vdsm_reg vdsm_hooks tests vdsm-tool +SUBDIRS = vdsm vdsm_cli vds_bootstrap vdsm_reg vdsm_hooks tests vdsm-tool vdsm_api include $(top_srcdir)/build-aux/Makefile.subs +ACLOCAL_AMFLAGS = -I m4 + # This is an *exception*, we ship also vdsm.spec so it's possible to build the # rpm from the tarball. EXTRA_DIST = \ diff --git a/build-aux/gitlog-to-changelog b/build-aux/gitlog-to-changelog old mode 100755 new mode 100644 diff --git a/configure.ac b/configure.ac index d4b6815..65d4ada 100644 --- a/configure.ac +++ b/configure.ac @@ -44,6 +44,37 @@ AC_PROG_LN_S AM_PATH_PYTHON([2.6]) +# Enable libtool for libvdsm +LT_INIT +AC_CONFIG_MACRO_DIR([m4]) + +AC_PATH_PROG([VALAC], [valac], []) + +# GLib dependencies for libvdsm +PKG_CHECK_MODULES([GLIB], [glib-2.0]) +AC_SUBST(GLIB_CFLAGS) +AC_SUBST(GLIB_LIBS) + +PKG_CHECK_MODULES([JSON_GLIB], [json-glib-1.0]) +AC_SUBST(JSON_GLIB_CFLAGS) +AC_SUBST(JSON_GLIB_LIBS) + +PKG_CHECK_MODULES([GEE], [gee-0.8]) +AC_SUBST(GEE_CFLAGS) +AC_SUBST(GEE_LIBS) + +PKG_CHECK_MODULES([GOBJECT], [gobject-2.0]) +AC_SUBST(GOBJECT_CFLAGS) +AC_SUBST(GOBJECT_LIBS) + +PKG_CHECK_MODULES([GOBJECT_INTROSPECTION], [gobject-introspection-1.0], [enable_introspection=yes]) +AC_SUBST(GOBJECT_INTROSPECTION_CFLAGS) +AC_SUBST(GOBJECT_INTROSPECTION_LIBS) + +AC_SUBST([G_IR_SCANNER], [$($PKG_CONFIG --variable=g_ir_scanner gobject-introspection-1.0)]) +AC_SUBST([G_IR_COMPILER], [$($PKG_CONFIG --variable=g_ir_compiler gobject-introspection-1.0)]) + + # Checking if hooks enables AC_ARG_ENABLE([hooks], [ --enable-hooks build hooks RPMs], @@ -79,6 +110,9 @@ AC_SUBST([vdsmpylibdir], ['${pyexecdir}/vdsm']) AC_SUBST([vdsmtooldir], ['${vdsmpylibdir}/tool']) AC_SUBST([vdsmtestsdir], ['${datarootdir}/vdsm/tests']) +AC_SUBST([vdsmapidir], ['${datarootdir}/vdsm_api']) +AC_SUBST([vdsmtypelibdir], ['${exec_prefix}/lib/girepository-1.0']) +AC_SUBST([vdsmgirdir], ['${datarootdir}/gir-1.0']) # VDSM registration default paths AC_SUBST([vdsmregdir], ['${datarootdir}/vdsm-reg']) @@ -120,6 +154,9 @@ # Checking for python modules (sorted, please keep in order) AX_PYTHON_MODULE([ethtool], [fatal]) AX_PYTHON_MODULE([libvirt], [fatal]) + +# Checking for vala compiler +AM_PROG_VALAC([ >=0.16.0]) # External programs (sorted, please keep in order) AC_PATH_PROG([BLKID_PATH], [blkid], [/sbin/blkid]) @@ -186,6 +223,7 @@ tests/Makefile tests/functional/Makefile vds_bootstrap/Makefile + vdsm_api/Makefile vdsm_cli/Makefile vdsm_hooks/directlun/Makefile vdsm_hooks/faqemu/Makefile diff --git a/vdsm.spec.in b/vdsm.spec.in index ce73f6e..0f9871f 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -128,6 +128,15 @@ %description rest A REST interface for interacting with vdsmd. +%package jsonrpc +Summary: VDSM API Server +BuildArch: noarch + +Requires: %{name}-python = %{version}-%{release} + +%description jsonrpc +A Json-based RPC interface that serves as the transport for libvdsm. + %package bootstrap Summary: VDSM bootstrapping package BuildArch: noarch @@ -169,6 +178,12 @@ %description tests A test suite for verifying the functionality of a running vdsm instance + +%package api +Summary: VDSM API + +%description api +Library and utilities for clients to use the vdsm API %package hook-vhostmd Summary: VDSM hook set for interaction with vhostmd @@ -731,6 +746,16 @@ %{_datadir}/%{vdsm_name}/tests/netmaskconversions %{_datadir}/%{vdsm_name}/tests/functional/*.py* +%files api +%{_includedir}/libvdsm-0.1/libvdsm/vdsm.h +%{_exec_prefix}/lib/girepository-1.0/vdsm-0.1.typelib +%{_datadir}/gir-1.0/vdsm-0.1.gir +%{_libdir}/libvdsm-0.1.a +%{_libdir}/libvdsm-0.1.la +%{_libdir}/libvdsm-0.1.so +%{_libdir}/libvdsm-0.1.so.0 +%{_libdir}/libvdsm-0.1.so.0.0.0 + %files hook-vhostmd %defattr(-, root, root, -) %doc COPYING @@ -853,6 +878,10 @@ %{_datadir}/%{vdsm_name}/rest/templates/api.xsd %{_datadir}/%{vdsm_name}/rest/templates/rsdl.xml +%files jsonrpc +%{_datadir}/%{vdsm_name}/BindingJsonRpc.py* +%{_datadir}/%{vdsm_name}/Bridge.py* + %files bootstrap %defattr(-, root, root, -) %doc COPYING diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index ab34eaa..b212da4 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -158,6 +158,14 @@ self.bindings['rest'] = BindingREST(self, self.log, ip, rest_port, templatePath) + def _loadBindingJsonRpc(self): + from BindingJsonRpc import BindingJsonRpc + from Bridge import VdsmBridge + ip = self._getServerIP(config.get('addresses', 'management_ip')) + port = config.getint('addresses', 'json_port') + self.bindings['json'] = BindingJsonRpc(VdsmBridge(), self.log, + ip, port) + def _prepareBindings(self): self.bindings = {} if config.getboolean('vars', 'xmlrpc_enable'): @@ -172,6 +180,12 @@ except ImportError: self.log.warn('Unable to load the rest server module. ' 'Please make sure it is installed.') + if config.getboolean('vars', 'jsonrpc_enable'): + try: + self._loadBindingJsonRpc() + except ImportError: + self.log.warn('Unable to load the json rpc server module. ' + 'Please make sure it is installed.') def _prepareMOM(self): try: diff --git a/vdsm/config.py.in b/vdsm/config.py.in index 56cd28e..8571365 100644 --- a/vdsm/config.py.in +++ b/vdsm/config.py.in @@ -130,6 +130,8 @@ ('rest_enable', 'true', 'Enable the REST server'), + ('jsonrpc_enable', 'true', 'Enable the JSON RPC server'), + ('report_host_threads_as_cores', 'false', 'Count each cpu hyperthread as an individual core'), ]), @@ -245,6 +247,10 @@ 'Port on which the vdsmd REST server listens to network ' 'clients.'), + ('json_port', '4444', + 'Port on which the vdsmd Json RPC server listens to network ' + 'clients.'), + ('management_ip', '', None), ('guests_gateway_ip', '', None), diff --git a/vdsm_api/BindingJsonRpc.py b/vdsm_api/BindingJsonRpc.py new file mode 100644 index 0000000..a2b9c9f --- /dev/null +++ b/vdsm_api/BindingJsonRpc.py @@ -0,0 +1,93 @@ +# VDSM JsonRPC Server +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +import threading +import SocketServer +import json + +import struct + +Size = struct.Struct("!Q") + +__log__ = None +__bridge__ = None + +class BindingJsonRpc: + def __init__(self, bridge, log, ip, port): + self.bridge = bridge + self.log = log + self.serverPort = port + if not ip: + self.serverIP = '0.0.0.0' + else: + self.serverIP = ip + self._create_server() + + def _create_server(self): + global __bridge__ + global __log__ + __bridge__ = self.bridge + __log__ = self.log + self.server = SocketServer.TCPServer((self.serverIP, self.serverPort), + JsonRpcTCPHandler) + + def start(self): + def threaded_start(): + self.server.serve_forever() + threading.Thread(target=threaded_start, + name='JsonRpc').start() + + def prepareForShutdown(self): + self.server.shutdown() + +class JsonRpcTCPHandler(SocketServer.StreamRequestHandler): + """ + The RequestHandler class for our server. + + It is instantiated once per connection to the server, and must + override the handle() method to implement communication to the + client. + """ + + def handle(self): + bridge = __bridge__ + log = __log__ + while True: + # self.request is the TCP socket connected to the client + try: + msgLen = Size.unpack(self.request.recv(Size.size))[0] + msg = json.loads(self.request.recv(msgLen)) + except: + return + + log.debug("{} wrote:".format(self.client_address[0])) + log.debug(msg) + + try: + ret = bridge._dispatch(msg['methodName'], msg.get('args', {})) + except Exception: + log.error("Dispatch error", exc_info=True) + continue + log.debug("ret = %s", ret) + ret['id'] = msg['id'] + msg = json.dumps(ret) + msg = msg.encode('utf-8') + msize = Size.pack(len(msg) - 1) + resp = msize + msg + log.debug(msg) + + self.wfile.write(resp) + self.wfile.flush() + diff --git a/vdsm_api/Bridge.py b/vdsm_api/Bridge.py new file mode 100644 index 0000000..f7e2a7f --- /dev/null +++ b/vdsm_api/Bridge.py @@ -0,0 +1,850 @@ +# VDSM API Connector +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +#import Fixups +try: + import API +except ImportError: + pass # Needed for the test bridge + +class VdsmError(Exception): + def __init__(self, code, message): + self.code = code + self.message = message + +class MethodBridge: + #def __init__(self): + # self.fixup = Fixups.Fixup() + + def _dispatch(self, name, argobj): + method = name.replace('.', '_').lower() + result = None + error = {'code': 0, 'message': 'Success'} + #argobj = self.fixup.fix_call(name, argobj, 'in') + try: + fn = getattr(self, method) + except AttributeError: + error = {'code': 4, + 'message': "Operation '%s' not supported" % name} + return {'result': result, 'error': error} + try: + result = fn(argobj) + except VdsmError, e: + error = {'code': e.code, 'message': e.message} + #result = self.fixup.fix_call(name, result, 'out') + return {'result': result, 'error': error} + + def get_args(self, argobj, arglist): + ret = () + for arg in arglist: + ret += (argobj.get(arg),) + return ret + + def check_error(self, response): + code = response['status']['code'] + if code != 0: + message = response['status']['message'] + raise VdsmError(code, message) + + def get_result(self, response, member=None): + self.check_error(response) + if member is None: + return None + try: + return response[member] + except KeyError: + raise VdsmError(5, "Response is missing '%s' member" % member) + +class VdsmBridge(MethodBridge): + def host_getvms(self, argobj): + ret = API.Global().getVMList(False, []) + ret = self.get_result(ret, 'vmList') + return [v['vmId'] for v in ret] + + def vm_getinfo(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + ret = API.Global().getVMList(True, [objargs[0]]) + ret = self.get_result(ret, 'vmList') + return ret[0] + + def host_addnetwork(self, argobj): + args = self.get_args(argobj, + ['bridge', 'vlan', 'bond', 'nics', 'options']) + ret = API.Global().addNetwork(*args) + return self.get_result(ret) + + def host_delnetwork(self, argobj): + args = self.get_args(argobj, + ['bridge', 'vlan', 'bond', 'nics', 'options']) + ret = API.Global().delNetwork(*args) + return self.get_result(ret) + + def host_editnetwork(self, argobj): + args = self.get_args(argobj, ['oldBridge', 'newBridge', 'vlan', + 'bond', 'nics', 'options']) + ret = API.Global().editNetwork(*args) + return self.get_result(ret) + + def host_setupnetworks(self, argobj): + args = self.get_args(argobj, ['networks', 'bondings', 'options']) + ret = API.Global().setupNetworks(*args) + return self.get_result(ret) + + def host_fencenode(self, argobj): + args = self.get_args(argobj, ['addr', 'port', 'agent', 'username', + 'password', 'action', 'secure', 'options']) + ret = API.Global().fenceNode(*args) + return self.get_result(ret, 'power') + + def host_getalltasksinfo(self, argobj): + args = self.get_args(argobj, []) + ret = API.Global().getAllTasksInfo(*args) + return self.get_result(ret, 'allTasksInfo') + + def host_getalltasksstatuses(self, argobj): + args = self.get_args(argobj, []) + ret = API.Global().getAllTasksStatuses(*args) + return self.get_result(ret, 'allTasksStatus') + + def host_getcapabilities(self, argobj): + """ + Returns 2 values: info and netConfigDirty which must be merged into + a single value + """ + args = self.get_args(argobj, []) + resp = API.Global().getCapabilities(*args) + ret_data = self.get_result(resp, 'info') + #ret_data['netConfigDirty'] = self.get_result(resp, 'netConfigDirty') + return ret_data + + def host_getconnectedstoragepools(self, argobj): + args = self.get_args(argobj, []) + ret = API.Global().getConnectedStoragePools(*args) + return self.get_result(ret, 'poollist') + + def host_getdeviceinfo(self, argobj): + args = self.get_args(argobj, ['guid']) + ret = API.Global().getDeviceInfo(*args) + return self.get_result(ret, 'info') + + def host_getdevicelist(self, argobj): + args = self.get_args(argobj, ['storageType']) + ret = API.Global().getDeviceList(*args) + return self.get_result(ret, 'devList') + + def host_getdevicesvisibility(self, argobj): + args = self.get_args(argobj, ['guidList']) + ret = API.Global().getDevicesVisibility(*args) + return self.get_result(ret, 'visibility') + + def host_getlvmvolumegroups(self, argobj): + args = self.get_args(argobj, ['storageType']) + ret = API.Global().getLVMVolumeGroups(*args) + return self.get_result(ret, 'vglist') + + def host_getstats(self, argobj): + args = self.get_args(argobj, []) + ret = API.Global().getStats(*args) + return self.get_result(ret, 'info') + + def host_getstoragedomains(self, argobj): + args = self.get_args(argobj, ['storagepoolID', 'domainClass', 'storageType', + 'remotePath']) + ret = API.Global().getStorageDomains(*args) + return self.get_result(ret, 'domlist') + + def host_getstoragerepostats(self, argobj): + """ + The returned dictionary doesn't separate the stats from the status code + so we need to rebuild the result. + """ + args = self.get_args(argobj, []) + resp = API.Global().getStorageRepoStats(*args) + self.check_error(resp) + del resp['status'] + return resp + + def host_getvmlist(self, argobj): + args = self.get_args(argobj, ['fullStatus', 'vmList']) + ret = API.Global().getVMList(*args) + return self.get_result(ret, 'vmList') + + def host_ping(self, argobj): + args = self.get_args(argobj, []) + ret = API.Global().ping(*args) + return self.get_result(ret) + + def host_setloglevel(self, argobj): + args = self.get_args(argobj, ['level']) + ret = API.Global().setLogLevel(*args) + return self.get_result(ret) + + def host_setsafenetworkconfig(self, argobj): + args = self.get_args(argobj, []) + ret = API.Global().setSafeNetworkConfig(*args) + return self.get_result(ret) + + def connectionrefs_acquire(self, argobj): + args = self.get_args(argobj, ['conRefArgs']) + ret = API.ConnectionRefs().acquire(*args) + return self.get_result(ret, 'results') + + def connectionrefs_release(self, argobj): + args = self.get_args(argobj, ['refIDs']) + ret = API.ConnectionRefs().release(*args) + return self.get_result(ret, 'results') + + def connectionrefs_statuses(self, argobj): + args = self.get_args(argobj, []) + ret = API.ConnectionRefs().statuses(*args) + return self.get_result(ret, 'connectionslist') + + def iscsiconnection_discoversendtargets(self, argobj): + """ + This returns two lists: 'targets' & 'fullTargets'. Just return + the 'fullTargets' one because it supercedes the old 'targets'. + """ + objargs = self.get_args(argobj['__obj__'], + ['host', 'port', 'username', 'password']) + args = self.get_args(argobj, []) + ret = API.ISCSIConnection(*objargs).discoverSendTargets(*args) + return self.get_result(ret, 'fullTargets') + + def image_delete(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['imageID', 'storagepoolID', 'storagedomainID']) + args = self.get_args(argobj, ['postZero', 'force']) + ret = API.Image(*objargs).delete(*args) + return self.get_result(ret, 'uuid') + + def image_deletevolumes(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['imageID', 'storagepoolID', 'storagedomainID']) + args = self.get_args(argobj, ['volumeList', 'postZero', 'force']) + ret = API.Image(*objargs).deleteVolumes(*args) + return self.get_result(ret, 'uuid') + + def image_getvolumes(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['imageID', 'storagepoolID', 'storagedomainID']) + args = self.get_args(argobj, []) + ret = API.Image(*objargs).getVolumes(*args) + return self.get_result(ret, 'uuidlist') + + def image_mergesnapshots(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['imageID', 'storagepoolID', 'storagedomainID']) + args = self.get_args(argobj, ['ancestor', 'successor', 'postZero']) + ret = API.Image(*objargs).mergeSnapshots(*args) + return self.get_result(ret, 'uuid') + + def image_move(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['imageID', 'storagepoolID', 'storagedomainID']) + args = self.get_args(argobj, ['dstSdUUID', 'operation', 'postZero', + 'force']) + ret = API.Image(*objargs).move(*args) + return self.get_result(ret, 'uuid') + + def lvmvolumegroup_create(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['lvmvolumegroupID']) + args = self.get_args(argobj, ['name', 'devlist', 'force']) + ret = API.LVMVolumeGroup(*objargs).create(*args) + return self.get_result(ret, 'uuid') + + def lvmvolumegroup_getinfo(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['lvmvolumegroupID']) + args = self.get_args(argobj, []) + ret = API.LVMVolumeGroup(*objargs).getInfo(*args) + return self.get_result(ret, 'info') + + def lvmvolumegroup_remove(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['lvmvolumegroupID']) + args = self.get_args(argobj, []) + ret = API.LVMVolumeGroup(*objargs).remove(*args) + return self.get_result(ret) + + def storagedomain_activate(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StorageDomain(*objargs).activate(*args) + return self.get_result(ret) + + def storagedomain_attach(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['storagepoolID']) + ret = API.StorageDomain(*objargs).attach(*args) + return self.get_result(ret) + + def storagedomain_create(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['_type', 'typeArgs', 'name', + 'domainClass', 'version']) + ret = API.StorageDomain(*objargs).create(*args) + return self.get_result(ret) + + def storagedomain_deactivate(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['masterSdUUID', 'masterVersion']) + ret = API.StorageDomain(*objargs).deactivate(*args) + return self.get_result(ret) + + def storagedomain_detach(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, + ['masterSdUUID', 'masterVersion', 'force']) + ret = API.StorageDomain(*objargs).detach(*args) + return self.get_result(ret) + + def storagedomain_extend(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['devlist']) + ret = API.StorageDomain(*objargs).extend(*args) + return self.get_result(ret) + + def storagedomain_format(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['autoDetach']) + ret = API.StorageDomain(*objargs).format(*args) + return self.get_result(ret) + + def storagedomain_getfilelist(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['pattern']) + ret = API.StorageDomain(*objargs).getFileList(*args) + return self.get_result(ret, 'files') + + def storagedomain_getimages(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StorageDomain(*objargs).getImages(*args) + return self.get_result(ret, 'imageslist') + + def storagedomain_getinfo(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StorageDomain(*objargs).getInfo(*args) + return self.get_result(ret, 'info') + + def storagedomain_getstats(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StorageDomain(*objargs).getStats(*args) + return self.get_result(ret, 'stats') + + def storagedomain_getvolumes(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['imageID']) + ret = API.StorageDomain(*objargs).getVolumes(*args) + return self.get_result(ret, 'uuidlist') + + def storagedomain_setdescription(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['description']) + ret = API.StorageDomain(*objargs).setDescription(*args) + return self.get_result(ret) + + def storagedomain_uploadvolume(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, ['imageID', 'volumeID', 'srcPath', 'size', + 'method']) + ret = API.StorageDomain(*objargs).uploadVolume(*args) + return self.get_result(ret) + + def storagedomain_validate(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagedomainID', 'storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StorageDomain(*objargs).validate(*args) + return self.get_result(ret) + + def storagepool_connect(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['hostID', 'scsiKey', 'masterSdUUID', + 'masterVersion']) + ret = API.StoragePool(*objargs).connect(*args) + return self.get_result(ret) + + def storagepool_connectstorageserver(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['domainType', 'connectionParams']) + ret = API.StoragePool(*objargs).connectStorageServer(*args) + return self.get_result(ret, 'statuslist') + + def storagepool_create(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, + ['name', 'masterSdUUID', 'masterVersion', 'domainList', + 'lockRenewalIntervalSec', 'leaseTimeSec', + 'ioOpTimeoutSec', 'leaseRetries']) + ret = API.StoragePool(*objargs).create(*args) + return self.get_result(ret) + + def storagepool_destroy(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['hostID', 'scsiKey']) + ret = API.StoragePool(*objargs).destroy(*args) + return self.get_result(ret) + + def storagepool_disconnect(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['hostID', 'scsiKey', 'remove']) + ret = API.StoragePool(*objargs).disconnect(*args) + return self.get_result(ret) + + def storagepool_disconnectstorageserver(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['domainType', 'connectionParams']) + ret = API.StoragePool(*objargs).disconnectStorageServer(*args) + return self.get_result(ret, 'statuslist') + + def storagepool_fence(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StoragePool(*objargs).fence(*args) + return self.get_result(ret, 'spm_st') + + def storagepool_getbackedupvmsinfo(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['storagedomainID', 'vmList']) + ret = API.StoragePool(*objargs).getBackedUpVmsInfo(*args) + return self.get_result(ret, 'vmlist') + + def storagepool_getbackedupvmslist(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['storagedomainID']) + ret = API.StoragePool(*objargs).getBackedUpVmsList(*args) + return self.get_result(ret, 'vmlist') + + def storagepool_getfloppylist(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StoragePool(*objargs).getFloppyList(*args) + return self.get_result(ret, 'isolist') + + def storagepool_getdomainscontainingimage(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['imageID', 'onlyDataDomains']) + ret = API.StoragePool(*objargs).getDomainsContainingImage(*args) + return self.get_result(ret, 'domainslist') + + def storagepool_getisolist(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, ['filenameExtension']) + ret = API.StoragePool(*objargs).getIsoList(*args) + return self.get_result(ret, 'isolist') + + def storagepool_getspmstatus(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StoragePool(*objargs).getSpmStatus(*args) + return self.get_result(ret, 'spm_st') + + def storagepool_getinfo(self, argobj): + """ + The result contains two data structures which must be merged + """ + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.get_args(argobj, []) + ret = API.StoragePool(*objargs).getInfo(*args) + ret_data = {} + ret_data['info'] = self.get_result(ret, 'info') + ret_data['dominfo'] = self.get_result(ret, 'dominfo') + return ret_data + + def storagepool_movemultipleimages(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, + ['srcSdUUID', 'dstSdUUID', 'imgDict', 'force']) + ret = API.StoragePool(*objargs).moveMultipleImages(*args) + return self.get_result(ret) + + def storagepool_reconstructmaster(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, + ['hostId', 'name', 'masterSdUUID', 'masterVersion', + 'domainDict', 'lockRenewalIntervalSec', 'leaseTimeSec', + 'ioOpTimeoutSec', 'leaseRetries']) + ret = API.StoragePool(*objargs).reconstructMaster(*args) + return self.get_result(ret) + + def storagepool_refresh(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, ['masterSdUUID', 'masterVersion']) + ret = API.StoragePool(*objargs).refresh(*args) + return self.get_result(ret) + + def storagepool_setdescription(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, ['description']) + ret = API.StoragePool(*objargs).setDescription(*args) + return self.get_result(ret) + + def storagepool_spmstart(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, + ['prevID', 'prevLver', 'enableScsiFencing', + 'maxHostID', 'domVersion']) + ret = API.StoragePool(*objargs).spmStart(*args) + return self.get_result(ret, 'uuid') + + def storagepool_spmstop(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, []) + ret = API.StoragePool(*objargs).spmStop(*args) + return self.get_result(ret) + + def storagepool_upgrade(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, ['targetDomVersion']) + ret = API.StoragePool(*objargs).upgrade(*args) + return self.get_result(ret, 'upgradeStatus') + + def storagepool_validatestorageserverconnection(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, ['domainType', 'connectionParams']) + ret = API.StoragePool(*objargs).validateStorageServerConnection(*args) + return self.get_result(ret, 'statuslist') + + def storagepool_updatevms(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, ['vmList', 'storagedomainID']) + ret = API.StoragePool(*objargs).updateVMs(*args) + return self.get_result(ret) + + def storagepool_removevm(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['storagepoolID']) + args = self.self.get_args(argobj, ['vmUUID', 'storagedomainID']) + ret = API.StoragePool(*objargs).removeVM(*args) + return self.get_result(ret) + + def task_clear(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['taskID']) + args = self.self.get_args(argobj, []) + ret = API.Task(*objargs).clear(*args) + return self.get_result(ret) + + def task_getinfo(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['taskID']) + args = self.self.get_args(argobj, []) + ret = API.Task(*objargs).getInfo(*args) + return self.get_result(ret, 'TaskInfo') + + def task_getstatus(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['taskID']) + args = self.self.get_args(argobj, []) + ret = API.Task(*objargs).getStatus(*args) + return self.get_result(ret, 'taskStatus') + + def task_revert(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['taskID']) + args = self.self.get_args(argobj, []) + ret = API.Task(*objargs).revert(*args) + return self.get_result(ret) + + def task_stop(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['taskID']) + args = self.self.get_args(argobj, []) + ret = API.Task(*objargs).stop(*args) + return self.get_result(ret) + + def vm_changecd(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, ['driveSpec']) + ret = API.VM(*objargs).changeCD(*args) + return self.get_result(ret, 'vmList') + + def vm_changefloppy(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, ['driveSpec']) + ret = API.VM(*objargs).changeFloppy(*args) + return self.get_result(ret, 'vmList') + + def vm_cont(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, []) + ret = API.VM(*objargs).cont(*args) + return self.get_result(ret) + + def vm_create(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, ['vmParams']) + ret = API.VM(*objargs).create(*args) + return self.get_result(ret, 'vmList') + + def vm_desktoplock(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, []) + ret = API.VM(*objargs).desktopLock(*args) + return self.get_result(ret) + + def vm_desktoplogin(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, ['domain', 'username', 'password']) + ret = API.VM(*objargs).desktopLogin(*args) + return self.get_result(ret) + + def vm_desktoplogoff(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.self.get_args(argobj, ['force']) + ret = API.VM(*objargs).desktopLogoff(*args) + return self.get_result(ret) + + def vm_desktopsendhccommand(self, argobj): + objargs = self.self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['message']) + ret = API.VM(*objargs).desktopSendHcCommand(*args) + return self.get_result(ret) + + def vm_destroy(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).destroy(*args) + return self.get_result(ret) + + def vm_getmigrationstatus(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).getMigrationStatus(*args) + return self.get_result(ret) + + def vm_getstats(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).getStats(*args) + return self.get_result(ret, 'statsList') + + def vm_hibernate(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['hibernationVolHandle']) + ret = API.VM(*objargs).hibernate(*args) + return self.get_result(ret) + + def vm_hotplugnic(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['params']) + ret = API.VM(*objargs).hotplugNic(*args) + return self.get_result(ret, 'vmList') + + def vm_hotunplugnic(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['params']) + ret = API.VM(*objargs).hotunplugNic(*args) + return self.get_result(ret, 'vmList') + + def vm_hotplugdisk(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['params']) + ret = API.VM(*objargs).hotplugDisk(*args) + return self.get_result(ret, 'vmList') + + def vm_hotunplugdisk(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['params']) + ret = API.VM(*objargs).hotunplugDisk(*args) + return self.get_result(ret, 'vmList') + + def vm_migrate(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['params']) + ret = API.VM(*objargs).migrate(*args) + return self.get_result(ret) + + def vm_migratecancel(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).migrateCancel(*args) + return self.get_result(ret) + + def vm_migrationcreate(self, argobj): + """ + Returns 2 values: migrationPort and params which must be merged into + a single value + """ + args = self.get_args(argobj, []) + resp = API.Global().getCapabilities(*args) + ret_data = {} + ret_data['params'] = self.get_result(resp, 'params') + ret_data['migrationPort'] = self.get_result(resp, 'migrationPort') + return ret_data + + def vm_monitorcommand(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['command']) + ret = API.VM(*objargs).monitorCommand(*args) + return self.get_result(ret) + + def vm_pause(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).pause(*args) + return self.get_result(ret) + + def vm_reset(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).reset(*args) + return self.get_result(ret) + + def vm_sendkeys(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['keySequence']) + ret = API.VM(*objargs).sendKeys(*args) + return self.get_result(ret) + + def vm_setticket(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, + ['password', 'ttl', 'existingConnAction', 'params']) + ret = API.VM(*objargs).setTicket(*args) + return self.get_result(ret) + + def vm_shutdown(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['delay', 'message']) + ret = API.VM(*objargs).shutdown(*args) + return self.get_result(ret) + + def vm_snapshot(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['snapDrives']) + ret = API.VM(*objargs).snapshot(*args) + return self.get_result(ret) + + def vm_merge(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, ['mergeDrives']) + ret = API.VM(*objargs).merge(*args) + return self.get_result(ret) + + def vm_mergestatus(self, argobj): + objargs = self.get_args(argobj['__obj__'], ['vmID']) + args = self.get_args(argobj, []) + ret = API.VM(*objargs).mergeStatus(*args) + return self.get_result(ret, 'mergeStatus') + + def volume_copy(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['dstSdUUID', 'dstImgUUID', 'dstVolUUID', + 'desc', 'volType', 'volFormat', 'preallocate', + 'postZero', 'force']) + ret = API.Volume(*objargs).copy(*args) + return self.get_result(ret, 'uuid') + + def volume_create(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['size', 'volFormat', 'preallocate', + 'diskType', 'desc', 'srcImgUUID', + 'srcVolUUID']) + ret = API.Volume(*objargs).create(*args) + return self.get_result(ret, 'uuid') + + def volume_delete(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['postZero', 'force']) + ret = API.Volume(*objargs).delete(*args) + return self.get_result(ret, 'uuid') + + def volume_extend(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['size', 'isShuttingDown']) + ret = API.Volume(*objargs).extend(*args) + return self.get_result(ret) + + def volume_getinfo(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, []) + ret = API.Volume(*objargs).getInfo(*args) + return self.get_result(ret, 'info') + + def volume_getpath(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, []) + ret = API.Volume(*objargs).getPath(*args) + return self.get_result(ret, 'path') + + def volume_getsize(self, argobj): + """ + This function returns 2 values: apparentsize and truesize. They have + to be merged into a dictionary together + """ + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, []) + ret_data = {} + resp = API.Volume(*objargs).getSize(*args) + ret_data['truesize'] = self.get_result(resp, 'truesize') + ret_data['apparentsize'] = self.get_result(resp, 'apparentsize') + return ret_data + + def volume_prepare(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['rw']) + ret = API.Volume(*objargs).prepare(*args) + return self.get_result(ret) + + def volume_refresh(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, []) + ret = API.Volume(*objargs).refresh(*args) + return self.get_result(ret) + + def volume_setdescription(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['description']) + ret = API.Volume(*objargs).setDescription(*args) + return self.get_result(ret) + + def volume_setlegality(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, ['legality']) + ret = API.Volume(*objargs).setLegality(*args) + return self.get_result(ret) + + def volume_teardown(self, argobj): + objargs = self.get_args(argobj['__obj__'], + ['volumeID', 'storagepoolID', 'storagedomainID', 'imageID']) + args = self.get_args(argobj, []) + ret = API.Volume(*objargs).tearDown(*args) + return self.get_result(ret) + + +class TestBridge(MethodBridge): + def test_ping(self, argobj): + return {} + + def test_pong(self, argobj): + return argobj['a'] + + def test_passobject(self, argobj): + argobj['a']['nics'][0] = 'Hello!' + return argobj + + def test_maptest(self, argobj): + m = argobj['a'] + ret = {} + for k,v in m.items(): + ret[k] = str(v) + return ret diff --git a/vdsm_api/Makefile.am b/vdsm_api/Makefile.am new file mode 100644 index 0000000..2126725 --- /dev/null +++ b/vdsm_api/Makefile.am @@ -0,0 +1,94 @@ +lib_LTLIBRARIES = libvdsm-0.1.la +libvdsm_0_1_ladir = $(includedir)/libvdsm-0.1/libvdsm +libvdsm_0_1_la_HEADERS = \ + vdsm.h \ + $(NULL) + +libvdsm_0_1_la_SOURCES = \ + libvdsm-base.vala \ + generated.vala \ + $(NULL) + +BUILT_SOURCES = \ + generated.vala \ + $(NULL) + +EXTRA_DIST = \ + generate.py \ + vdsmapi-schema.json \ + $(NULL) + +dist_vdsmtypelib_DATA = \ + vdsm-0.1.typelib \ + $(NULL) + +dist_vdsmgir_DATA = \ + vdsm-0.1.gir \ + $(NULL) + +dist_vdsm_PYTHON = \ + BindingJsonRpc.py \ + Bridge.py \ + $(NULL) + +libvdsm_0_1_la_LIBADD = \ + $(GLIB_LIBS) \ + $(GOBJECT_LIBS) \ + $(JSON_GLIB_LIBS) \ + $(GEE_LIBS) \ + $(NULL) + +AM_CFLAGS = \ + -Werror \ + $(GLIB_CFLAGS) \ + $(JSON_GLIB_CFLAGS) \ + $(GEE_CFLAGS) \ + $(GOBJECT_CFLAGS) \ + $(GOBJECT_INTROSPECTION_CFLAGS) \ + -I. + $(NULL) + +VALAFLAGS = \ + --save-temps \ + --pkg glib-2.0 \ + --target-glib=2.32 \ + --pkg gio-2.0 \ + --pkg json-glib-1.0 \ + --pkg gee-0.8 \ + -g \ + -X -fPIC -X -shared \ + --library=libvdsm \ + --gir=vdsm-0.1.gir \ + -o libvdsm.so \ + -H vdsm.h \ + --vapi=vdsm-0.1.vapi \ + $(NULL) + +generated.vala: generate.py vdsmapi-schema.json + python generate.py + +vdsm-0.1.typelib: libvdsm-0.1.la vdsm-0.1.gir + $(G_IR_COMPILER) \ + --shared-library=libvdsm-0.1.la \ + --output=$@ \ + vdsm-0.1.gir + +vdsm-0.1.gir: libvdsm-0.1.la + +c_test: test.c + $(CC) $^ -o $@ $(AM_CFLAGS) ./.libs/libvdsm-0.1.a $(GLIB_LIBS) $(GOBJECT_LIBS) $(JSON_GLIB_LIBS) $(GEE_LIBS) + +GENERATED = \ + generated.* \ + libvdsm-base.c \ + vdsm.h \ + $(NULL) + +CLEANFILES = \ + $(GENERATED) \ + *.gir \ + *.vapi \ + *.typelib \ + *.pyc \ + *.html \ + $(NULL) diff --git a/vdsm_api/generate.py b/vdsm_api/generate.py new file mode 100644 index 0000000..c330518 --- /dev/null +++ b/vdsm_api/generate.py @@ -0,0 +1,895 @@ +# VDSM API Code Generator +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +import sys +import os +from collections import OrderedDict +sys.path.append("../vdsm_api") +import vdsmapi + + +def genindent(count): + ret = "" + for i in range(count): + ret += " " + return ret + + +def push_indent(indent_amount=4): + global indent_level + indent_level += indent_amount + + +def pop_indent(indent_amount=4): + global indent_level + indent_level -= indent_amount + + +def cgen(code, **kwds): + indent = genindent(indent_level) + lines = code.split('\n') + lines = map(lambda x: indent + x, lines) + return '\n'.join(lines) % kwds + '\n' + + +def mcgen(code, **kwds): + return cgen('\n'.join(code.split('\n')[1:-1]), **kwds) + + +def parse_args(typeinfo): + for member in typeinfo: + argname = member + argentry = typeinfo[member] + optional = False + structured = False + if member.startswith('*'): + argname = member[1:] + optional = True + if isinstance(argentry, OrderedDict): + structured = True + yield (argname, argentry, optional, structured) + +def map_type(name): + return map_types[name]['value'] + +def map_cast(name): + valType = map_type(name) + outType = vala_type(valType) + if valType in ('double', 'int', 'uint'): + outType += '?' + return 'HashTable<string, %s>' % outType + +def vala_type(name): + if name == 'str': + return 'string' + elif name == 'int': + return 'int64' + elif name == 'uint': + return 'uint64' + elif name == 'bool': + return 'bool' + elif name == 'float': + return 'double' + elif type(name) == list: + return '%s[]' % vala_type(name[0]) + elif is_enum(name): + return name + elif is_object(name): + return name + elif is_command_class(name): + return name + elif is_map(name): + mapType = vala_type(map_type(name)) + return 'HashTable<string, %s>' % mapType + elif is_alias(name): + return vala_type(aliased_types[name]) + elif is_union(name): + return name + elif name == None or len(name) == 0: + return 'void' + else: + raise ValueError ("Unhandled type: %s", name) + + +def type_as_function_name(name): + if name == 'str': + return 'string' + elif name == 'int': + return 'int' + elif name == 'uint': + return 'uint' + elif name == 'bool': + return 'boolean' + elif name == 'float': + return 'double' + elif type(name) == list: + return "%s_array" % type_as_function_name(name[0]) + elif is_enum(name): + return 'enum' + elif is_object(name):# or is_union(name): + return 'object' + elif is_union(name): + return 'union' + elif is_map(name): + mapType = type_as_function_name(map_type(name)) + return '%s_map' % mapType + elif is_alias(name): + return type_as_function_name(aliased_types[name]) + else: + raise ValueError ("Unhandled type: %s", name) + + +def vala_type_to_json_type(name): + if name in ('int', 'uint'): + return 'int' + elif name == 'float': + return 'double' + elif name == 'str': + return 'string'; + elif name == 'bool': + return 'boolean' + elif is_enum(name): + return 'int' + elif is_array(name): + return 'array' + elif is_object(name) or is_map(name): + return 'object' + elif is_command_class(name): # The object is identified by UUID + return 'string' + elif is_alias(name): + return vala_type_to_json_type(aliased_types[name]) + elif is_union(name): + return 'object' + else: + raise ValueError ("Unhandled type: %s", name) + + +def vala_var(name): + return name.replace('-', '_').lstrip("*") + + +def vala_enum_val(name): + return vala_var(name).replace(' ', '_').upper() + + +def vala_fun(name): + return vala_var(name).replace('.', '_') + + +def is_enum(name): + global enum_types + return (name in enum_types) + + +def is_object(name): + global object_types + return (name in object_types) + + +def is_array(name): + return type(name) is list + + +def is_map(name): + return not is_array(name) and (name in map_types) + + +def is_alias(name): + return (name in aliased_types) + + +def is_union(name): + return (name in union_types) + + +def is_command_class(name): + return (name in class_types.keys()) + + +def generate_errors(f, errors): + ret = cgen(''' +public errordomain VdsmError { +''') + push_indent() + for error in errors: + ret += mcgen(''' +%(error_name)s = %(error_code)i, +''', error_name=error['name'], error_code=error['code']) + pop_indent() + ret += cgen('''}''') + f.write(ret) + + +def generate_enum(f, e): + strings = "" + for s in e['data']: + strings += '"%s", ' % s + strings = strings[:-2] + + ret = mcgen(''' +const string[] %(type)s_strings = { %(strings)s}; +''', type=e['enum'], strings=strings) + + ret += mcgen(''' +public enum %(type)s { +''', type=e['enum']) + + idents = ', '.join(map(vala_enum_val, e['data'])) + ';' + for ident in idents.split(): + ret += mcgen(''' + %(ident)s +''', ident=ident) + + ret += mcgen(''' + public unowned string to_string() { + return enum_to_string((string[])%(type)s_strings, this); + } + + public static %(type)s from_string (string val) { + return (%(type)s) enum_from_string((string[])%(type)s_strings, val); + } +} +''', type=e['enum']) + f.write(ret) + + +def generate_type_constructor_field_setter(argname, argtype, optional): + ret = "" + if optional: + ret += mcgen(''' +if (%(argname)s != null) { +''', argname=argname) + push_indent() + + + ret += mcgen(''' +set_%(argname)s(%(argname)s); +''', argname=argname) + if optional: + pop_indent() + ret += cgen('''}''') + return ret + + +def marshall_arg_list(fields, retval=None): + def maybe_optional(argtype, optional): + if optional: + return argtype + '?' + return argtype + + arglist = "" + for argname, argtype, optional, structured in parse_args(fields): + if is_map(argtype): + _type = maybe_optional(map_cast(argtype), optional) + else: + _type = maybe_optional(vala_type(argtype), optional) + arglist += "%s %s, " % (_type, vala_var(argname)) + + arglist = arglist[:-2] # strip the last comma + + return arglist + + +def generate_type_constructor(name, fields): + arglist = marshall_arg_list(fields) + + ret = mcgen(''' +public %(name)s(%(args)s) { + base(); +''', name=vala_fun(name), args=arglist) + + push_indent() + for argname, argtype, optional, structured in parse_args(fields): + ret += generate_type_constructor_field_setter(argname, argtype, + optional) + pop_indent() + + ret += cgen('''}''') + + # Add the json constructor as well + ret += mcgen(''' +public %(name)s.from_json_object(owned Json.Object? obj) { + base.from_json_object(obj); +} +''', name=vala_fun(name)) + + return ret + + +def setter_value_cast(vartype): + if is_array(vartype): + return "%s[]" % setter_value_cast(vartype[0]) + elif is_object(vartype): + return "BaseType" + elif is_union(vartype): + return "BaseUnion" + elif is_enum(vartype): + return "int64" + else: + return vala_type(vartype) + + +def generate_enum_type_setter(varname, vartype): + return mcgen(''' +public void set_%(name)s(%(type)s value) { + set_string_member("%(name)s", value.to_string()); +} +''', name=varname, type=vartype) + +def generate_type_setter(varname, vartype, setVar=False): + if is_enum(vartype): + return generate_enum_type_setter(varname, vartype) + + argtype = vala_type(vartype) + fname = "set_%s_member" % type_as_function_name(vartype) + + value = "(%s) value" % setter_value_cast(vartype) + + + if setVar: + varSetter = mcgen(''' + this.%(varname)s = %(varname)s;''', varname=varname) + else: + varSetter = '' + + return mcgen(''' +public void set_%(name)s(%(type)s value) { + %(fname)s("%(name)s", %(value)s);%(varSetter)s +} +''', fname=fname, name=varname, type=argtype, value=value, varSetter=varSetter) + + +def generate_object_type_getter(varname, vartype): + # Note: Object getters may always return null even if the member is not + # optional + rettype = vala_type(vartype) + + return mcgen(''' +public %(rettype)s? get_%(name)s() { + var res = get_object_member("%(name)s"); + if (res == null) { + return null; + } + return new %(rettype)s.from_json_object(res.get_internal_object()); +} +''', rettype=rettype, name=varname) + +def generate_enum_type_getter(varname, vartype, optional): + rettype = vala_type(vartype) + if optional: + rettype += '?' + return mcgen(''' +public %(rettype)s get_%(name)s() { + return %(enum)s.from_string(get_string_member("%(name)s")); +} +''', rettype=rettype, enum=vartype, name=varname) + +def generate_map_type_getter(varname, vartype, optional): + mapType = map_type(vartype) + valaType = vala_type(mapType) + jsonType = vala_type_to_json_type(mapType) + if mapType in ('double', 'int', 'uint'): + valaType += '?' + if is_command_class(valaType): + mapObj = get_command_class_constructor_call(valaType, 'elem') + elif jsonType == 'object': + mapObj = "new %s.from_json_object(elem)" % valaType + else: + mapObj = "elem" + rettype = map_cast(vartype) + rettype += '?' + return mcgen(''' +public %(rettype)s get_%(name)s() { + var res = _obj.get_object_member("%(name)s"); + if (res == null) { + return null; + } + var ret = new HashTable<string, %(type)s>(str_hash, str_equal); + var members = res.get_members(); + for (int i = 0; i < members.length(); i++) { + var elem = res.get_%(jsonType)s_member(members.nth_data(i)); + ret.insert(members.nth_data(i), %(mapObj)s); + } + return ret; +} +''', rettype=rettype, name=varname, type=valaType, jsonType=jsonType, mapObj=mapObj) + +def generate_type_getter(varname, vartype, optional): + if is_object(vartype): + return generate_object_type_getter(varname, vartype) + elif is_enum(vartype): + return generate_enum_type_getter(varname, vartype, optional) + elif is_map(vartype): + return generate_map_type_getter(varname, vartype, optional) + else: + rettype = vala_type(vartype) + fname = "get_%s_member" % type_as_function_name(vartype) + if optional: + rettype += '?' + return mcgen(''' +public %(rettype)s get_%(name)s() { + return (%(rettype)s) %(fname)s("%(name)s"); +} +''', fname=fname, name=varname, rettype=rettype) + +def generate_type_setters_and_getters(name, fields): + ret = "" + for argname, argtype, optional, structured in parse_args(fields): + ret += generate_type_setter(argname, argtype) + ret += generate_type_getter(argname, argtype, optional) + return ret + + +def generate_type(f, symbol): + ret = mcgen(''' +public class %(name)s : BaseType { +''', name=symbol['type']) + push_indent() + + ret += generate_type_constructor(symbol['type'], symbol['data']) + ret += generate_type_setters_and_getters(symbol['type'], symbol['data']) + + pop_indent() + ret += cgen('''}''') + f.write(ret) + + +def generate_union(f, symbol): + ret = mcgen(''' +public class %(name)s : BaseUnion { +''', name=symbol['union']) + + ret += mcgen(''' +public %(name)s.from_json_object(owned Json.Object? obj) { + base.from_json_object(obj); +} +''', name=symbol['union']) + + for field in symbol['data']: + if not is_object(field): + raise ValueError("%s: Unions may only contain object types", + symbol['union']) + ret += mcgen(''' + public %(type)s to_%(type)s() { + return new %(type)s.from_json_object(get_internal_object()); + } +''', type=field) + + ret += cgen('''}''') + f.write(ret) + +def generate_handle_command_parameter(argname, argtype, optional): + ret = "" + + fname = "set_%s_member" % vala_type_to_json_type(argtype) + + # Get the real type in case it was aliased + if not is_array(argtype) and is_alias(argtype): + argtype = aliased_types[argtype] + + if is_array(argtype): + json_type = vala_type_to_json_type(argtype[0]) + if json_type == 'object': + cast = '(BaseType[])' + else: + cast = '' + arg = "json_array_from_%ss(%s%s)" % (json_type, cast, argname) + elif is_object(argtype) or is_union(argtype): + arg = "%s.get_internal_object()" % argname + elif is_map(argtype): + arg = "json_object_from_%s(%s)" % \ + (type_as_function_name(argtype), argname) + #elif is_union(argtype): + # arg = "json_object_from_union((BaseType)%s)" % argname + else: + arg = argname + if (optional): + ret += mcgen(''' +if (%(argname)s != null) { +''', argname=argname) + push_indent() + ret += mcgen(''' +object.%(fname)s("%(argname)s", %(arg)s); +''', argname=argname, fname=fname, type=vala_type(argtype), arg=arg) + if (optional): + pop_indent() + ret += cgen('''}''') + return ret + + +def get_command_class_constructor_call(commandClass, id_field=None): + params = class_types[commandClass]['data'] + paramStr = '' + for (argname, argentry, optional, structured) in parse_args(params): + paramStr += argname + ', ' + paramStr = paramStr[:-2] + + # If id_field is given, replace the xxID field in params with the indicated + # in-scope variable reference + if id_field is not None: + paramStr = paramStr.replace('%sID' % commandClass.lower(), id_field) + + return "new %s(%s)" % (commandClass, paramStr) + +def generate_command_array_ret(rettype): + valaType = vala_type(rettype) + jsonType = vala_type_to_json_type(rettype) + if is_command_class(valaType): + arrayObj = get_command_class_constructor_call(valaType, 'elem') + elif jsonType == 'object': + arrayObj = "new %s.from_json_object(elem)" % valaType + else: + arrayObj = "elem" + code = mcgen(''' + var array = res.get_array_member("result"); + var length = array.get_length(); + %(type)s[] ret = new %(type)s*[length]; + + for (int i = 0; i < length; i++) { + var elem = array.get_%(jsonType)s_element(i); + ret[i] = %(arrayObj)s; + } + return ret; +''', type=valaType, jsonType=jsonType, arrayObj=arrayObj) + return code + +# +# TODO: Merge this with generate_map_type_getter +# +def generate_command_map_ret(rettype): + valaType = vala_type(rettype) + if rettype in ('double', 'int', 'uint'): + valaType += '?' + jsonType = vala_type_to_json_type(rettype) + if is_command_class(valaType): + mapObj = get_command_class_constructor_call(valaType, 'elem') + elif jsonType == 'object': + mapObj = "new %s.from_json_object(elem)" % valaType + else: + mapObj = "elem" + code = mcgen(''' + var map = res.get_object_member("result"); + var ret = new HashTable<string, %(type)s>(str_hash, str_equal); + var members = map.get_members(); + for (int i = 0; i < members.length(); i++) { + var elem = map.get_%(jsonType)s_member(members.nth_data(i)); + ret.insert(members.nth_data(i), %(mapObj)s); + } + return ret; +}''', type=valaType, jsonType=jsonType, mapObj=mapObj) + return code + +def generate_command_ret(rettype): + getter = None + if rettype != 'void': + valaType = vala_type(rettype) + getter = 'res.get_%s_member("result")' % \ + vala_type_to_json_type(rettype) + else: + valaType = 'void' + + if is_object (valaType) or is_union (valaType): + retval = '(%s) new %s.from_json_object(%s)' % \ + (valaType, valaType, getter) + elif is_map (valaType): + retval = '(%s) %s_from_json(%s)' % \ + (valaType, type_as_function_name(valaType), getter) + elif is_command_class(valaType): + retval = get_command_class_constructor_call(valaType, getter) + elif valaType == 'void': + retval = '' + else: + retval = '(%s) %s' % (valaType, getter) + + return mcgen(''' + return %(retval)s; +''', retval=retval) + + +def generate_command(f, command): + args = command.get('data',{}) + rettype = command.get('returns', 'void') + + name = command['command']['name'] + # Overriding GObject method names must add the 'new' keyword + new = '' + if name in ('connect', 'disconnect'): + new = 'new' + + arglist = marshall_arg_list(args, rettype) + if rettype != 'void': + if is_map(rettype): + retcast = map_cast(rettype) + else: + retcast = vala_type(rettype) + else: + retcast = 'void' + + push_indent() + ret = mcgen(''' +public %(new)s %(retcast)s %(name)s(%(args)s) throws VdsmError { + var object = new Json.Object(); + var base_type = (BaseType) this; +''', new=new, retcast=retcast, name=name, args=arglist) + + push_indent() + # Set parameters for this object + ret += cgen(''' +object.set_object_member("__obj__", base_type.get_internal_object());''') + + # Set normal parmeters + for argname, argtype, optional, structured in parse_args(args): + ret += generate_handle_command_parameter(argname, argtype, optional) + pop_indent() + + # Send command + methodname = "%s.%s" % (command['command']['class'], command['command']['name']) + if rettype == 'void': + return_stmt = 'return;' + else: + return_stmt = 'throw new VdsmError.INTERNAL_ERROR' \ + '("The response contained no data");' + ret += mcgen(''' + var res = this.send_command("%(methodname)s", object); + response_check_error(res); + if (res.get_null_member("result")) + %(return_stmt)s +''', methodname=methodname, return_stmt=return_stmt) + + # return result + if is_array (rettype): + ret += generate_command_array_ret(rettype[0]) + elif is_map (rettype): + ret += generate_command_map_ret(map_type(rettype)) + else: + ret += generate_command_ret(rettype) + ret += cgen('''}''') + f.write(ret) + pop_indent() + +def generate_comand_class_constructor(name, fields): + arglist = marshall_arg_list(fields) + + ret = mcgen(''' +public %(name)s(%(arglist)s) { +''', name=vala_fun(name), arglist=arglist) + + push_indent() + ret += cgen(''' +base(conn); +this.conn = conn; +''') + for argname, argtype, optional, structured in parse_args(fields): + if argname == 'conn': + # We already dealt with 'conn' + continue + ret += generate_type_constructor_field_setter(argname, argtype, + optional) + pop_indent() + + ret += cgen('''}''') + + return ret + + +def generate_command_class(f, class_name, class_info): + ret = mcgen(''' +public class %(name)s : BaseCommandClass { +''', name=class_name) + + push_indent() + + ## + # All classes have these implicit parameters: + # conn - A vdsm Host object that contains the connection to the server + # xxID - A UUID associated with this object. It is prefixed with the + # object type to ease automatic construction of other objects from + # this one. + # + # XXX: This is broken! Auto-construction of objects is not always correct. + # Sometimes we do not have all of the xxIDs we need to properly construct an + # object. See StorageDomain.getImages for an example. + ### + ctor_args = OrderedDict() + ctor_args['conn'] = 'Host' + ctor_args['%sID' % class_name.lower()] = 'UUID' + ctor_args.update(class_info.get('data', {})) + + for argname, argtype, optional, structured in parse_args(ctor_args): + ret += mcgen(''' +private %(argtype)s %(argname)s; +''', argtype=vala_type(argtype), argname=argname) + + ret += generate_comand_class_constructor(class_name, ctor_args) + # We don't want a getter/setter for 'conn' + del ctor_args['conn'] + for argname, argtype, optional, structured in parse_args(ctor_args): + ret += generate_type_setter(argname, argtype, setVar=True) + ret += generate_type_getter(argname, argtype, optional) + + pop_indent() + f.write(ret) + + for c in class_info['commands']: + print "Command: %s" % c['command']['name'] + generate_command(f, c) + + f.write(cgen('''}''')) + + +def generate_commands(f, command_list): + # Generate the VDSM base class separately because it is special + ret = cgen(''' +public class Host : BaseCommandClass { + private Host conn; + public Host(string hostname, uint16 port, bool sync_mode) { + RPC rpc = new RPC(hostname, port, sync_mode); + base.forHost(rpc); + conn = this; + } + public new void connect() throws VdsmError { + try { + this._rpc.connect_host(); + } catch (GLib.Error e) { + throw new VdsmError.CONNECTION_ERROR(e.message); + } + } + public new void disconnect() { + this._rpc.disconnect_host(); + } + public bool is_connected() { + return !this._rpc.is_closed(); + } + public int get_fd() { + return this._rpc.get_fd(); + } + + public void process() { + this._rpc.process_incoming_messages(); + } +''') + f.write(ret) + + # Generate the commands for the Host base class + for command in class_types['Host']['commands']: + print "%s.%s" % (command['command']['class'], command['command']['name']) + generate_command(f, command) + + # Close out the VDSM base class + ret = cgen('''}''') + f.write(ret) + + del class_types['Host'] + for k, v in class_types.items(): + print "Class: %s Data: %s" % (k, v.get('data')) + generate_command_class(f, k, v) + + +def generate_file_top(f): + f.write(cgen(''' +/* AUTOMATICALLY GENERATED, DO NOT MODIFY */ + +/* + * Copyright (C) 2012 Adam Litke, IBM Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + * + * Refer to the README and COPYING files for full details of the license + */ + + namespace vdsm { +''')) + + +def generate_file_bottom(f): + f.write(cgen('''} /* namespace vdsm */''')) + + +################################################################################ +indent_level = 0 +class_types = {} +enum_types = [] +object_types = ['Host',] +union_types = [] +map_types = {} +aliased_types = {} + + +errors = [ + {'code': 1, 'name': 'PROTOCOL_ERROR', 'message': ''}, + {'code': 2, 'name': 'CONNECTION_ERROR', 'message': ''}, + {'code': 3, 'name': 'OPERATION_CANCELLED', 'message': ''}, + {'code': 4, 'name': 'OPERATION_NOT_SUPPORTED', 'message': ''}, + {'code': 5, 'name': 'INTERNAL_ERROR', 'message': ''}, + {'code': 666, 'name': 'GENERAL_ERROR', 'message': ''}, +] + + +schema = 'vdsmapi-schema.json' +with open(schema) as f: + symbols = vdsmapi.parse_schema(f) + + +for s in symbols: + if 'enum' in s: + enum_types.append(s['enum']) + elif 'type' in s and isinstance(s['data'], OrderedDict): + object_types.append(s['type']) + elif 'map' in s: + map_types[s['map']] = {'key': s['key'], 'value': s['value']} + elif 'alias' in s: + aliased_types[s['alias']] = s['data'] + elif 'union' in s: + union_types.append(s['union']) + +try: + f = open("generated.vala", "w") + generate_file_top(f) + errors.extend([s for s in symbols if 'error' in s]) + types = [s for s in symbols if 'type' in s] + unions = [s for s in symbols if 'union' in s] + enums = [s for s in symbols if 'enum' in s] + commands = [s for s in symbols if 'command' in s] + classes = [s for s in symbols if 'class' in s] + generate_errors(f, errors) + + f.write("/* Enum types */\n") + for e in enums: + generate_enum(f, e) + + f.write("/* Object types */\n") + for t in types: + print t['type'] + generate_type(f, t) + + f.write("/* Union types */\n") + for u in unions: + generate_union(f, u) + + for cmd in commands: + cls = cmd['command']['class'] + if cls not in class_types: + class_types[cls] = {} + if 'commands' not in class_types[cls]: + class_types[cls]['commands'] = [cmd,] + else: + class_types[cls]['commands'].append(cmd) + for cls in classes: + name = cls['class'] + if 'data' in cls: + class_types[name]['data'] = cls['data'] + + f.write("/* Command Classes */\n") + generate_commands(f, commands) + + generate_file_bottom(f) + f.close() +except: + os.unlink("generated.vala") + raise + diff --git a/vdsm_api/gir-fixes.patch b/vdsm_api/gir-fixes.patch new file mode 100644 index 0000000..881c5f2 --- /dev/null +++ b/vdsm_api/gir-fixes.patch @@ -0,0 +1,155 @@ +--- vdsm-0.1.gir.orig 2012-08-08 07:45:24.430402843 -0500 ++++ vdsm-0.1.gir 2012-08-08 07:49:53.882407762 -0500 +@@ -919,7 +919,7 @@ + </method> + <method name="get_boolean_array_member" c:identifier="vdsm_base_type_get_boolean_array_member"> + <return-value transfer-ownership="full" allow-none="1"> +- <array length="0"> ++ <array length="1"> + <type name="gboolean" c:type="gboolean"/> + </array> + </return-value> +@@ -975,7 +975,7 @@ + </method> + <method name="get_double_array_member" c:identifier="vdsm_base_type_get_double_array_member"> + <return-value transfer-ownership="full" allow-none="1"> +- <array length="0"> ++ <array length="1"> + <type name="gdouble" c:type="gdouble"/> + </array> + </return-value> +@@ -1031,7 +1031,7 @@ + </method> + <method name="get_int_array_member" c:identifier="vdsm_base_type_get_int_array_member"> + <return-value transfer-ownership="full" allow-none="1"> +- <array length="0"> ++ <array length="1"> + <type name="glong" c:type="glong"/> + </array> + </return-value> +@@ -1087,7 +1087,7 @@ + </method> + <method name="get_string_array_member" c:identifier="vdsm_base_type_get_string_array_member"> + <return-value transfer-ownership="full" allow-none="1"> +- <array length="0"> ++ <array length="1"> + <type name="utf8" c:type="gchar*"/> + </array> + </return-value> +@@ -1143,7 +1143,7 @@ + </method> + <method name="get_object_array_member" c:identifier="vdsm_base_type_get_object_array_member"> + <return-value transfer-ownership="full" allow-none="1"> +- <array length="0"> ++ <array length="1"> + <type name="vdsm.BaseType" c:type="vdsmBaseType*"/> + </array> + </return-value> +@@ -1222,7 +1222,7 @@ + </method> + <method name="get_union_array_member" c:identifier="vdsm_base_type_get_union_array_member"> + <return-value transfer-ownership="full" allow-none="1"> +- <array length="0"> ++ <array length="1"> + <type name="vdsm.BaseUnion" c:type="vdsmBaseUnion*"/> + </array> + </return-value> +@@ -13546,7 +13546,7 @@ + </method> + <method name="getDeviceList" c:identifier="vdsm_api_getDeviceList" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="1"> + <type name="vdsm.BlockDeviceInfo" c:type="vdsmBlockDeviceInfo*"/> + </array> + </return-value> +@@ -13576,7 +13576,7 @@ + </method> + <method name="getLVMVolumeGroups" c:identifier="vdsm_api_getLVMVolumeGroups" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="1"> + <type name="vdsm.VolumeGroupInfo" c:type="vdsmVolumeGroupInfo*"/> + </array> + </return-value> +@@ -13596,7 +13596,7 @@ + </method> + <method name="getStorageDomains" c:identifier="vdsm_api_getStorageDomains" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="4"> + <type name="utf8" c:type="gchar*"/> + </array> + </return-value> +@@ -13625,7 +13625,7 @@ + </method> + <method name="getVMList" c:identifier="vdsm_api_getVMList" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="3"> + <type name="vdsm.VmDefinition" c:type="vdsmVmDefinition*"/> + </array> + </return-value> +@@ -14762,7 +14762,7 @@ + </method> + <method name="getVolumes" c:identifier="vdsm_storage_domain_getVolumes" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="1"> + <type name="utf8" c:type="gchar*"/> + </array> + </return-value> +@@ -14983,7 +14983,7 @@ + </method> + <method name="connectStorageServer" c:identifier="vdsm_storage_pool_connectStorageServer" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="3"> + <type name="vdsm.ConnectStorageServerStatus" c:type="vdsmConnectStorageServerStatus*"/> + </array> + </return-value> +@@ -15071,7 +15071,7 @@ + </method> + <method name="disconnectStorageServer" c:identifier="vdsm_storage_pool_disconnectStorageServer" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="3"> + <type name="vdsm.ConnectStorageServerStatus" c:type="vdsmConnectStorageServerStatus*"/> + </array> + </return-value> +@@ -15117,7 +15117,7 @@ + </method> + <method name="getBackedUpVmsList" c:identifier="vdsm_storage_pool_getBackedUpVmsList" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="1"> + <type name="utf8" c:type="gchar*"/> + </array> + </return-value> +@@ -15144,7 +15144,7 @@ + </method> + <method name="getDomainsContainingImage" c:identifier="vdsm_storage_pool_getDomainsContainingImage" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="2"> + <type name="utf8" c:type="gchar*"/> + </array> + </return-value> +@@ -15162,7 +15162,7 @@ + </method> + <method name="getIsoList" c:identifier="vdsm_storage_pool_getIsoList" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="1"> + <type name="utf8" c:type="gchar*"/> + </array> + </return-value> +@@ -15297,7 +15297,7 @@ + </method> + <method name="validateStorageServerConnection" c:identifier="vdsm_storage_pool_validateStorageServerConnection" throws="1"> + <return-value transfer-ownership="full"> +- <array length="0"> ++ <array length="3"> + <type name="vdsm.StorageServerConnectionValidateStatus" c:type="vdsmStorageServerConnectionValidateStatus*"/> + </array> + </return-value> diff --git a/vdsm_api/libvdsm-base.vala b/vdsm_api/libvdsm-base.vala new file mode 100644 index 0000000..ec8b368 --- /dev/null +++ b/vdsm_api/libvdsm-base.vala @@ -0,0 +1,1068 @@ +/* + * Copyright (C) 2012 Adam Litke, IBM Corporation + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program; if not, write to the Free Software + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA + * + * Refer to the README and COPYING files for full details of the license + */ +using Json; +using Gee; + +namespace vdsm { + +delegate void SyncProcess(); + +private class Request : GLib.Object { + private Json.Object? _result; + private Mutex _mtx; + private Cond _cond; + public bool finished {get; private set;} + + public Request() { + this._result = null; + this.finished = false; + } + + public bool wait(SyncProcess? fn) { + this._mtx.lock(); + try { + if (this.finished) { + return true; + } + + if (fn != null) { + this._mtx.unlock(); + fn(); + this._mtx.lock(); + return this.finished; + } else { + this._cond.wait(this._mtx); + return true; + } + } finally { + this._mtx.unlock(); + } + } + + public void set_result(owned Json.Object? res) { + this._mtx.lock(); + if (this.finished) { + return; + } + + this._result = res; + this.finished = true; + this._cond.broadcast(); + this._mtx.unlock(); + } + + public Json.Object? get_result() { + this._mtx.lock(); + try { + return this._result; + } finally { + this._mtx.unlock(); + } + } +} + +protected class RPC : GLib.Object { + public string hostname {get; private set;} + public uint16 port {get; private set;} + private uint64 max_message_len {get; set;} + private int _reqid; + private AbstractMap<int, Request> _pending_requests; + private SocketClient _sock; + private SocketConnection? _conn; + private DataOutputStream? _out; + private DataInputStream? _in; + private bool sync_mode; + + [CCode(notify = false)] + public bool closed { + get { + return this.is_closed(); + } + } + + // TODO: Support TLS + public RPC(string hostname, uint16 port, bool sync_mode) { + this.hostname = hostname; + this.port = port; + this._sock = new SocketClient(); + this._reqid = 0; + // TBD: Is 1MB enough? + this.max_message_len = (uint64) (Math.pow(2, 20)); + this._pending_requests = new HashMap<int, Request>(); + this.sync_mode = sync_mode; + } + + public bool is_closed() { + lock(this._conn) { + if (this._conn == null) { + return true; + } + + return this._conn.closed; + } + } + + + public void connect_host() throws GLib.Error { + var resolver = Resolver.get_default(); + var addresses = resolver.lookup_by_name(hostname, null); + var address = addresses.nth_data (0); + var end_point = new InetSocketAddress(address, port); + + lock(this._conn) { + lock(this._in) { + lock(this._out) { + if (this._conn != null) { + if (!this._conn.closed) { + // Already connected + return; + } + } + + this._conn = this._sock.connect(end_point); + this._out = new DataOutputStream(this._conn.output_stream); + this._in = new DataInputStream(this._conn.input_stream); + }}} + } + + public void disconnect_host() { + lock(this._pending_requests) { + lock(this._conn) { + foreach(var req in this._pending_requests) { + if (req == null) { + continue; + } + + if (req.finished) { + continue; + } + + req.set_result(null); + } + + if (this._conn == null) { + // Not connected + return; + } + + try { + this._conn.close(); + } catch { + // Stream already closed, don't announce state change + return; + } + + }} + } + + private int get_request_id() { + lock(_reqid) { + _reqid++; + var id = _reqid; + return id; + } + } + + private string _build_request(int64 id, + string methodName, Json.Object? args) { + + var gen = new Json.Generator(); + var root = new Json.Node(NodeType.OBJECT); + var object = new Json.Object(); + + root.set_object(object); + gen.set_root(root); + object.set_int_member("id", id); + object.set_string_member("methodName", methodName); + if (args == null) { + object.set_null_member("args"); + } else { + object.set_object_member("args", args); + } + + return gen.to_data(null); + + } + + public Json.Object? send_command(string methodName, Json.Object? args) + throws GLib.Error, GLib.IOError { + var id = this.get_request_id(); + var json = this._build_request(id, methodName, args); + + var req_token = new Request(); + lock(this._pending_requests) { + this._pending_requests[id] = req_token; + } + + try { + this._send_message(json); + } catch (GLib.IOError e) { + lock(this._pending_requests) { + this._pending_requests.unset(id); + req_token.set_result(null); + } + + throw e; + } + + if (this.sync_mode) { + req_token.wait(process_incoming_messages); + } else { + req_token.wait(null); + } + Json.Object? res = null; + lock(this._pending_requests) { + if (req_token.finished) { + res = req_token.get_result(); + } else { + throw new VdsmError.OPERATION_CANCELLED("Operation was cancelled"); + } + + this._pending_requests.unset(id); + } + + return res; + } + + // Sends a complete message in a thread safe manner + private void _send_message(string msg) throws GLib.IOError, GLib.Error { + lock(this._out) { + if (this._out == null) { + throw new IOError.CLOSED("Not connected to host"); + } + + _out.put_uint64(msg.length); + _out.put_string(msg); + _out.flush(); + } + } + + // Reads a complete message in a thread safe manner + private string _read_message() throws GLib.Error, GLib.IOError { + lock(this._in) { + if (this._in == null) { + throw new IOError.CLOSED("Not connected to host"); + } + + var size = _in.read_uint64(); + if (size > this.max_message_len) { + // FIXME: Return error instead of disconnect so the client + // knows what happened. Also, read the message but don't store + // or parse it so we don't go out of sync. + this.disconnect_host(); + // TODO: Better error + throw new IOError.FAILED( + "Requested message size is too large"); + } + var bread = size; + var buff = new uint8[size + 1]; + // Make sure buffer is null terminated + buff[size] = '\0'; + + var ok = _in.read_all(buff, out bread); + // The only reason read_all fails without an error is if the stream + // ended. + if (!ok) { + throw new IOError.CLOSED("Connection droped unexpectedly"); + } + // Safe because we made sure it's null terminated + return (string) buff; + } + } + + public void process_incoming_messages() { + Json.Object resp; + try { + var msg = _read_message(); + resp = json_parse(msg); + } catch { + // If we didn't get a proper json object, we are either connected + // to something that isn't vdsm or we are out of sync. + this.disconnect_host(); + return; + } + + if (resp.get_null_member("id")) { + return; + // TODO: events + } + var reqId = (int) resp.get_int_member("id"); + lock(this._pending_requests) { + if (this._pending_requests.has_key(reqId)) { + var req = this._pending_requests[reqId]; + if (req != null) { + req.set_result(resp); + } + } + } + } + + public int get_fd() { + return this._conn.get_socket().fd; + } + + public Json.Object json_parse(string json) throws GLib.Error { + var parser = new Json.Parser(); + parser.load_from_data(json); + return parser.get_root().get_object(); + } +} + +public unowned string? +enum_to_string(string[] types, int val) +{ + if (val < 0 || val >= types.length) + return null; + + return types[val]; +} + +public int +enum_from_string(string[] types, string val) +{ + int i; + if (val == null) + return -1; + + for (i = 0 ; i < types.length ; i++) + if (types[i] == val) + return i; + + return -1; +} + +public class BaseCommandClass : vdsm.BaseType { + protected RPC _rpc; + + public BaseCommandClass(vdsm.Host conn) { + base(); + this._rpc = conn._rpc; + } + + protected BaseCommandClass.forHost(RPC rpc) { + base(); + this._rpc = rpc; + } + + public Json.Object send_command(string methodname, Json.Object? args) + throws VdsmError { + try { + return this._rpc.send_command(methodname, args); + } catch { + throw new VdsmError.PROTOCOL_ERROR( + "problem communicating with host"); + } + } +} + +VdsmError vdsm_error_from_json(Json.Object obj) { + var code = obj.get_int_member("code"); + var msg = obj.get_string_member("message"); + if (code == 1) { + return new VdsmError.PROTOCOL_ERROR(msg); + } else if (code == 2) { + return new VdsmError.CONNECTION_ERROR(msg); + } else if (code == 3) { + return new VdsmError.OPERATION_CANCELLED(msg); + } else { + return new VdsmError.GENERAL_ERROR(msg); + } +} + +void response_check_error(Json.Object res) throws VdsmError { + var errObj = res.get_object_member("error"); + if (errObj != null) { + if (errObj.get_int_member("code") != 0) + throw vdsm_error_from_json(errObj); + } +} + +public class BaseUnion : GLib.Object { + protected Json.Object _obj; + + public BaseUnion.from_json_object(owned Json.Object? obj) { + if (obj == null) { + this._obj = new Json.Object(); + } else { + this._obj = obj; + } + } + + public BaseUnion() { + this._obj = new Json.Object(); + } + + internal virtual Json.Object get_internal_object() { + return this._obj; + } +} + +public class BaseType : GLib.Object { + protected Json.Object _obj; + + private void debug_pretty_print() { + var node = new Json.Node(Json.NodeType.OBJECT); + node.set_object(get_internal_object()); + var gen = new Json.Generator(); + gen.set_root(node); + stdout.printf("%s\n", gen.to_data(null)); + } + + public BaseType.from_json_object(owned Json.Object? obj) { + base(); + if (obj == null) { + this._obj = new Json.Object(); + } else { + this._obj = obj; + } + //debug_pretty_print(); + } + + public BaseType() { + base(); + this._obj = new Json.Object(); + } + + internal Json.Object get_internal_object() { + return this._obj; + } + + public bool get_null_member(string member_name) { + return _obj.get_null_member(member_name); + } + + public bool has_member(string member_name) { + return _obj.has_member(member_name); + } + + /* + * Supported types: boolean, double, int, uint, string, BaseType (object), + * map, enum + * + * For each supported type we implement: + * - get: Return the object from the internal JSON object + * - set: Set the value in the internal JSON object + * - array_get: Retrurn an array of values from the internal JSON object + * - array_set: Set an array of valued in the internal JSON object + */ + + /* + * Boolean + */ + public bool? get_boolean_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + + /* XXX: handle evil booleans from vdsm */ + Json.Node node = _obj.get_member(member_name); + if (node.get_value_type() == typeof(string)) + return parse_evil_bool(node.get_string()); + else + return node.get_boolean(); + + /* + * This is what the method should look like with the hack removed + return _obj.get_boolean_member(member_name); + */ + } + + public void set_boolean_member(string member_name, bool value) { + _obj.set_boolean_member(member_name, value); + } + + public bool[]? get_boolean_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return boolean_array_from_json(array); + } + + public void set_boolean_array_member(string member_name, bool[] data) { + _obj.set_array_member(member_name, json_array_from_booleans(data)); + } + + private bool parse_evil_bool(string evil_bool) { + if (evil_bool == "True" || evil_bool == "true" || + evil_bool == "on" || evil_bool == "yes") + return true; + else + return false; + } + + /* + * Double + */ + public double? get_double_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + + /* XXX: handle evil floats from vdsm */ + Json.Node node = _obj.get_member(member_name); + if (node.get_value_type() == typeof(string)) + return double.parse(node.get_string()); + else + return node.get_double(); + + /* + * This is what the method should look like with the hack removed + return _obj.get_double_member(member_name); + */ + } + + public void set_double_member(string member_name, double value) { + _obj.set_double_member(member_name, value); + } + + public double[]? get_double_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return double_array_from_json(array); + } + + public void set_double_array_member(string member_name, double[] data) { + _obj.set_array_member(member_name, json_array_from_doubles(data)); + } + + /* + * Int + */ + public int64? get_int_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + + /* XXX: handle evil ints from vdsm */ + Json.Node node = _obj.get_member(member_name); + if (node.get_value_type() == typeof(string)) + return int64.parse(node.get_string()); + else + return (int64) node.get_int(); + + /* + * This is what the method should look like with the hack removed + return (int64) _obj.get_int_member(member_name); + */ + } + + public void set_int_member(string member_name, int64 value) { + _obj.set_int_member(member_name, value); + } + + public int64[]? get_int_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return int_array_from_json(array); + } + + public void set_int_array_member(string member_name, int64[] data) { + _obj.set_array_member(member_name, json_array_from_ints(data)); + } + + /* + * Uint + */ + public uint64? get_uint_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + + /* XXX: handle evil ints from vdsm */ + Json.Node node = _obj.get_member(member_name); + if (node.get_value_type() == typeof(string)) + return uint64.parse(node.get_string()); + else + return (uint64) node.get_int(); + + /* + * This is what the method should look like with the hack removed + return (uint64) _obj.get_int_member(member_name); + */ + } + + public void set_uint_member(string member_name, uint64 value) { + _obj.set_int_member(member_name, (int64)value); + } + + public uint64[]? get_uint_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return uint_array_from_json(array); + } + + public void set_uint_array_member(string member_name, uint64[] data) { + _obj.set_array_member(member_name, json_array_from_uints(data)); + } + + /* + * String + */ + public string? get_string_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + return _obj.get_string_member(member_name); + } + + public void set_string_member(string member_name, string value) { + _obj.set_string_member(member_name, value); + } + + public string[]? get_string_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return string_array_from_json(array); + } + + public void set_string_array_member(string member_name, string[] data) { + _obj.set_array_member(member_name, json_array_from_strings(data)); + } + + public HashTable<string, string>? + get_string_map_member(string member_name) { + Json.Object object; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + object = _obj.get_object_member(member_name); + return string_map_from_json(object); + } + + public void set_string_map_member(string member_name, + HashTable<string, string> map) { + _obj.set_object_member(member_name, json_object_from_string_map(map)); + } + + /* + * BaseType (object) + */ + public BaseType? get_object_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + return new BaseType.from_json_object(_obj.get_object_member(member_name)); + } + + public void set_object_member(string member_name, BaseType value) { + _obj.set_object_member(member_name, value.get_internal_object()); + } + + public BaseType[]? get_object_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return object_array_from_json(array); + } + + public void set_object_array_member(string member_name, BaseType[] data) { + _obj.set_array_member(member_name, json_array_from_objects(data)); + } + + public HashTable<string, BaseType>? + get_object_map_member(string member_name) { + Json.Object object; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + object = _obj.get_object_member(member_name); + return object_map_from_json(object); + } + + public void set_object_map_member(string member_name, + HashTable<string, BaseType> map) { + _obj.set_object_member(member_name, json_object_from_object_map(map)); + } + + /* + * Union: Unions are like BaseObjects except they can contain one of several + * different types of objects. + */ + public BaseUnion? get_union_member(string member_name) { + if (!has_member(member_name) || get_null_member(member_name)) + return null; + return new BaseUnion.from_json_object(_obj.get_object_member(member_name)); + } + + public void set_union_member(string member_name, BaseUnion value) { + _obj.set_object_member(member_name, value.get_internal_object()); + } + + public BaseUnion[]? get_union_array_member(string member_name) { + Json.Array array; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + array = _obj.get_array_member(member_name); + return union_array_from_json(array); + } + + public void set_union_array_member(string member_name, BaseUnion[] data) { + _obj.set_array_member(member_name, json_array_from_unions(data)); + } + + public HashTable<string, BaseUnion>? + get_union_map_member(string member_name) { + Json.Object object; + if (!has_member(member_name) || get_null_member(member_name)) + return null; + object = _obj.get_object_member(member_name); + return union_map_from_json(object); + } + + public void set_union_map_member(string member_name, + HashTable<string, BaseUnion> map) { + _obj.set_object_member(member_name, json_object_from_union_map(map)); + } + + /* + * Enum + */ + public int64[]? get_enum_array_member(string member_name) { + /* + * XXX: We need an instance of the proper enum to be able to call the + * from_string method + */ + return null; + } + + public void set_enum_array_member(string member_name, int64[] data) { + /* + * XXX: We need to know the type to be able to convert the ints to + * strings + */ + } + + public HashTable<string, int64?>? + get_enum_map_member(string member_name) { + /* + * XXX: We need an instance of the proper enum to be able to call the + * from_string method + */ + return null; + } + + public void set_enum_map_member(string member_name, + HashTable<string, int64?> map) { + /* + * XXX: We need to know the type to be able to convert the ints to + * strings + */ + } +} + +/* + * Convert between Vala arrays and Json arrays for the supported types + */ + +/* + * Boolean + */ +bool[] boolean_array_from_json(Json.Array array) { + var length = array.get_length(); + bool[] ret = new bool[length]; + + for (int i = 0; i < length; i++) + ret[i] = array.get_boolean_element(i); + return ret; +} + +Json.Array json_array_from_booleans(bool[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) { + array.add_boolean_element (data[i]); + } + return array; +} + +HashTable<string, bool> boolean_map_from_json(Json.Object object) { + var ret = new HashTable<string, bool>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + bool value = object.get_boolean_member(members.nth_data(i)); + ret.insert(members.nth_data(i), value); + } + return ret; +} + +Json.Object json_object_from_boolean_map(HashTable<string, bool> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + ret.set_boolean_member(key, map.get(key)); + } + return ret; +} + +/* + * Double + */ +double[] double_array_from_json(Json.Array array) { + var length = array.get_length(); + double[] ret = new double[length]; + + for (int i = 0; i < length; i++) + ret[i] = array.get_double_element(i); + return ret; +} + +Json.Array json_array_from_doubles(double[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) { + array.add_double_element (data[i]); + } + return array; +} + +HashTable<string, double?> double_map_from_json(Json.Object object) { + var ret = new HashTable<string, double?>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + double value = object.get_double_member(members.nth_data(i)); + ret.insert(members.nth_data(i), value); + } + return ret; +} + +Json.Object json_object_from_double_map(HashTable<string, double?> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + ret.set_double_member(key, map.get(key)); + } + return ret; +} + +/* + * Int + */ +int64[] int_array_from_json(Json.Array array) { + var length = array.get_length(); + int64[] ret = new int64[length]; + + for (int i = 0; i < length; i++) + ret[i] = (int64)array.get_int_element(i); + return ret; +} + +Json.Array json_array_from_ints(int64[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) { + array.add_int_element (data[i]); + } + return array; +} + +HashTable<string, int64?> int_map_from_json(Json.Object object) { + var ret = new HashTable<string, int64?>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + int64 value = object.get_int_member(members.nth_data(i)); + ret.insert(members.nth_data(i), value); + } + return ret; +} + +Json.Object json_object_from_int_map(HashTable<string, int64?> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + var val = map.get(key); + ret.set_int_member(key, val); + } + return ret; +} + +/* + * Uint + */ +uint64[] uint_array_from_json(Json.Array array) { + var length = array.get_length(); + uint64[] ret = new uint64[length]; + + for (int i = 0; i < length; i++) + ret[i] = (uint64)array.get_int_element(i); + return ret; +} + +Json.Array json_array_from_uints(uint64[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) { + array.add_int_element ((int64)data[i]); + } + return array; +} + +HashTable<string, uint64?> uint_map_from_json(Json.Object object) { + var ret = new HashTable<string, uint64?>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + uint64 value = object.get_int_member(members.nth_data(i)); + ret.insert(members.nth_data(i), value); + } + return ret; +} + +Json.Object json_object_from_uint_map(HashTable<string, uint64?> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + ret.set_int_member(key, (int64)map.get(key)); + } + return ret; +} + +/* + * String + */ +string[] string_array_from_json(Json.Array array) { + var length = array.get_length(); + string[] ret = new string[length]; + + for (int i = 0; i < length; i++) + ret[i] = array.get_string_element(i); + return ret; +} + +Json.Array json_array_from_strings(string[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) { + array.add_string_element(data[i]); + } + return array; +} + +HashTable<string, string> string_map_from_json(Json.Object object) { + var ret = new HashTable<string, string>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + string value = object.get_string_member(members.nth_data(i)); + ret.insert(members.nth_data(i), value); + } + return ret; +} + +Json.Object json_object_from_string_map(HashTable<string, string> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + ret.set_string_member(key, map.get(key)); + } + return ret; +} + +/* + * BaseType (object) + */ +BaseType[] object_array_from_json(Json.Array array) { + var length = array.get_length(); + BaseType[] ret = new BaseType*[length]; + + for (int i = 0; i < length; i++) { + Json.Object jsonobj = array.get_object_element(i); + ret[i] = new BaseType.from_json_object(jsonobj); + } + return ret; +} + +Json.Array json_array_from_objects(BaseType[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) + array.add_object_element(data[i].get_internal_object()); + return array; +} + +HashTable<string, BaseType> object_map_from_json(Json.Object object) { + var ret = new HashTable<string, BaseType>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + ret.insert(members.nth_data(i), + new BaseType.from_json_object(object.get_object_member(members.nth_data(i)))); + } + return ret; +} + +Json.Object json_object_from_object_map(HashTable<string, BaseType> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + ret.set_object_member(key, map.get(key).get_internal_object()); + } + return ret; +} + +/* + * Unions + */ +BaseUnion[] union_array_from_json(Json.Array array) { + var length = array.get_length(); + BaseUnion[] ret = new BaseUnion*[length]; + + for (int i = 0; i < length; i++) { + Json.Object jsonobj = array.get_object_element(i); + ret[i] = new BaseUnion.from_json_object(jsonobj); + } + return ret; +} + +Json.Array json_array_from_unions(BaseUnion[] data) { + var array = new Json.Array(); + for (uint i = 0; i < data.length; i++) + array.add_object_element(data[i].get_internal_object()); + return array; +} + +HashTable<string, BaseUnion> union_map_from_json(Json.Object object) { + var ret = new HashTable<string, BaseUnion>(str_hash, str_equal); + var members = object.get_members(); + for (int i = 0; i < members.length(); i++) { + var value = object.get_object_member(members.nth_data(i)); + ret.insert(members.nth_data(i), + new BaseUnion.from_json_object(value)); + } + return ret; +} + +Json.Object json_object_from_union_map(HashTable<string, BaseUnion> map) { + var ret = new Json.Object(); + var members = map.get_keys(); + for (int i = 0; i < members.length(); i++) { + var key = members.nth_data(i); + ret.set_object_member(key, map.get(key).get_internal_object()); + } + return ret; +} + +/* + * Enum + */ +Json.Object json_object_from_enum_map(HashTable<string, int64?> map) { + /* TODO */ + return new Json.Object(); +} + +} /* namespace vdsm */ diff --git a/vdsm_api/schema.json b/vdsm_api/schema.json new file mode 100644 index 0000000..5412bbe --- /dev/null +++ b/vdsm_api/schema.json @@ -0,0 +1,721 @@ +# +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +# +# API API Schema +# + +## +# @API.ping: +# +# Test connectivity to vdsm. +# +# Since: 4.10.0 +## +{'command': {'class': 'API', 'name': 'ping'}, + 'data': {}} + +# +## +# @API.pong: +# +# echoes a +# +# Since: 4.10.0 +## +{'command': {'class': 'API', 'name': 'pong'}, + 'data': {'a': 'int', '*b': 'str'} + 'returns': 'int'} + +# +## +# @API.fake: +# +# Will invoke exception +# +# Since: 4.10.0 +## +{'command': {'class': 'API', 'name': 'fake'}, + 'data': {'a': 'int', '*b': 'str'} + 'returns': 'str'} + + +{'command': {'class': 'API', 'name': 'pass'}, + 'data': {'a': 'SetupNetworkBondAttributes' }, + 'returns': ['SetupNetworkBondAttributes']} + +{'map': 'IntMap', 'key': 'str', 'value': 'int'} +{'map': 'StrMap', 'key': 'str', 'value': 'str'} + + +{'command': {'class': 'API', 'name': 'maptest'}, + 'data': {'a': 'IntMap'}, + 'returns': 'StrMap'} + +## +# @SetupNetworkBondAttributes: +# +# Configuration attributes for a single bonded network interface device. +# +# @nics: An array of network device names to include in the bond +# +# @options: #optional A string of space-separated <option>=<value> pairs +# +# @remove: #optional If True, remove existing bond only +# +# Since: 4.10.0 +## +{'type': 'SetupNetworkBondAttributes', + 'data': {'nics': ['str'], '*options': 'str', '*remove': 'bool'}} + +{'type': 'TestA', + 'data': {'foo': 'str', '*bar': 'int'}} + +{'type': 'TestB', + 'data': {'name': 'str', 'obj': 'TestA'}} + +## +# @FenceNodePowerStatus: +# +# Indicates the power state of a remote host. +# +# @on: The remote host is powered on +# +# @off: The remote host is powered off +# +# @unknown: The power status is not known +# +# Since: 4.10.0 +## +{'enum': 'FenceNodePowerStatus', 'data': ['on', 'off', 'unknown']} + +## +# @FenceNodeAction: +# +# Specifies the type of fencing operation to perform. +# +# @status: Just fetch the current power status +# +# @on: Turn on the remote host +# +# @off: Shut down the remote host +# +# @reboot: Reboot the remote host +# +# Since: 4.10.0 +## +{'enum': 'FenceNodeAction', 'data': ['status', 'on', 'off', 'reboot']} + +## +# @TaskState: +# +# An enumeration of possible task states. +# +# @unknown: The state of the task is not known +# +# @init: The task is initializing +# +# @running: The task is currently executing +# +# @finished: The task has finished executing +# +# @aborting: The task has been interrupted by a user has encountered an error +# +# @cleaning: The task has failed and recovery actions are underway +# +# Since: 4.10.0 +## +{'enum': 'TaskState', + 'data': ['unknown', 'init', 'running', 'finished', 'aborting', 'cleaning']} + +## +# @TaskResult: +# +# An enumeration of the possible final task results. +# +# @success: The task was successful +# +# @cleanSuccess: The task failed but was successfully recovered +# +# @cleanFailure: The task failed and recovery also failed +# +# Since: 4.10.0 +## +{'enum': 'TaskResult', 'data': ['success', 'cleanSuccess', 'cleanFailure']} + +## +# @OSName: +# +# An enumeration of recognized operating system names. +# +# @unknown: The operating system could not be detected +# +# @oVirt Node: oVirt Node standalone hypervisor +# +# @RHEL: Red Hat Enterprise Linux +# +# @Fedora: Fedora Linux +# +# @RHEV Hypervisor: Red Hat Enterprise Virtualization Hypervisor +# +# @Debian: A Debian-based distribution (including Ubuntu) +# +# Since: 4.10.0 +# XXX: Extension: Spaces in enum values +## +{'enum': 'OSName', + 'data': ['unknown', 'oVirt Node', 'RHEL', 'Fedora', 'RHEV Hypervisor', + 'Debian']} + +## +# @SoftwarePackage: +# +# An enumeration of aliases for important software components. +# +# @kernel: The Linux kernel +# +# @qemu-kvm: Qemu virtual machine emulator +# +# @qemu-img: Qemu VM disk image manipulation utility +# +# @vdsm: Virtual Desktop and Server Manager (this software) +# +# @spice-server: The server for SPICE protocol +# +# @libvirt: Low level virtualization API +# +# Since: 4.10.0 +## +{'enum': 'SoftwarePackage', + 'data': ['kernel', 'qemu-kvm', 'qemu-img', 'vdsm', 'spice-server', 'libvirt']} + +## +# @VmType: +# +# Enumeration of supported virtual machine types. +# +# @kvm: VM runs on Linux Kernel Virtual Machine +# +# Since: 4.10.0 +## +{'enum': 'VmType', 'data': ['kvm']} + +## +# @BlockDeviceType: +# +# Enumeration of possible block device types. +# +# @iSCSI: The device is pure iSCSI +# +# @FCP: The device is purely FibreChannel +# +# @MIXED: The device consists of a mix of iSCSI and FibreChannel paths +# +# Since: 4.10.0 +## +{'enum': 'BlockDeviceType', 'data': ['iSCSI', 'FCP', 'MIXED']} + +## +# @BlockDevicePathState: +# +# Enumeration of possible states for a block device path. +# +# @active: The path is active +# +# @failed: The path has failed +# +# Since: 4.10.0 +## +{'enum': 'BlockDevicePathState', 'data': ['active', 'failed']} + +## +# @VolumeGroupState: +# +# An enumeration of possible Volume Group states. +# +# @OK: All Physical Volumes are online +# +# @PARTIAL: One or more Physical Volumes are not available +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupState', 'data': ['OK', 'PARTIAL']} + +## +# @VolumeGroupAttributePermission: +# +# An enumeration of possible values for the permission attribute. +# +# @w: The Volume Group is writable +# +# @r: The Volume Group is read-only +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupAttributePermission', 'data': ['w', 'r']} + +## +# @VolumeGroupAttributeResizeable: +# +# An enumeration of possible values for the resizeable attribute. +# +# @-: The Volume Group is not resizeable +# +# @z: The Volume Group is resizeable +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupAttributeResizeable', 'data': ['-', 'z']} + +## +# @VolumeGroupAttributeExported: +# +# An enumeration of possible values for the exported attribute. +# +# @-: The Volume Group has not been exported +# +# @x: The Volume Group has been exported +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupAttributeExported', 'data': ['-', 'x']} + +## +# @VolumeGroupAttributePartial: +# +# An enumeration of possible values for the partial status attribute. +# +# @-: The Volume Group can access all of its Physical Volumes +# +# @p: The Volume Group cannot access some Physical Volumes +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupAttributePartial', 'data': ['-', 'p']} + +## +# @VolumeGroupAttributeAllocation: +# +# An enumeration of possible values for the allocation policy attribute. +# +# @c: Indicates a contiguous allocation policy +# +# @n: Indicates a normal allocation policy +# +# @i: Indicates that the allocation policy is inherited +# +# @l: Indicates a cling allocation policy +# +# @a: Indicates an anywhere allocation policy +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupAttributeAllocation', 'data': ['c', 'n', 'i', 'l', 'a']} + +## +# @VolumeGroupAttributeClustered: +# +# An enumeration of possible values for the clustered attribute. +# +# @-: Indicates that clustered locking is disabled +# +# @c: Indicates that clustered locking is enabled +# +# Since: 4.10.0 +## +{'enum': 'VolumeGroupAttributeClustered', 'data': ['-', 'c']} + +## +# @NetworkInterfaceState: +# +# An enumeration of possible network interface states. +# +# @up: The interface is active +# +# @down: The interface is not active +# +# Since: 4.10.0 +## +{'enum': 'NetworkInterfaceState', 'data': ['up', 'down']} + +## +# @THPStates: +# +# An enumeration of possible states for the Transparent Huge Pages feature. +# +# @always: All memory regions will be scanned +# +# @madvise: Only memory regions indicated by a call to madvise will be scanned +# +# @never: No memory regions will be scanned +# +# Since: 4.10.0 +## +{'enum': 'THPStates', 'data': ['always', 'madvise', 'never']} + +## +# @StorageDomainClass: +# +# An enumeration of Storage Domain classes. +# +# @Unknown: The class is not known +# @Data: The Storage Domain is used for virtual machine disk images +# @Iso: The Storage Domain is used for storing ISO images +# @Backup: The Storage Domain is used for import and export of disk images +# +# Since: 4.10.0 +# XXX: Need to specify explicit values for the enum +## +{'enum': 'StorageDomainClass', 'data': ['Unknown', 'Data', 'Iso', 'Backup']} + +## +# @StorageDomainType: +# +# An enumeration of Storage Domain types. +# +# @UNKNOWN: The type is not known +# +# @NFS: The Storage Domain uses Network File System based storage +# +# @FCP: The Storage Domain uses FibreChannel based storage +# +# @ISCSI: The Storage Domain uses iSCSI based storage +# +# @LOCALFS: The Storage Domain uses storage on the local file system +# +# @CIFS: The Storage Domain uses CIFS/SMB based storage +# +# @SHAREDFS: The Storage Domain uses storage from a Linux VFS file system +# +# Since: 4.10.0 +# XXX: Need to specify explicit values for the enum +## +{'enum': 'StorageDomainType', + 'data': ['UNKNOWN', 'NFS', 'FCP', 'ISCSI', 'LOCALFS', 'CIFS', 'SHAREDFS']} + +## +# @VmStatus: +# +# An enumeration of possible virtual machine statuses. +# +# @Down: The VM is powered off +# +# @Migration Destination: The VM is migrating to this host +# +# @Migration Source: The VM is migrating away from this host +# +# @Paused: The VM is paused +# +# @Powering down: A shutdown command has been sent to the VM +# +# @RebootInProgress: The VM is currently rebooting +# +# @Restoring state: The VM is waking from hibernation +# +# @Saving State: The VM is preparing for hibernation +# +# @Up: The VM is running +# +# @WaitForLaunch: The VM is being created +# +# Since: 4.10.0 +## +{'enum': 'VmStatus', + 'data': ['Down', 'Migration Destination', 'Migration Source', 'Paused', + 'Powering down', 'RebootInProgress', 'Restoring state', + 'Saving State', 'Up', 'WaitForLaunch']} + +## +# @VmDisplayType: +# +# An enumeration of VM display modes. +# +# @vnc: Graphical VM interaction is available via the VNC protocol +# +# @qxl: Graphical VM interaction is available via SPICE +# +# Since: 4.10.0 +## +{'enum': 'VmDisplayType', 'data': ['vnc', 'qxl']} + +## +# @VmDeviceType: +# +# An enumeration of VM device types. +# +# @disk: A hard disk, floppy, or cdrom device +# +# @interface: A network interface +# +# @video: A video card +# +# @sound: A sound card or PC speaker +# +# @controller: An internal controller that usually provides a bus +# +# @balloon: A memory balloon device +# +# @channel: A host-guest communication channel +# +# Since: 4.10.0 +## +{'enum': 'VmDeviceType', + 'data': ['disk', 'interface', 'video', 'sound', 'controller', 'balloon', + 'channel']} + +## +# @VmDiskDeviceType: +# +# An enumeration of VM disk device types. +# +# @disk: A hard disk drive +# +# @cdrom: A CD-ROM drive (always readonly) +# +# @floppy: A floppy disk drive +# +# Since: 4.10.0 +## +{'enum': 'VmDiskDeviceType', 'data': ['disk', 'cdrom', 'floppy']} + +## +# @VmDiskDeviceInterface: +# +# An enumeration of VM disk device interfaces. +# +# @ide: The device is connected via an emulated IDE bus +# +# @virtio: The device uses a virtio para-virtualized interconnect +# +# @fdc: The device is connected to an emulated floppy disk controller +# +# Since: 4.10.0 +## +{'enum': 'VmDiskDeviceInterface', 'data': ['ide', 'virtio', 'fdc']} + +## +# @VmDiskDeviceFormat: +# +# An enumeration of VM disk device formats. +# +# @raw: Raw data is written directly to backing storage +# +# @cow: Data is written using the Qemu qcow format +# +# Since: 4.10.0 +## +{'enum': 'VmDiskDeviceFormat', 'data': ['raw', 'cow']} + +# PCI +# IDE +# virtio-serial +## +# @VmDeviceAddressType: +# +# An enumeration of VM device address types. +# +# @pci: The address describes a location on an PCI bus +# +# @drive: The address describes a location on an IDE bus +# +# @virtio-serial: Describes a location on a virtio-serial controller +# +# Since: 4.10.0 +## +{'enum': 'VmDeviceAddressType', 'data': ['pci', 'drive', 'virtio-serial']} + +## +# @VmInterfaceDeviceType: +# +# An enumeration of VM network device types. +# +# @bridge: The device is connected via a host bridge device +# +# Since: 4.10.0 +## +{'enum': 'VmInterfaceDeviceType', 'data': ['bridge']} + +## +# @VmInterfaceDeviceModel: +# +# An enumeration of VM network device models. +# +# @pv: Alias for @virtio +# +# @virtio: Paravirtual network interface +# +# @ne2k_pci: Emulated Realtek(R) PCI NIC +# +# @i82551: Emulated Intel(R) 82551ER NIC +# +# @i82557b: Emulated Intel i82557B NIC +# +# @i82559er: Emulated Intel i82559ER NIC +# +# @rtl8139: Emulated rtl8139 NIC +# +# @e1000: Emulated Intel(R) Gigabit Ethernet NIC +# +# @pcnet: Emulated AMD(R) PC-Net II NIC +# +# Since: 4.10.0 +## +{'enum': 'VmInterfaceDeviceModel', + 'data': ['pv', 'virtio', 'ne2k_pci', 'i82551', 'i82557b', 'i82559er', + 'rtl8139', 'e1000', 'pcnet']} + +## +# @VmVideoDeviceType: +# +# An enumeration of VM video device types. +# +# @cirrus: An emulated Cirrus Logic GD5446 video card +# +# @std: An emulated standard VESA 2.0 VBE video card +# +# @vmware: A paravirtualized VGA video device from VMWare(R) +# +# @qxl: A paravirtualized video device meant for use with SPICE +# +# @none: No video device is emulated +# +# Since: 4.10.0 +## +{'enum': 'VmVideoDeviceType', + 'data': ['cirrus', 'std', 'vmware', 'qxl', 'none']} + +## +# @VmSoundDeviceType: +# +# An enumeration of VM sound device types. +# +# @ac97: An emulated Intel 82801AA AC97 Audio card +# +# @pcspk: An emulated PC speaker +# +# @sb16: An emulated Creative Labs Sound Blaster 16 card +# +# @es1370: An emulated ENSONIQ AudioPCI ES1370 card +# +# @hda: An emulated Intel High-Definition Audio card +# +# @ich6: An emulated Intel ICH6 card +# +# Since: 4.10.0 +## +{'enum': 'VmSoundDeviceType', + 'data': ['ac97', 'pcspk', 'sb16', 'es1370', 'hda', 'ich6']} + +## +# @VmControllerDeviceType: +# +# An enumeration of VM controller device types. +# +# @ide: An IDE controller +# +# @fdc: A floppy disk controller +# +# @scsi: A SCSI controller +# +# @sata: A Serial ATA controller +# +# @usb: A USB controller +# +# @ccid: A ccid (smart card) controller +# +# @virtio-serial: A virtio-serial controller +# +# Since: 4.10.0 +## +{'enum': 'VmControllerDeviceType', + 'data': ['ide', 'fdc', 'scsi', 'sata', 'usb', 'ccid', 'virtio-serial']} + +## +# @VmBalloonDeviceType: +# +# An enumeration of VM balloon device types. +# +# @memballoon: A memory balloon +# +# Since: 4.10.0 +## +{'enum': 'VmBalloonDeviceType', 'data': ['memballoon']} + +## +# @VmBalloonDeviceModel: +# +# An enumeration of VM balloon device models. +# +# @none: Indicates that no device is present +# +# @virtio: A balloon implemented using the virtio specification +# +# Since: 4.10.0 +## +{'enum': 'VmBalloonDeviceModel', 'data': ['none', 'virtio']} + +## +# @VmChannelDeviceType: +# +# An enumeration of VM channel device types. +# +# @unix: The host end of the channel is a unix socket +# +# @spicevmc: This channel is used for SPICE communication +# +# @virtio-serial: This channel supports a virtio-serial connection +# +# Since: 4.10.0 +## +{'enum': 'VmChannelDeviceType', 'data': ['unix', 'spicevmc']} + +## +# @LoggingLevel: +# +# An enumeration of logging verbosity levels. +# +# @DEBUG: Log everything (including debugging messages) +# +# @INFO: Log informational messages and anything more severe +# +# @WARNING: Log warning messages and anything more severe +# +# @ERROR: Log error messages and anything more severe +# +# @CRITICAL: Log only critical messages +# +# Since: 4.10.0 +# XXX: These need to map to specific integers +## +{'enum': 'LoggingLevel', + 'data': ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']} + +## +# @IscsiCredentialsType: +# +# An enumeration of ISCSI login credentials types. +# +# @chap: Challenge-Handshake Authentication Protocol +# +# Since: 4.10.0 +## +{'enum': 'IscsiCredentialsType', 'data': ['chap']} + + diff --git a/vdsm_api/test.c b/vdsm_api/test.c new file mode 100644 index 0000000..74970b0 --- /dev/null +++ b/vdsm_api/test.c @@ -0,0 +1,131 @@ +#include <inttypes.h> +#include <stdlib.h> +#include <stdio.h> +#include "glib.h" +#include "json-glib/json-glib.h" + +#include "vdsm.h" + +void check_error(const char *operation, GError **error, int exit_on_fail) { + if (*error) { + int code = (*error)->code; + printf ("%s failed: %s\n", operation, (*error)->message); + g_error_free (*error); + error = NULL; + if (exit_on_fail) + exit(code); + } +} + +void test_capabilities(vdsmHost *host) { + printf ("test_capabilities()\n"); + + vdsmVdsmCapabilities *caps; + GError *error = NULL; + + caps = vdsm_host_getCapabilities(host, &error); + check_error("API.getCapabilities", &error, 1); + int kvmEnabled = vdsm_vdsm_capabilities_get_kvmEnabled(caps); + + printf ("\tI am talking to vdsm version: %s.%s\n", + vdsm_vdsm_capabilities_get_software_version(caps), + vdsm_vdsm_capabilities_get_software_revision(caps)); + printf ("\tKVM enabled: %s\n", kvmEnabled? "True" : "False"); + g_object_unref (caps); +} + +void test_stats(vdsmHost *host) { + printf ("test_stats()\n"); + + vdsmHostStats *stats; + GError *error = NULL; + + stats = vdsm_host_getStats(host, &error); + check_error("API.getStats", &error, 1); + + printf("\tHost CPU Load: %f\n", vdsm_host_stats_get_cpuLoad(stats)); + printf ("\tTransparent huge pages status: %s\n", + vdsm_thp_states_to_string(vdsm_host_stats_get_thpState(stats))); + g_object_unref (stats); +} + +void test_vms(vdsmHost *host) { + printf ("test_vms()\n"); + + GError *error = NULL; + int i, length; + gboolean fullstatus = TRUE; + vdsmVM **ret; + ret = vdsm_host_getVMs(host, &length, &error); + check_error("Host.getVMs", &error, 1); + + printf ("\tThere are %i active VMs\n", length); + for (i = 0; i < length; i++) { + vdsmVmFullStatus *status; + status = vdsm_vm_getInfo(ret[i], &error); + check_error("vmList.getInfo", &error, 1); + printf ("\t%s: %s\n", vdsm_vm_full_status_get_vmName(status), + vdsm_vm_status_to_string(vdsm_vm_full_status_get_status(status))); + g_object_unref (status); + g_object_unref (ret[i]); + } + + g_free (ret); +} + +void test_walk_images(vdsmStorageDomain *domain) { + GError *error = NULL; + int i, length; + vdsmImage **images; + + images = vdsm_storage_domain_getImages(domain, &length, &error); + check_error("StorageDomain.getImages", &error, 1); + for (i = 0; i < length; i++) { + printf (" Image: %s\n", vdsm_image_get_imageID(images[i])); + g_object_unref (images[i]); + } + g_free (images); +} + +void test_walk_storagedomains(vdsmHost *host) { + printf ("test_walk_storagedomains()\n"); + GError *error = NULL; + int i, length; + vdsmStorageDomain **domains; + + domains = vdsm_host_getStorageDomains(host, NULL, NULL, NULL, NULL, + &length, &error); + check_error("Host.getStorageDomains", &error, 1); + for (i = 0; i < length; i++) { + vdsmStorageDomainInfo *domainInfo; + domainInfo = vdsm_storage_domain_getInfo(domains[i], &error); + check_error("StorageDomain.getInfo", &error, 1); + printf (" Domain '%s':\n", + vdsm_storage_domain_info_get_name(domainInfo)); + test_walk_images (domains[i]); + g_object_unref (domainInfo); + g_object_unref (domains[i]); + } + + g_free (domains); +} + +int main(int argc, char **argv) { + g_type_init(); + + g_assert (argc == 3); + char *addr = argv[1]; + int port = atoi(argv[2]); + + vdsmHost *host = vdsm_host_new (addr, port, TRUE); + vdsm_host_connect (host, NULL); + + test_capabilities(host); + test_stats(host); + test_vms(host); + test_walk_storagedomains(host); + + vdsm_host_disconnect (host); + g_object_unref (host); + +} diff --git a/vdsm_api/test.py b/vdsm_api/test.py new file mode 100644 index 0000000..e84aea5 --- /dev/null +++ b/vdsm_api/test.py @@ -0,0 +1,101 @@ +# VDSM API Python Test Client +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA + +from gi.repository import vdsm +from gi.repository import GLib +from gi.repository import GObject +from gi.repository import Json +import select +import threading + +# Don't forget to turn this on or the GIL will not be released on FFI calls. +# I lurned this the hard way!!! +GLib.threads_init() + + +def manage_host(host): + while True: + select.select([host.get_fd()], [], [], 1) + if not host.is_connected(): + return + + host.process() + + if not host.is_connected(): + return + +def json_to_map(j): + parser = Json.Parser.new() + parser.load_from_data(j, len(j)) + node = parser.get_root() + return node.dup_object() + +def map_to_json(m): + node = Json.Node.new(Json.NodeType.OBJECT) + node.set_object(m) + gen = Json.Generator.new() + gen.set_root(node) + return gen.to_data()[0] + +def test_capabilities(host): + print "test_capabilities()" + caps = host.getCapabilities() + print "\tI am talking to vdsm version: %s.%s" % \ + (caps.get_software_version(), caps.get_software_revision()) + print "\tKVM enabled: %s" % caps.get_kvmEnabled() + +def test_getstoragedomains(host): + print "test_getstoragedomains()" + sdList = host.getStorageDomains(None, 0, 0, None) + for sd in sdList: + info = sd.getInfo() + stats = sd.getStats() + print "\t%s (%s) has %lu space free out of %lu" % \ + (info.get_name(), info.get_uuid(), + stats.get_diskfree(), stats.get_disktotal()) + +def test_stats(host): + print "test_stats()" + stats = host.getStats() + print "\tHost CPU Load: %f" % stats.get_cpuLoad() + print "\tTransparent huge pages status: %s" % \ + stats.get_thpState().value_nick + # XXX: bindings do not seem to support the to_string method for enums + # stats.get_thpState().to_string() + +def test_vms(host): + print "test_vms()" + vms = host.getVMs() + print "\tThere are %i active VMs" % len(vms) + #print vms + for vm in vms: + info = vm.getInfo() + print "\t%s: %s" % (info.get_vmName(), + info.get_status().value_nick) + + +host = vdsm.Host.new("127.0.0.1", 4444, False) +host.connect() +t = threading.Thread(target=manage_host, args=(host,)) +t.setDaemon(True) +t.start() + +test_capabilities(host) +test_getstoragedomains(host) +test_stats(host) +test_vms(host) + +host.disconnect() diff --git a/vdsm_api/test_serv.py b/vdsm_api/test_serv.py new file mode 100644 index 0000000..d542241 --- /dev/null +++ b/vdsm_api/test_serv.py @@ -0,0 +1,37 @@ +# VDSM API Test Server +# Copyright (C) 2012 Adam Litke, IBM Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, but +# WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# General Public License for more details. +# +# You should have received a copy of the GNU General Public +# License along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA +import logging +import time +from BindingJsonRpc import BindingJsonRpc +from Bridge import TestBridge + +def setup_logging(): + FORMAT = "%(message)s" + logging.basicConfig(format=FORMAT) + logger = logging.getLogger('test') + logger.setLevel(logging.DEBUG) + return logger + +log = setup_logging() +server = BindingJsonRpc(TestBridge(), log, '127.0.0.1', 4444) +server.start() + +while True: + try: + time.sleep(1) + except KeyboardInterrupt: + server.prepareForShutdown() + break diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 1fc7585..e576fb0 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -942,6 +942,9 @@ {'command': {'class': 'Host', 'name': 'getConnectedStoragePools'}, 'returns': ['UUID']} +{'command': {'class': 'Host', 'name': 'getStoragePools'}, + 'returns': ['StoragePool']} + ## # @BlockDeviceType: # @@ -1559,13 +1562,13 @@ # # Get a list of known Storage Domain UUIDs. # -# @spUUID: #optional Limit to Domains belonging to this Storage Pool +# @storagepoolID: #optional Limit to Domains belonging to this Storage Pool # -# @domainClass: #optional Limit to Domains of this @StorageDomainImageClass +# @domainClass: #optional Limit to Domains of this @StorageDomainImageClass # -# @storageType: #optional Limit to Domains of this @StorageDomainType +# @storageType: #optional Limit to Domains of this @StorageDomainType # -# @remotePath: #optional Limit to Domains having this remotePath +# @remotePath: #optional Limit to Domains having this remotePath # # Returns: # A list of known Storage Domains @@ -1573,9 +1576,9 @@ # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getStorageDomains'}, - 'data': {'*spUUID': 'UUID', '*domainClass': 'StorageDomainImageClass', + 'data': {'*storagepoolID': 'UUID', '*domainClass': 'StorageDomainImageClass', '*storageType': 'StorageDomainType', '*remotePath': 'str'}, - 'returns': ['UUID']} + 'returns': ['StorageDomain']} ## # @StorageDomainStatsMap: @@ -2370,8 +2373,14 @@ # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getVMList'}, - 'data': {'*fullStatus': 'bool', '*vmList': ['UUID']}, + 'data': {'fullStatus': 'bool', 'vmList': ['UUID']}, 'returns': ['VmDefinition']} + +{'command': {'class': 'Host', 'name': 'getVMs'}, + 'returns': ['VM']} + +{'command': {'class': 'VM', 'name': 'getInfo'}, + 'returns': 'VmFullStatus'} ## # @Host.ping: @@ -2748,18 +2757,19 @@ # # Image API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: The UUID of the Image +# @imageID: The UUID of the Image # -# @spUUID: The UUID of the Storage Pool associated with the Image +# @storagepoolID: The UUID of the Storage Pool associated with the Image # -# @sdUUID: The UUID of the Storage Domain associated with the Image +# @storagedomainID: The UUID of the Storage Domain associated with the Image # # Since: 4.10.0 ## {'class': 'Image', - 'data': {'conn': 'Host', 'UUID': 'UUID', 'spUUID': 'UUID', 'sdUUID': 'UUID'}} + 'data': {'conn': 'Host', 'imageID': 'UUID', 'storagepoolID': 'UUID', + 'storagedomainID': 'UUID'}} ## # @Image.delete: @@ -2805,12 +2815,13 @@ # Get a list of Volumes associated with this Image. # # Returns: -# A list of Image UUIDs +# A list of Images # # Since: 4.10.0 ## {'command': {'class': 'Image', 'name': 'getVolumes'}, - 'returns': ['UUID']} + 'returns': ['Volume']} + ## # @Image.mergeSnapshots: @@ -2878,11 +2889,13 @@ # # @conn: A connected base API object # -# @UUID: #optional Associate this object with an existing LVM Volume Group +# @lvmvolumegroupID: #optional Associate this object with an existing LVM +# Volume Group # # Since: 4.10.0 ## -{'class': 'LVMVolumeGroup', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'LVMVolumeGroup', + 'data': {'conn': 'Host', 'lvmvolumegroupID': 'UUID'}} ## # @LVMVolumeGroup.create: @@ -2935,16 +2948,17 @@ # # StorageDomain API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with a new or existing Storage Domain +# @storagedomainID: Associate this object with a new or existing Storage Domain # -# @spUUID: #optional The Storage Pool UUID if this Storage Domain is attached +# @storagepoolID: #optional The Storage Pool UUID if this Storage Domain is +# attached # # Since: 4.10.0 ## {'class': 'StorageDomain', - 'data': {'conn': 'Host', 'UUID': 'UUID', '*spUUID': 'UUID'}} + 'data': {'conn': 'Host', 'storagedomainID': 'UUID', '*storagepoolID': 'UUID'}} ## # @StorageDomain.activate: @@ -2960,12 +2974,12 @@ # # Attach a Storage Domain to a Storage Pool. # -# @spUUID: The Storage Pool to which the Storage Domain should be attached +# @storagepoolID: The Storage Pool to which the Storage Domain should be attached # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'attach'}, - 'data': {'spUUID': 'UUID'}} + 'data': {'storagepoolID': 'UUID'}} ## # @StorageDomainCreateArgumentsBlock: @@ -3145,12 +3159,13 @@ # Get a list of Images associated with this Storage Domain. # # Returns: -# An array of Image UUIDs +# A list of Images # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'getImages'}, - 'returns': ['UUID']} + 'returns': ['Image']} + ## # @StorageDomainRole: @@ -3252,16 +3267,16 @@ # # Get a list of Volumes contained within a Storage Domain. # -# @imgUUID: Limit results to Volumes associated with a single Image +# @imageID: Limit results to Volumes associated with a single Image # # Returns: -# A list of Volume UUIDs +# A list of Volumes # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'getVolumes'}, - 'data': {'imgUUID': 'UUID'}, - 'returns': ['UUID']} + 'data': {'imageID': 'UUID'}, + 'returns': ['Volume']} ## # @StorageDomain.setDescription: @@ -3293,9 +3308,9 @@ # # Upload a Volume into a Storage Domain from a remote location. # -# @imgUUID: The UUID of the image that is associated with the Volume +# @imageID: The UUID of the image that is associated with the Volume # -# @volUUID: The UUID of an existing Volume where the data will be uploaded +# @volumeID: The UUID of an existing Volume where the data will be uploaded # # @srcPath: The remote location of the Volume data. Must be in a format that # can be understood by the command indicated in @method. @@ -3307,7 +3322,7 @@ # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'uploadVolume'}, - 'data': {'imgUUID': 'UUID', 'volUUID': 'UUID', 'srcPath': 'str', + 'data': {'imageID': 'UUID', 'volumeID': 'UUID', 'srcPath': 'str', 'size': 'int', 'method': 'UploadVolumeMethod'}} ## @@ -3325,13 +3340,13 @@ # # StoragePool API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with a new or existing Storage Pool +# @storagepoolID: Associate this object with a new or existing Storage Pool # # Since: 4.10.0 ## -{'class': 'StoragePool', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'StoragePool', 'data': {'conn': 'Host', 'storagepoolID': 'UUID'}} ## # @StoragePool.connect: @@ -3536,7 +3551,7 @@ # # Get information about backed-up virtual machines from a Backup Storage Domain. # -# @sdUUID: The UUID of the Backup Storage Domain to check +# @storagedomainID: The UUID of the Backup Storage Domain to check # # @vmList: Limit results to a list of VM UUIDs # @@ -3549,7 +3564,7 @@ # previously injected it into the Backup Storage Domain. ## {'command': {'class': 'StoragePool', 'name': 'getBackedUpVmsInfo'}, - 'data': {'sdUUID': 'UUID', 'vmList': ['UUID']}, + 'data': {'storagedomainID': 'UUID', 'vmList': ['UUID']}, 'returns': 'OVFMap'} ## @@ -3557,7 +3572,7 @@ # # Get a list of backed up virtual machines from a Backup Storage Domain. # -# @sdUUID: The UUID of the Backup Storage Domain to check +# @storagedomainID: The UUID of the Backup Storage Domain to check # # Returns: # A list of VM UUIDs @@ -3565,7 +3580,7 @@ # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'getBackedUpVmsList'}, - 'data': {'sdUUID': 'UUID'}, + 'data': {'storagedomainID': 'UUID'}, 'returns': ['UUID']} ## @@ -3586,18 +3601,18 @@ # # Get a list of Storage Domains that contain an Image. # -# @imgUUID: The UUID of the Image to search for +# @imageID: The UUID of the Image to search for # # @onlyDataDomains: #optional Only include Data Storage Domains # # Returns: -# A list of Storage Domain UUIDs that contain the Image +# A list of Storage Domains that contain the Image # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'getDomainsContainingImage'}, - 'data': {'imgUUID': 'UUID', '*onlyDataDomains': 'bool'}, - 'returns': ['UUID']} + 'data': {'imageID': 'UUID', '*onlyDataDomains': 'bool'}, + 'returns': ['StorageDomain']} ## # @StoragePool.getIsoList: @@ -4001,12 +4016,12 @@ # # @vmList: A list of virtual machine definitions to store # -# @sdUUID: The Storage Domain to use for storing the VM definitions +# @storagedomainID: The Storage Domain to use for storing the VM definitions # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'updateVMs'}, - 'data': {'vmList': ['UpdateVmDefinition'], 'sdUUID': 'UUID'}} + 'data': {'vmList': ['UpdateVmDefinition'], 'storagedomainID': 'UUID'}} ## # @StoragePool.removeVM: @@ -4015,12 +4030,12 @@ # # @vmUUID: Remove the saved definition of the VM with this UUID # -# @sdUUID: The Storage Domain where the VM is stored +# @storagedomainID: The Storage Domain where the VM is stored # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'removeVM'}, - 'data': {'vmUUID': 'UUID', 'sdUUID': 'UUID'}} + 'data': {'vmUUID': 'UUID', 'storagedomainID': 'UUID'}} ## Category: @Task ############################################################# ## @@ -4028,13 +4043,13 @@ # # Task API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with an existing Task +# @taskID: Associate this object with an existing Task # # Since: 4.10.0 ## -{'class': 'Task', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'Task', 'data': {'conn': 'Host', 'taskID': 'UUID'}} ## # @Task.clear: @@ -4097,11 +4112,11 @@ # # @conn: A connected base API object # -# @UUID: Associate this object with an existing VM +# @vmID: Associate this object with an existing VM # # Since: 4.10.0 ## -{'class': 'VM', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'VM', 'data': {'conn': 'Host', 'vmID': 'UUID'}} ## # @DriveSpecVolume: @@ -5109,21 +5124,21 @@ # # Volume API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: The UUID of the Volume +# @volumeID: The UUID of the Volume # -# @spUUID: The Storage Pool associated with @UUID +# @storagepoolID: The Storage Pool associated with @UUID # -# @sdUUID: The Storage Domain associated with @UUID +# @storagedomainID: The Storage Domain associated with @UUID # -# @imgUUID: The Image associated with @UUID +# @imageID: The Image associated with @UUID # # Since: 4.10.0 ## {'class': 'Volume', - 'data': {'conn': 'Host', 'UUID': 'UUID', 'spUUID': 'UUID', 'sdUUID': 'UUID', - 'imgUUID': 'UUID'}} + 'data': {'conn': 'Host', 'volumeID': 'UUID', 'storagepoolID': 'UUID', + 'storagedomainID': 'UUID', 'imageID': 'UUID'}} ## # @VolumeRole: -- To view, visit http://gerrit.ovirt.org/7516 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If6bd34700b86aa84c7e289f02c0e9f2ac6fcba63 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

4 9

Change in vdsm[master]: libvirtvm: Cast text to string when creating text node
by agl＠us.ibm.com 11 Jan '13

11 Jan '13

Adam Litke has uploaded a new change for review. Change subject: libvirtvm: Cast text to string when creating text node ...................................................................... libvirtvm: Cast text to string when creating text node When creating a text node for a libvirt domain xml file, the content of the node is expected to be a string. To avoid errors in the xml file, cast the parameter to a string representation when creating the child node. Change-Id: Id22db83062eaf85645c2d61c83f3f20943918413 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/32/7832/1 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 3306cc2..eaf899d 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -542,7 +542,7 @@ def appendChildWithText(self, childName, text): childNode = self.doc.createElement(childName) - textNode = self.doc.createTextNode(text) + textNode = self.doc.createTextNode(str(text)) childNode.appendChild(textNode) self.dom.appendChild(childNode) -- To view, visit http://gerrit.ovirt.org/7832 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id22db83062eaf85645c2d61c83f3f20943918413 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

5 7

Change in vdsm[master]: Move gluster/{cli.py, exception.py} to vdsm rpm
by deepakcs＠linux.vnet.ibm.com 10 Jan '13

10 Jan '13

Deepak C Shetty has uploaded a new change for review. Change subject: Move gluster/{cli.py, exception.py} to vdsm rpm ...................................................................... Move gluster/{cli.py, exception.py} to vdsm rpm This move is being done so that vdsm code using gluster cli doesn't have the need to depend on vdsm-gluster rpm Change-Id: I079facc5d4fc7c3bfd8460731e6708fce242164e Signed-off-by: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com> --- M vdsm.spec.in 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/33/8033/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 5b1fa92..797a2ed 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -692,6 +692,8 @@ %{_datadir}/%{vdsm_name}/neterrors.py* %{_datadir}/%{vdsm_name}/respawn %{_datadir}/%{vdsm_name}/set-conf-item +%{_datadir}/%{vdsm_name}/gluster/cli.py* +%{_datadir}/%{vdsm_name}/gluster/exception.py* %{python_sitelib}/sos/plugins/vdsm.py* /lib/udev/rules.d/12-vdsm-lvm.rules /etc/security/limits.d/99-vdsm.conf @@ -929,8 +931,6 @@ %doc COPYING %{_datadir}/%{vdsm_name}/gluster/__init__.py* %{_datadir}/%{vdsm_name}/gluster/api.py* -%{_datadir}/%{vdsm_name}/gluster/cli.py* -%{_datadir}/%{vdsm_name}/gluster/exception.py* %{_datadir}/%{vdsm_name}/gluster/hostname.py* %changelog -- To view, visit http://gerrit.ovirt.org/8033 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I079facc5d4fc7c3bfd8460731e6708fce242164e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Deepak C Shetty <deepakcs(a)linux.vnet.ibm.com>

7 31

Change in vdsm[master]: Remove duplicate implementation of tobool
by wudxw＠linux.vnet.ibm.com 09 Jan '13

09 Jan '13

Mark Wu has uploaded a new change for review. Change subject: Remove duplicate implementation of tobool ...................................................................... Remove duplicate implementation of tobool This patch removes the duplicate tobool function from hooking.py, and replace the usage of it with utils.tobool. Also make hooking.py pep8 clean. Change-Id: Ie0d382dabc9df68139070e8ac10f84ba1ca3774b Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M Makefile.am M vdsm/hooking.py M vdsm_hooks/vhostmd/after_vm_destroy.py M vdsm_hooks/vhostmd/before_vm_start.py 4 files changed, 6 insertions(+), 15 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/8446/1 diff --git a/Makefile.am b/Makefile.am index 7548f42..26ce956 100644 --- a/Makefile.am +++ b/Makefile.am @@ -59,6 +59,7 @@ vdsm/gluster/exception.py \ vdsm/gluster/hostname.py \ vdsm/guestIF.py \ + vdsm/hooking.py \ vdsm/hooks.py \ vdsm/libvirtev.py \ vdsm/libvirtvm.py \ diff --git a/vdsm/hooking.py b/vdsm/hooking.py index a324311..714e046 100644 --- a/vdsm/hooking.py +++ b/vdsm/hooking.py @@ -42,21 +42,10 @@ from storage.misc import execCmd execCmd # make pyflakes happy -def tobool(s): - """Convert the argument into a boolean""" - try: - if s == None: - return False - if type(s) == bool: - return s - if s.lower() == 'true': - return True - return bool(int(s)) - except: - return False def read_domxml(): return minidom.parseString(file(os.environ['_hook_domxml']).read()) + def write_domxml(domxml): file(os.environ['_hook_domxml'], 'w').write(domxml.toxml(encoding='utf-8')) diff --git a/vdsm_hooks/vhostmd/after_vm_destroy.py b/vdsm_hooks/vhostmd/after_vm_destroy.py index c129ded..7c505de 100644 --- a/vdsm_hooks/vhostmd/after_vm_destroy.py +++ b/vdsm_hooks/vhostmd/after_vm_destroy.py @@ -21,7 +21,7 @@ import os import subprocess -import hooking +from vdsm import utils from vdsm import vdscli s = vdscli.connect() @@ -30,6 +30,6 @@ if res['status']['code'] == 0: if not [v for v in res['vmList'] if v.get('vmId') != os.environ.get('vmId') and - hooking.tobool(v.get('custom', {}).get('sap_agent', False))]: + utils.tobool(v.get('custom', {}).get('sap_agent', False))]: subprocess.call(['/usr/bin/sudo', '-n', '/sbin/service', 'vhostmd', 'stop']) diff --git a/vdsm_hooks/vhostmd/before_vm_start.py b/vdsm_hooks/vhostmd/before_vm_start.py index 47d0d0d..7e38a7f 100644 --- a/vdsm_hooks/vhostmd/before_vm_start.py +++ b/vdsm_hooks/vhostmd/before_vm_start.py @@ -22,9 +22,10 @@ import os import subprocess import hooking +from vdsm import utils -if hooking.tobool(os.environ.get('sap_agent', False)): +if utils.tobool(os.environ.get('sap_agent', False)): domxml = hooking.read_domxml() subprocess.call(['/usr/bin/sudo', '-n', '/sbin/service', 'vhostmd', -- To view, visit http://gerrit.ovirt.org/8446 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie0d382dabc9df68139070e8ac10f84ba1ca3774b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

4 9

Change in vdsm[master]: WIP storage: Alert for using vdsm.conf nfs_mount_options
by gpadgett＠redhat.com 07 Jan '13

07 Jan '13

Greg Padgett has uploaded a new change for review. Change subject: WIP storage: Alert for using vdsm.conf nfs_mount_options ...................................................................... WIP storage: Alert for using vdsm.conf nfs_mount_options This adds a framework and an alert for when nfs_mount_options is used. The nfs_mount_options setting is deprecated, so if it is set, send an alert to the engine when it calls getVdsCapabilities. A mechanism to send the alert was placed here in particular because it the interval in which the engine calls this API most closely matches the interval in which the problem can be found and/or corrected. Note that this was originally for bug 826921, but a new bug will be created specifically for the alerting code in subsequent patches. Change-Id: I8520238e0b41eb95d035128681dc3b71a953e644 Bug-Url: https://bugzilla.redhat.com/826921 Signed-off-by: Greg Padgett <gpadgett(a)redhat.com> --- M Makefile.am M vdsm.spec.in M vdsm/caps.py M vdsm/storage/Makefile.am M vdsm/storage/storage_exception.py M vdsm_api/vdsmapi-schema.json 6 files changed, 42 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/79/8979/1 diff --git a/Makefile.am b/Makefile.am index 2850256..3d7cb2c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -68,6 +68,7 @@ vdsm/parted_utils.py \ vdsm/rest/BindingREST.py \ vdsm/rest/Controller.py \ + vdsm/storage/alerts.py \ vdsm/storage/blockVolume.py \ vdsm/storage/devicemapper.py \ vdsm/storage/domainMonitor.py \ diff --git a/vdsm.spec.in b/vdsm.spec.in index 37f37a3..7e35e4b 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -643,6 +643,7 @@ %{_libexecdir}/%{vdsm_name}/ovirt_functions.sh %{_libexecdir}/%{vdsm_name}/vdsm-gencerts.sh %{_datadir}/%{vdsm_name}/storage/__init__.py* +%{_datadir}/%{vdsm_name}/storage/alerts.py* %{_datadir}/%{vdsm_name}/storage/blockSD.py* %{_datadir}/%{vdsm_name}/storage/blockVolume.py* %{_datadir}/%{vdsm_name}/storage/devicemapper.py* diff --git a/vdsm/caps.py b/vdsm/caps.py index cfe4caf..c33e23f 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -40,6 +40,7 @@ import hooks from vdsm import utils import storage.hba +from storage.alerts import getStorageAlerts # For debian systems we can use python-apt if available try: @@ -295,9 +296,16 @@ config.getint('vars', 'extra_mem_reserve')) caps['guestOverhead'] = config.get('vars', 'guest_ram_overhead') + caps['alerts'] = _getAlerts() + return caps +def _getAlerts(): + # Only storage alerts are currently propagated + return getStorageAlerts() + + def _getIfaceByIP(addr, fileName='/proc/net/route'): remote = struct.unpack('I', socket.inet_aton(addr))[0] for line in itertools.islice(file(fileName), 1, None): diff --git a/vdsm/storage/Makefile.am b/vdsm/storage/Makefile.am index cff09be..96cc368 100644 --- a/vdsm/storage/Makefile.am +++ b/vdsm/storage/Makefile.am @@ -23,6 +23,7 @@ vdsmstoragedir = $(vdsmdir)/storage dist_vdsmstorage_PYTHON = \ __init__.py \ + alerts.py \ blockSD.py \ blockVolume.py \ devicemapper.py \ diff --git a/vdsm/storage/storage_exception.py b/vdsm/storage/storage_exception.py index 54e64e6..9275ae7 100644 --- a/vdsm/storage/storage_exception.py +++ b/vdsm/storage/storage_exception.py @@ -174,6 +174,17 @@ message = "Directory cleanup failure" +class MiscDeprecatedNFSOptions(StorageException): + def __init__(self, options): + self.value = "options=%s" % (options,) + code = 2009 + message = "Using deprecated nfs_mount_options parameter from vdsm.conf."\ + " This parameter will continue to be supported in versions 3.x,"\ + " but may be removed in a future version of oVirt. Please"\ + " upgrade domains to V3 or greater and set the parameters from"\ + " the GUI to ensure compatibility with future versions." + + ################################################# # Volumes Exceptions ################################################# diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 5f0b391..5c7cc92 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -839,6 +839,22 @@ 'key': 'SoftwarePackage', 'value': 'SoftwarePackageInfo'} ## +# @Alert: +# +# Alert information for this Host. +# +# @code: The alert code +# +# @message: A detailed alert string +# +# @value: #optional Detailed alert parameters +# +# Since: 4.10.1 +## +{'type': 'Alert', + 'data': {'code': 'int', 'message': 'str', '*value': 'str'}} + +## # @VdsmCapabilities: # # Host information and capabilities. @@ -900,6 +916,8 @@ # @netConfigDirty: Indicates if there are uncommitted changes to the # network configuration # +# @alerts: A list of alerts for the host +# # Since: 4.10.0 # # Notes: Since ovirt-engine cannot parse software versions in 'x.y.z' format, @@ -919,7 +937,8 @@ 'emulatedMachines': ['str'], 'ISCSIInitiatorName': 'str', 'HBAInventory': 'HbaInventory', 'vmTypes': ['VmType'], 'memSize': 'uint', 'reservedMem': 'uint', - 'guestOverhead': 'uint', 'netConfigDirty': 'bool'}} + 'guestOverhead': 'uint', 'netConfigDirty': 'bool', + 'alerts': ['Alert']}} ## # @Host.getCapabilities: -- To view, visit http://gerrit.ovirt.org/8979 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8520238e0b41eb95d035128681dc3b71a953e644 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Greg Padgett <gpadgett(a)redhat.com>

4 6

Change in vdsm[master]: [WIP] Fix mounts parsing
by smizrahi＠redhat.com 03 Jan '13

03 Jan '13

Saggi Mizrahi has uploaded a new change for review. Change subject: [WIP] Fix mounts parsing ...................................................................... [WIP] Fix mounts parsing Change-Id: I8fdeda5f2900d818912ef5a939b60687462d5344 --- M vdsm/storage/mount.py 1 file changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/61/3261/1 -- To view, visit http://gerrit.ovirt.org/3261 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8fdeda5f2900d818912ef5a939b60687462d5344 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

5 11

Change in vdsm[master]: Add callback to the plethora of retry halting possibilities
by smizrahi＠redhat.com 02 Jan '13

02 Jan '13

Saggi Mizrahi has uploaded a new change for review. Change subject: Add callback to the plethora of retry halting possibilities ...................................................................... Add callback to the plethora of retry halting possibilities Change-Id: Idb5a2158f008b41133352dcfb4926ad21dcceea1 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M tests/miscTests.py M vdsm/storage/misc.py 2 files changed, 35 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/93/8093/1 diff --git a/tests/miscTests.py b/tests/miscTests.py index bec6539..c8bcf09 100644 --- a/tests/miscTests.py +++ b/tests/miscTests.py @@ -621,6 +621,35 @@ self.assertEquals(misc._alignData(1, 1), (1, 1, 1)) +class RetryTests(TestCaseBase): + def testStopCallback(self): + counter = [0] + limit = 4 + def stopCallback(): + counter[0] += 1 + if counter[0] == limit: + return True + + return False + + def foo(): + print counter[0] + if counter[0] == (limit - 1): + return + + raise RuntimeError("If at first you don't succeed, try, try again." + "Then quit. There's no point in being a damn" + "fool about it.") + # W. C. Fields + + self.assertRaises(RuntimeError, misc.retry, foo, tries=(limit - 1), sleep=0, + stopCallback=stopCallback) + + counter[0] = 0 + misc.retry(foo, RuntimeError, tries=limit, sleep=0, + stopCallback=stopCallback) + + class ValidateDDBytes(TestCaseBase): def testValidInputTrue(self): """ diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 161726b..ef0c74e 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -728,7 +728,7 @@ def retry(func, expectedException=Exception, tries=None, - timeout=None, sleep=1): + timeout=None, sleep=1, stopCallback=None): """ Retry a function. Wraps the retry logic so you don't have to implement it each time you need it. @@ -741,6 +741,8 @@ the method. It will just not run it if it ended after the timeout. :param sleep: Time to sleep between calls in seconds. + :param stopCallback: A function that takes no parameters and invokes a + bail-out when it returns with a positive value. """ if tries in [0, None]: tries = -1 @@ -761,6 +763,9 @@ if (timeout > 0) and ((time.time() - startTime) > timeout): raise + if stopCallback is not None and stopCallback(): + raise + time.sleep(sleep) -- To view, visit http://gerrit.ovirt.org/8093 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idb5a2158f008b41133352dcfb4926ad21dcceea1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

6 16

Change in vdsm[master]: pylint:vdsm/*.py cast output to str() for .strip()/splitline...
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: pylint:vdsm/*.py cast output to str() for .strip()/splitlines() ...................................................................... pylint:vdsm/*.py cast output to str() for .strip()/splitlines() ************* Module caps E:187,35:osversion: Instance of 'list' has no 'splitlines' member (but some types could not be inferred ************* Module configNetwork E: 68,7:ifdown: Instance of 'list' has no 'strip' member (but some types could not be inferred) E: 70,7:ifdown: Instance of 'list' has no 'strip' member (but some types could not be inferred) E: 71,49:ifdown: Instance of 'list' has no 'splitlines' member (but some types could not be inferred) E: 80,7:ifup: Instance of 'list' has no 'strip' member (but some types could not be inferred) E: 82,7:ifup: Instance of 'list' has no 'strip' member (but some types could not be inferred) ************* Module netinfo E: 45,23:nics: Instance of '_Chainmap' has no 'split' member (but some types could not be inferred) E:137,22:ifconfig: Instance of 'list' has no 'split' member (but some types could not be inferred) ************* Module utils E:681,42:getHostUUID: Instance of 'list' has no 'splitlines' member (but some types could not be inferred) wrapper command and config output with str() to help pylint. Change-Id: Iae2c0ece02d5e41e0080bf41c0d9ea3544965274 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/caps.py M vdsm/configNetwork.py M vdsm/netinfo.py M vdsm/utils.py 4 files changed, 9 insertions(+), 9 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/3444/1 -- To view, visit http://gerrit.ovirt.org/3444 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iae2c0ece02d5e41e0080bf41c0d9ea3544965274 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

5 8

Change in vdsm[master]: Update comments to remove rhev-specific paths
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: Update comments to remove rhev-specific paths ...................................................................... Update comments to remove rhev-specific paths Change-Id: Ide161d8f23e9f49e70e1ba8322fbcec31eba5886 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/storage/fileSD.py M vdsm/storage/fileVolume.py M vdsm/storage/sp.py 3 files changed, 3 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/94/3294/1 -- To view, visit http://gerrit.ovirt.org/3294 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ide161d8f23e9f49e70e1ba8322fbcec31eba5886 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

2 2

Change in vdsm[master]: Remove rhev strings from vdsm_hooks directory and scripts
by Ryan Harper 02 Jan '13

02 Jan '13

Ryan Harper has uploaded a new change for review. Change subject: Remove rhev strings from vdsm_hooks directory and scripts ...................................................................... Remove rhev strings from vdsm_hooks directory and scripts Change-Id: I3a6012529af0e81e9c4163db33fa02a046c42920 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm_hooks/README M vdsm_hooks/persist-vdsm-hooks.in M vdsm_hooks/sriov/README M vdsm_hooks/unpersist-vdsm-hook 4 files changed, 13 insertions(+), 13 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/93/3293/1 -- To view, visit http://gerrit.ovirt.org/3293 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3a6012529af0e81e9c4163db33fa02a046c42920 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

3 3

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches November 2012