vdsm-patches August 2013

vdsm-patches@lists.fedorahosted.org

39 participants
459 discussions

Change in vdsm[master]: bz#831938 raise exception when error occurs in connectStorag...
by lvroyce＠linux.vnet.ibm.com 03 Dec '13

03 Dec '13

Royce Lv has uploaded a new change for review. Change subject: bz#831938 raise exception when error occurs in connectStorageServer ...................................................................... bz#831938 raise exception when error occurs in connectStorageServer connectStorageServer returned OK when error occurs: iscsi login fails, local directory lack of permission,etc change it to raise error to make early alarm Change-Id: I4c363f41d08669eec022797eb1eb440fe193f9c2 Signed-off-by: Royce Lv <lvroyce(a)linux.vnet.ibm.com> --- M vdsm/storage/hsm.py 1 file changed, 2 insertions(+), 8 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/37/6137/1 -- To view, visit http://gerrit.ovirt.org/6137 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I4c363f41d08669eec022797eb1eb440fe193f9c2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

3 3

Change in vdsm[master]: Unified network persistence [4/4] - Upgrade updates running ...
by amuller＠redhat.com 03 Dec '13

03 Dec '13

Assaf Muller has uploaded a new change for review. Change subject: Unified network persistence [4/4] - Upgrade updates running conf ...................................................................... Unified network persistence [4/4] - Upgrade updates running conf Run a VDSM upgrade that gathers net info and updates the running config of the unified network persistence. Change-Id: I051b0f168b6357e60184409845ea410c5891b083 Signed-off-by: Assaf Muller <amuller(a)redhat.com> --- M vdsm.spec.in M vdsm/Makefile.am M vdsm/upgrade/Makefile.am A vdsm/upgrade/__init__.py A vdsm/upgrade/unifiedPersistence.py A vdsm/vdsm-unified-network-persistence M vdsm/vdsmd.init.in 7 files changed, 140 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/25/18425/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 73fde5b..4e03b4f 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -832,6 +832,7 @@ %{_datadir}/%{vdsm_name}/vdsm %{_datadir}/%{vdsm_name}/vdsm-restore-net-config %{_datadir}/%{vdsm_name}/vdsm-store-net-config +%{_datadir}/%{vdsm_name}/vdsm-unified-network-persistence %{_datadir}/%{vdsm_name}/vm.py* %{_datadir}/%{vdsm_name}/zombieReaper.py* %config(noreplace) %{_sysconfdir}/%{vdsm_name}/logger.conf @@ -973,6 +974,7 @@ %{_datadir}/%{vdsm_name}/set-conf-item %dir %{_datadir}/%{vdsm_name}/upgrade %{_datadir}/%{vdsm_name}/upgrade/upgrade.py* +%{_datadir}/%{vdsm_name}/upgrade/unifiedPersistence.py* %if 0%{?with_gluster} %dir %{_datadir}/%{vdsm_name}/gluster %{_datadir}/%{vdsm_name}/gluster/__init__.py* diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 4c5f02a..2712c4e 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -93,6 +93,7 @@ set-conf-item \ vdsm \ vdsm-restore-net-config \ + vdsm-unified-network-persistence \ $(NULL) nodist_man8_MANS = \ diff --git a/vdsm/upgrade/Makefile.am b/vdsm/upgrade/Makefile.am index 014864a..558a19a 100644 --- a/vdsm/upgrade/Makefile.am +++ b/vdsm/upgrade/Makefile.am @@ -23,4 +23,6 @@ upgradedir = $(vdsmdir)/upgrade dist_upgrade_PYTHON = \ - upgrade.py + upgrade.py \ + unifiedPersistence.py + diff --git a/vdsm/upgrade/__init__.py b/vdsm/upgrade/__init__.py new file mode 100644 index 0000000..8e88115 --- /dev/null +++ b/vdsm/upgrade/__init__.py @@ -0,0 +1,18 @@ +# Copyright 2013 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# diff --git a/vdsm/upgrade/unifiedPersistence.py b/vdsm/upgrade/unifiedPersistence.py new file mode 100644 index 0000000..7db6646 --- /dev/null +++ b/vdsm/upgrade/unifiedPersistence.py @@ -0,0 +1,84 @@ +# Copyright 2013 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +from upgrade import Upgrade +from vdsm import netinfo + + +class UnifiedPersistence(Upgrade): + def run(self): + networks, bondings = self._getNetInfo() + self._persist(networks, bondings) + + def _getNetInfo(self): + _netinfo = netinfo.NetInfo() + networks = {} + bondings = {} + whitelist = ['bridged', 'vlan', 'mtu', 'qosInbound', 'qosOutbound', + 'stp'] + + """ + bridged="True | False" + vlan=<id> + bonding="<name>" | nic="<name>" + ipaddr="<ip>" + netmask="<ip>" + gateway="<ip>" + bootproto="dhcp | none" + delay="..." + onboot="yes | no" + """ + + for network, netParams in _netinfo.networks.iteritems(): + networks[network] = {} + for k, v in netParams.iteritems(): + if k in whitelist: + networks[network][k] = v + if netParams.get('ports'): + device = "".join(netParams['ports']) + else: + device = netParams['interface'] + topLevelDevice = \ + netParams['iface'] if netParams['bridged'] else device + bootproto = netinfo.getBootProtocol(topLevelDevice) + networks[network]['bootproto'] = bootproto + if bootproto != 'dhcp': + networks[network]['ipaddr'] = netParams['addr'] + networks[network]['netmask'] = netParams['netmask'] + networks[network]['gateway'] = netParams['gateway'] + if device in _netinfo.bondings: + bondings[device] = \ + {'nics': _netinfo.bondings[device]['slaves']} + networks[network]['bonding'] = device + else: + networks[network]['nic'] = device + + return networks, bondings + + def _persist(self, networks, bondings): + self.log.debug("networks: %s, bondings: %s" % (networks, bondings)) + print("networks: %s, bondings: %s" % (networks, bondings)) + + +def main(): + UnifiedPersistence().runIfNeeded() + + +if __name__ == "__main__": + main() diff --git a/vdsm/vdsm-unified-network-persistence b/vdsm/vdsm-unified-network-persistence new file mode 100755 index 0000000..64a4db9 --- /dev/null +++ b/vdsm/vdsm-unified-network-persistence @@ -0,0 +1,30 @@ +#! /usr/bin/python +# +# Copyright 2013 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +from upgrade.unifiedPersistence import UnifiedPersistence + + +def main(): + UnifiedPersistence().runIfNeeded() + + +if __name__ == '__main__': + main() diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 1bdb911..215b0ff 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -261,6 +261,8 @@ return 1 fi + "@VDSMDIR@/vdsm-unified-network-persistence" + echo $"Starting up vdsm daemon: " local vdsm_nice="$("${GETCONFITEM}" "${CONF_FILE}" vars vdsm_nice -5)" -- To view, visit http://gerrit.ovirt.org/18425 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I051b0f168b6357e60184409845ea410c5891b083 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Assaf Muller <amuller(a)redhat.com>

6 44

Change in vdsm[master]: New upgrade system
by amuller＠redhat.com 03 Dec '13

03 Dec '13

Assaf Muller has uploaded a new change for review. Change subject: New upgrade system ...................................................................... New upgrade system New simplistic upgrade system to support running code exactly once. Change-Id: Iba3c9c34f03134c192db1c2add31084824e195d9 Signed-off-by: Assaf Muller <amuller(a)redhat.com> --- M configure.ac M vdsm.spec.in M vdsm/Makefile.am A vdsm/upgrade/Makefile.am A vdsm/upgrade/upgrade.py 5 files changed, 81 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/26/17726/1 diff --git a/configure.ac b/configure.ac index 0ccca95..0504a8a 100644 --- a/configure.ac +++ b/configure.ac @@ -217,6 +217,7 @@ vdsm/storage/Makefile vdsm/storage/imageRepository/Makefile vdsm/storage/protect/Makefile + vdsm/upgrade/Makefile vdsm_api/Makefile vdsm_hooks/Makefile vdsm_hooks/checkimages/Makefile diff --git a/vdsm.spec.in b/vdsm.spec.in index 87bc231..4389575 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -964,6 +964,8 @@ %{_datadir}/%{vdsm_name}/respawn %{_datadir}/%{vdsm_name}/sampling.py* %{_datadir}/%{vdsm_name}/set-conf-item +%dir %{_datadir}/%{vdsm_name}/upgrade +%{_datadir}/%{vdsm_name}/upgrade/upgrade.py* %if 0%{?with_gluster} %dir %{_datadir}/%{vdsm_name}/gluster %{_datadir}/%{vdsm_name}/gluster/__init__.py* @@ -997,6 +999,7 @@ %config(noreplace) %{_sysconfdir}/pki/%{vdsm_name}/keys/libvirt_password %dir %{_localstatedir}/lib/%{vdsm_name} %dir %{_localstatedir}/lib/%{vdsm_name}/netconfback +%dir %{_localstatedir}/lib/%{vdsm_name}/upgrade %dir %{_localstatedir}/run/%{vdsm_name} %dir %{_localstatedir}/run/%{vdsm_name}/sourceRoutes %dir %{_localstatedir}/run/%{vdsm_name}/trackedInterfaces diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index c30dcd9..33f91d6 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -18,7 +18,7 @@ # Refer to the README and COPYING files for full details of the license # -SUBDIRS = netconf sos storage gluster +SUBDIRS = netconf sos storage gluster upgrade include $(top_srcdir)/build-aux/Makefile.subs @@ -172,6 +172,7 @@ $(MKDIR_P) $(DESTDIR)$(vdsmrundir)/trackedInterfaces $(MKDIR_P) $(DESTDIR)$(vdsmrundir)/payload $(MKDIR_P) $(DESTDIR)$(vdsmlibdir)/netconfback + $(MKDIR_P) $(DESTDIR)$(vdsmlibdir)/upgrade $(MKDIR_P) $(DESTDIR)$(vdsmpoolsdir) $(MKDIR_P) $(DESTDIR)$(vdsmbackupdir) $(MKDIR_P) $(DESTDIR)$(localstatedir)/lib/libvirt/qemu/channels diff --git a/vdsm/upgrade/Makefile.am b/vdsm/upgrade/Makefile.am new file mode 100644 index 0000000..014864a --- /dev/null +++ b/vdsm/upgrade/Makefile.am @@ -0,0 +1,26 @@ +# +# Copyright 2013 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +include $(top_srcdir)/build-aux/Makefile.subs + +upgradedir = $(vdsmdir)/upgrade + +dist_upgrade_PYTHON = \ + upgrade.py diff --git a/vdsm/upgrade/upgrade.py b/vdsm/upgrade/upgrade.py new file mode 100644 index 0000000..60913bc --- /dev/null +++ b/vdsm/upgrade/upgrade.py @@ -0,0 +1,49 @@ +# Copyright 2013 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +import logging +import os + +import dsaversion +from vdsm import constants + + +class Upgrade(object): + def __init__(self): + self.upgradeName = self.__class__.__name__ + self.log = logging.getLogger('vds') + + def runIfNeeded(self): + if self.isNeeded(): + self.log.debug("Running upgrade %s" % self.upgradeName) + self.run() + self.done() + + def isNeeded(self): + return not os.path.exists(self._getUpgradeFilePath()) + + def run(self): + raise NotImplementedError + + def done(self): + with open(self._getUpgradeFilePath(), 'w') as upgradeFile: + upgradeFile.write(dsaversion.raw_version_revision) + + def _getUpgradeFilePath(self): + return constants.P_VDSM_LIB + "upgrade/%s" % self.upgradeName -- To view, visit http://gerrit.ovirt.org/17726 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iba3c9c34f03134c192db1c2add31084824e195d9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Assaf Muller <amuller(a)redhat.com>

9 60

Change in vdsm[master]: Add uploadIso API call for pushing ISOs into an active iso_d...
by Ryan Harper 03 Dec '13

03 Dec '13

Ryan Harper has uploaded a new change for review. Change subject: Add uploadIso API call for pushing ISOs into an active iso_domain via pool ...................................................................... Add uploadIso API call for pushing ISOs into an active iso_domain via pool The only way to get an ISO into the vdsm storage service is via ovirt-engine's engine-iso-uploader. This requires a functioning ovirt-engine install. Instead provide a simple API call to push the iso file out to the correct location. Given an pool UUID, we can determine where the iso file should go via isoprefix so use either wget or rsync to pull the iso file and place it in the pool's iso_domain. wget a remote iso % vdsClient -s 0 uploadIso d3b3948a-ead6-11e1-a3a5-00fffe0000df \ http://server/path/Fedora-17-x86_64-netinst.iso rsync in a local file % vdsClient -s 0 uploadIso d3b3948a-ead6-11e1-a3a5-00fffe0000df \ /tmp/Fedora-17-x86_64-netinst.iso Change-Id: Id78e46513c38789d08e63a38026b28bebb9a2b12 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm/API.py M vdsm/BindingXMLRPC.py M vdsm/storage/hsm.py M vdsm/storage/sp.py M vdsm/storage/storage_exception.py M vdsm_api/vdsmapi-schema.json M vdsm_cli/vdsClient.py 7 files changed, 136 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/49/7849/1 diff --git a/vdsm/API.py b/vdsm/API.py index 827f73b..734f40d 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -956,6 +956,9 @@ return self._irs.upgradeStoragePool(self._UUID, targetDomVersion) + def uploadIso(self, srcPath, method): + return self._irs.uploadIso(self._UUID, srcPath, method) + def validateStorageServerConnection(self, domainType, connectionParams): return self._irs.validateStorageServerConnection(domainType, diff --git a/vdsm/BindingXMLRPC.py b/vdsm/BindingXMLRPC.py index 826c125..ca92779 100644 --- a/vdsm/BindingXMLRPC.py +++ b/vdsm/BindingXMLRPC.py @@ -563,6 +563,10 @@ pool = API.StoragePool(spUUID) return pool.upgrade(targetDomVersion) + def poolUploadIso(self, spUUID, srcPath, method): + pool = API.StoragePool(spUUID) + return pool.uploadIso(srcPath, method) + def poolValidateStorageServerConnection(self, domType, spUUID, conList, options=None): pool = API.StoragePool(spUUID) @@ -809,6 +813,7 @@ (self.poolSpmStart, 'spmStart'), (self.poolSpmStop, 'spmStop'), (self.poolUpgrade, 'upgradeStoragePool'), + (self.poolUploadIso, 'uploadIso'), (self.poolValidateStorageServerConnection, 'validateStorageServerConnection'), (self.poolUpdateVMs, 'updateVM'), diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 1e9311b..2fb9fae 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1189,6 +1189,19 @@ pool = self.getPool(spUUID) pool.uploadVolume(sdUUID, imgUUID, volUUID, srcPath, size, method="rsync") + @public + def uploadIso(self, spUUID, srcPath, method): + """ + Uploads an ISO to <pool UUID>'s iso domain + + :param spUUID: The UUID of the storage pool that will contain the new iso. + :type spUUID: UUID + :param method: The desired method of upload. Currently only *'wget'* and *'rsync'* are supported. + :type method: str + """ + vars.task.getSharedLock(STORAGE, spUUID) + pool = self.getPool(spUUID) + pool.uploadIso(srcPath, method) @public def createVolume(self, sdUUID, spUUID, imgUUID, size, volFormat, preallocate, diskType, volUUID, desc, srcImgUUID=volume.BLANK_UUID, srcVolUUID=volume.BLANK_UUID): diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index c10974c..1c9dec7 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -2004,6 +2004,69 @@ except: self.log.warning("SP %s SD %s img %s Vol %s - teardown failed") + + def uploadIso(self, srcPath, method): + self.log.info("uploadIso - srcPath=%s method=%s"%(srcPath, method)) + + # find the iso domain within the pool + info = self.getInfo()['info'] + if 'isoprefix' in info.keys() and len(info['isoprefix']) > 0: + targetPath = info['isoprefix'] + else: + self.log.error("uploadIso - target pool(%s) does not" + "have an ISO domain" %(self.spUUID)) + raise se.StoragePoolNoIsoDomain() + + self.log.info("uploadIso - targetPath=%s"%(targetPath)) + rc = 0 + + # we can infer the method to use by inspection of srcPath + if len(method) == 0: + if srcPath.startswith("http://"): + method = "wget" + else: + method = "rsync" + + try: + if method.lower() == "wget": + if not srcPath.startswith("http://"): + self.log.error("uploadIso - wget method requires HTTP URL") + raise se.InvalidParameterException('srcPath mismatch with' + ' method', method) + + outFile = os.path.join(targetPath, os.path.basename(srcPath)) + cmd = [constants.EXT_WGET, "-O", outFile, srcPath] + (rc, out, err) = misc.execCmd(cmd, sudo=False) + + if rc: + self.log.error("uploadIso - error while trying to retrieve:" + "%s into: %s, stderr: %s" % (srcPath, + targetPath, + err)) + raise se.VolumeCopyError(srcPath, err) + elif method.lower() == "rsync": + if srcPath.startswith("http://"): + self.log.error("uploadIso - rsync method cannot use" + "HTTP URL") + raise se.InvalidParameterException('srcPath mismatch with' + ' method', method) + cmd = [constants.EXT_RSYNC, "-aq", srcPath, targetPath] + (rc, out, err) = misc.execCmd(cmd, sudo=False) + + if rc: + self.log.error("uploadIso - error while trying to copy:" + "%s into: %s, stderr: %s" % (srcPath, + targetPath, + err)) + raise se.VolumeCopyError(srcPath, err) + else: + self.log.error("uploadIso - method '%s' not supported", method) + raise se.InvalidParameterException('method', method) + + finally: + if rc == 0: + self.log.info("uploadIso - completed via %s", method) + def preDeleteRename(self, sdUUID, imgUUID): repoPath = os.path.join(self.storage_repository, self.spUUID) return image.Image(repoPath).preDeleteRename(sdUUID, imgUUID) diff --git a/vdsm/storage/storage_exception.py b/vdsm/storage/storage_exception.py index f1edad4..88ac2d5 100644 --- a/vdsm/storage/storage_exception.py +++ b/vdsm/storage/storage_exception.py @@ -508,6 +508,10 @@ code = 330 message = "Cannot get files list" +class StoragePoolNoIsoDomain(StorageException): + code = 331 + message = "No ISO Domain available in Storage Pool" + ################################################# # Domains Exceptions ################################################# diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index a773177..d898d0a 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -3309,6 +3309,38 @@ 'size': 'int', 'method': 'UploadVolumeMethod'}} ## +# @UploadIsoMethod: +# +# An enumeration of uploadIso methods. +# +# @wget: Fetch data using the wget command +# +# @rsync: Fetch data using the rsync command +# +# Since: 4.10.0 +## +{'enum': 'UploadIsoMethod', 'data': ['wget', 'rsync']} + +## +# @StoragePool.uploadIso: +# +# Upload an ISO file into a Storage Pool from a remote location. +# +# @spUUID: The UUID of the pool in which the ISO will be uploaded +# +# @srcPath: The remote location of the ISO data. Must be in a format that +# can be understood by the command indicated in @method. +# +# @method: The method (command) to use to upload the data +# +# Since: 4.10.0 +## +{'command': {'class': 'StoragePool', 'name': 'uploadIso'}, + 'data': {'spUUID': 'UUID', 'srcPath': 'str', + 'method': 'UploadVolumeMethod'}} + + +## # @StorageDomain.validate: # # Check that the Storage Domain is accessible. diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index ff7ccc8..c1f5268 100644 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -962,6 +962,17 @@ return status['status']['code'], status['status']['message'] return 0, '' + def uploadIso(self, args): + spUUID = args[0] + srcPath = args[1] + method = '' + if len(args) > 2: + method = args[2] + status = self.s.uploadIso(spUUID, srcPath, method) + if status['status']['code']: + return status['status']['code'], status['status']['message'] + return 0, '' + def uploadVolume(self, args): sdUUID = args[0] spUUID = args[1] @@ -2048,6 +2059,11 @@ '<new disk size>', 'Extend volume (SAN only)' )), + 'uploadIso': (serv.uploadIso, + ('<spUUID> <srcPath> [<method>]', + 'Upload ISO file into the specified storage pool.' + ' <method> is one of [wget|rsync] and is optional' + )), 'uploadVolume': (serv.uploadVolume, ('<sdUUID> <spUUID> <imgUUID> <volUUID> <srcPath> ' '<size>', -- To view, visit http://gerrit.ovirt.org/7849 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id78e46513c38789d08e63a38026b28bebb9a2b12 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

2 4

Change in vdsm[master]: [WIP]add mem cgroup support to vdsm
by lvroyce＠linux.vnet.ibm.com 03 Dec '13

03 Dec '13

Royce Lv has uploaded a new change for review. Change subject: [WIP]add mem cgroup support to vdsm ...................................................................... [WIP]add mem cgroup support to vdsm add vdsm mem cgroup support, New version libvirt memtune function has a bug, tested uncompleted until I fix the bug. Change-Id: If2ad220e21b114985ead4bd64e81133173566444 Signed-off-by: Royce Lv<lvroyce(a)linux.vnet.ibm.com> --- M vdsm/API.py M vdsm/BindingXMLRPC.py M vdsm/define.py M vdsm/libvirtvm.py M vdsm_api/vdsmapi-schema.json M vdsm_cli/vdsClient.py 6 files changed, 90 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/31/8031/1 diff --git a/vdsm/API.py b/vdsm/API.py index 566d6e5..f5889f9 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -37,6 +37,7 @@ import storage.volume import storage.sd import storage.image +import storage.storage_exception as se from vdsm.define import doneCode, errCode, Kbytes, Mbytes import caps from vdsm.config import config @@ -555,6 +556,19 @@ message = USER_SHUTDOWN_MESSAGE return v.shutdown(delay, message) + def tuneMem(self, tuneParams): + try: + v = self._cif.vmContainer[self._UUID] + for key in tuneParams: + tuneParams[key] = storage.misc.validateN( + tuneParams[key], key) + except KeyError: + return errCode['noVM'] + except se.InvalidParameterException: + return errCode['tuneMemErr'] + + return v.tuneMem(tuneParams) + def _createSysprepFloppyFromInf(self, infFileBinary, floppyImage): try: rc, out, err = utils.execCmd([constants.EXT_MK_SYSPREP_FLOPPY, diff --git a/vdsm/BindingXMLRPC.py b/vdsm/BindingXMLRPC.py index 826c125..fe7b9d5 100644 --- a/vdsm/BindingXMLRPC.py +++ b/vdsm/BindingXMLRPC.py @@ -265,6 +265,10 @@ vm = API.VM(vmId) return vm.snapshot(snapDrives) + def vmTuneMem(self, params): + vm = API.VM(params.pop('vmId')) + return vm.tuneMem(params) + def vmMerge(self, vmId, mergeDrives): vm = API.VM(vmId) return vm.merge(mergeDrives) @@ -765,7 +769,8 @@ (self.vmHotplugDisk, 'hotplugDisk'), (self.vmHotunplugDisk, 'hotunplugDisk'), (self.vmHotplugNic, 'hotplugNic'), - (self.vmHotunplugNic, 'hotunplugNic')) + (self.vmHotunplugNic, 'hotunplugNic'), + (self.vmTuneMem, 'tuneMem')) def getIrsMethods(self): return ((self.domainActivate, 'activateStorageDomain'), diff --git a/vdsm/define.py b/vdsm/define.py index 049d36c..ba71bf6 100644 --- a/vdsm/define.py +++ b/vdsm/define.py @@ -117,6 +117,9 @@ 'momErr': {'status': {'code': 54, 'message': 'Failed to set mom policy'}}, + 'tuneMemErr': {'status': + {'code': 56, + 'message': 'Failed to tune Mem'}}, 'recovery': {'status': {'code': 99, 'message': diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 4dc47e3..048fa7d 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -1887,6 +1887,22 @@ (snapFlags & libvirt.VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE == libvirt.VIR_DOMAIN_SNAPSHOT_CREATE_QUIESCE)} + def tuneMem(self, tuneParams): + try: + self._dom.setMemoryParameters(tuneParams) + for param in tuneParams: + self.conf[param] = tuneParams[param] + self.saveState() + return {'status': doneCode} + except libvirt.libvirtError, e: + self.log.error("TuneMem failed:%s", + e.message, exc_info=True) + return errCode['tuneMemErr'] + except LookupError: + self.log.error('TuneMem failed: unrecongnized params', + exc_info=True) + return errCode['tuneMemErr'] + def _runMerge(self): for mergeStatus in self.conf.get('liveMerge', []): if mergeStatus['status'] != MERGESTATUS.NOT_STARTED: diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index c0440d2..509a90c 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -5099,6 +5099,34 @@ {'command': {'class': 'VM', 'name': 'mergeStatus'}, 'returns': ['MergeStatus']} +## +# @TuneMemParams: +# +# Parameters for @VM.tuneMem. +# +# @hard_limit: Maximum memory the VM can consume +# +# @soft_limit: Memory limit during contention +# +# @min_gurantee: Min memory guranteed to the VM +# +# Since: 4.10.0 +## +{'type': 'TuneMemParams', + 'data': {'hard_limit': 'uint', 'soft_limit': 'uint', + 'min_gurantee': 'uint'}} +## +# @VM.tuneMem: +# +# Tune memory hard limits and min allocation guranteed. +# +# @TuneMemParams: Tunning parameters for memory +# +# Since: 4.10.0 +## +{'command': {'class': 'VM', 'name': 'tuneMem'}, + 'data': {'tuneParams': 'TuneMemParams'}} + ## Category: @Volume ########################################################### ## # @Volume: diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index fe2f083..8787c28 100644 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -204,6 +204,16 @@ return self.ExecAndExit(self.s.create(params)) + def tuneMem(self, args): + params = {} + params['vmId'] = args[0] + if len(args) > 1: + for param in args[1:]: + if '=' in param: + key, value = line.split("=", 1) + params[key] = value + return self.ExecAndExit(self.s.tuneMem(params)) + def hotplugNic(self, args): nic = self._parseDriveSpec(args[1]) nic['type'] = 'interface' @@ -1801,6 +1811,19 @@ 'Optional additional parameters in dictionary format,' ' name:value,name:value' )), + 'tuneMem': (serv.tuneMem, + ('<vmId>' + '[hard_limit=value soft_limit=value min_gurantee]', + 'Parameters list: r=required, o=optional', + 'r vmId: The vm to be tuned', + 'o hard_limit=<value>: ' + 'maximum memory the VM can use(in KiB)', + 'o soft_limit=<value>:' + 'memory limit to enforce during memory contention', + 'o min_gurantee=<value>:' + 'min memory guranteed to the VM', + 'Tune memory hard limits and min allocation gurantee' + )), 'migrate': (serv.do_migrate, ('vmId=<id> method=<offline|online> src=<host:[port]> ' 'dst=<host:[port]>', -- To view, visit http://gerrit.ovirt.org/8031 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If2ad220e21b114985ead4bd64e81133173566444 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

4 5

Change in vdsm[master]: No need to call receive if send is timeout
by shuming＠linux.vnet.ibm.com 03 Dec '13

03 Dec '13

Shu Ming has uploaded a new change for review. Change subject: No need to call receive if send is timeout ...................................................................... No need to call receive if send is timeout In callCrabRPCFunction(), the request packet is sent to the crab server first and then wait for the response packet from the crab server. When the request packet is not sent correctly, there is no need to wait for the response packet back. So raising exception after the send operation is timeout. Change-Id: Ib233b24d108a986d2f9e07a66f2a8f7d8cfb221a Signed-off-by: Shu Ming <shuming(a)linux.vnet.ibm.com> --- M vdsm/storage/remoteFileHandler.py 1 file changed, 8 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/59/7759/1 diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py index 622ea5d..6995c39 100644 --- a/vdsm/storage/remoteFileHandler.py +++ b/vdsm/storage/remoteFileHandler.py @@ -173,8 +173,13 @@ def callCrabRPCFunction(self, timeout, name, *args, **kwargs): request = pickle.dumps((name, args, kwargs)) - self._sendAll(pack(LENGTH_STRUCT_FMT, len(request)), timeout) - self._sendAll(request, timeout) + try: + self._sendAll(pack(LENGTH_STRUCT_FMT, len(request)), timeout) + self._sendAll(request, timeout) + except: + self.log.warn("Problem with handler send, treating as timeout", + exc_info=True) + raise Timeout() try: rawLength = self._recvAll(LENGTH_STRUCT_LENGTH, timeout) @@ -184,7 +189,7 @@ except: # If for some reason the connection drops\gets out of sync we treat # it as a timeout so we only have one error path - self.log.warn("Problem with handler, treating as timeout", + self.log.warn("Problem with handler receive, treating as timeout", exc_info=True) raise Timeout() -- To view, visit http://gerrit.ovirt.org/7759 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib233b24d108a986d2f9e07a66f2a8f7d8cfb221a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shu Ming <shuming(a)linux.vnet.ibm.com>

3 3

Change in vdsm[master]: [WIP]extend setPolicy to support mom multi-policy
by lvroyce＠linux.vnet.ibm.com 03 Dec '13

03 Dec '13

Royce Lv has uploaded a new change for review. Change subject: [WIP]extend setPolicy to support mom multi-policy ...................................................................... [WIP]extend setPolicy to support mom multi-policy set all policies at a single str brings difficulty to enable and disable a single policy. extend the setPolicy from str to policyDict. Depend on mom implementation to be ready. Change-Id: I94e601a8b1cb30d43005918dbee7d83c64f7f19b Signed-off-by: Royce Lv<lvroyce(a)linux.vnet.ibm.com> --- M tests/functional/momTests.py M vdsm/API.py M vdsm/BindingXMLRPC.py M vdsm/momIF.py M vdsm_api/vdsmapi-schema.json M vdsm_cli/vdsClient.py 6 files changed, 22 insertions(+), 12 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/52/9152/1 diff --git a/tests/functional/momTests.py b/tests/functional/momTests.py index 1981e05..30da052 100644 --- a/tests/functional/momTests.py +++ b/tests/functional/momTests.py @@ -44,8 +44,9 @@ (Host.Control "ksm_run" %d) (Host.Control "ksm_pages_to_scan" %d)""" % \ (run, pages_to_scan) + testPolicyDict = {"ksm": testPolicyStr} s = vdscli.connect() - r = s.setMOMPolicy(testPolicyStr) + r = s.setMOMPolicy(testPolicyDict) self.assertEqual(r['status']['code'], 0, str(r)) # Wait for the policy taking effect @@ -54,3 +55,10 @@ hostStats = s.getVdsStats()['info'] self.assertEqual(bool(run), hostStats['ksmState']) self.assertEqual(pages_to_scan, hostStats['ksmPages']) + + def testMultiPolicy(self): + testPolicyDict = {"10_test": "(+1 1)", "20_test": "(-1 1)"} + + s = vdscli.connect() + r = s.setMOMPolicy(testPolicyDict) + self.assertEqual(r['status']['code'], 0, str(r)) diff --git a/vdsm/API.py b/vdsm/API.py index 896767c..7c1dad9 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -1329,9 +1329,11 @@ def getAllTasks(self): return self._irs.getAllTasks() - def setMOMPolicy(self, policy): + def setMOMPolicy(self, policyDict): try: - self._cif.mom.setPolicy(policy) + for policyName, policyStr in policyDict.iteritems(): + self._cif.mom.setPolicyItem(policyName, policyStr) + return dict(status=doneCode) except: return errCode['momErr'] diff --git a/vdsm/BindingXMLRPC.py b/vdsm/BindingXMLRPC.py index b833cf3..de35a3f 100644 --- a/vdsm/BindingXMLRPC.py +++ b/vdsm/BindingXMLRPC.py @@ -376,9 +376,9 @@ api = API.Global() return api.setLogLevel(level) - def setMOMPolicy(self, policy): + def setMOMPolicy(self, policyDict): api = API.Global() - return api.setMOMPolicy(policy) + return api.setMOMPolicy(policyDict) def domainActivate(self, sdUUID, spUUID, options=None): domain = API.StorageDomain(sdUUID, spUUID) diff --git a/vdsm/momIF.py b/vdsm/momIF.py index 827e9e4..c95d5fc 100644 --- a/vdsm/momIF.py +++ b/vdsm/momIF.py @@ -47,9 +47,9 @@ ret['ksmCpu'] = stats['ksmd_cpu_usage'] return ret - def setPolicy(self, policyStr): + def setPolicyItem(self, policyName, policyStr): # mom.setPolicy will raise an exception on failure. - self._mom.setPolicy(policyStr) + self._mom.setNamedPolicy(policyName, policyStr) def stop(self): if self._mom is not None: diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 5f0b391..03bc7c1 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -444,12 +444,12 @@ # # Set MOM policy for different level of overcommitments. # -# @policy: MOM policy string. +# @policyDict: MOM policy dictionary. # # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'setMOMPolicy'}, - 'data': {'policy': 'str'}} + 'data': {'policyDict': 'StringMap'}} ## # @TaskDetails: diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 8cb78ed..b65ecd7 100644 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -1387,8 +1387,8 @@ return stats['status']['code'], stats['status']['message'] return 0, '' - def do_setMOMPolicy(self, policyFile): - stats = self.s.setMOMPolicy(policyFile) + def do_setMOMPolicy(self, policyDict): + stats = self.s.setMOMPolicy(policyDict) if stats['status']['code']: return stats['status']['code'], stats['status']['message'] return 0, '' @@ -2207,7 +2207,7 @@ ' level (10=DEBUG, 50=CRITICAL' )), 'setMOMPolicy': (serv.do_setMOMPolicy, - ('<policyfile>', 'set MOM policy')), + ('<policyDict>', 'set MOM policy')), 'deleteImage': (serv.deleteImage, ('<sdUUID> <spUUID> <imgUUID> [<postZero>] [<force>]', 'Delete Image folder with all volumes.', -- To view, visit http://gerrit.ovirt.org/9152 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I94e601a8b1cb30d43005918dbee7d83c64f7f19b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

2 2

Change in vdsm[master]: make misc.pgrep for general usage
by lvroyce＠linux.vnet.ibm.com 02 Dec '13

02 Dec '13

Royce Lv has uploaded a new change for review. Change subject: make misc.pgrep for general usage ...................................................................... make misc.pgrep for general usage pgrep now just grep specific named process, Make it accept other params(e.g.ppid, state, etc) for general usage. Change-Id: I2034b04a4787d2f222cb98dfe3ffcdbb2a94ebb8 Signed-off-by: Royce Lv<lvroyce(a)linux.vnet.ibm.com> --- M tests/miscTests.py M vdsm/storage/misc.py 2 files changed, 21 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/33/11033/1 diff --git a/tests/miscTests.py b/tests/miscTests.py index a949a09..885b913 100644 --- a/tests/miscTests.py +++ b/tests/miscTests.py @@ -63,7 +63,7 @@ class PgrepTests(TestCaseBase): - def test(self): + def testGrepName(self): sleepProcs = [] for i in range(3): sleepProcs.append(misc.execCmd([EXT_SLEEP, "3"], sync=False, @@ -78,6 +78,21 @@ proc.kill() proc.wait() + def testGrepPpid(self): + sleepProcs = [] + for i in range(3): + sleepProcs.append(misc.execCmd([EXT_SLEEP, "3"], sync=False, + sudo=False)) + + pids = misc.pgrep(os.getpid(), 'ppid') + for proc in sleepProcs: + self.assertTrue(proc.pid in pids, "pid %d was not located by pgrep" + % proc.pid) + + for proc in sleepProcs: + proc.kill() + proc.wait() + class GetCmdArgsTests(TestCaseBase): def test(self): diff --git a/vdsm/storage/misc.py b/vdsm/storage/misc.py index 5ec13e4..2438335 100644 --- a/vdsm/storage/misc.py +++ b/vdsm/storage/misc.py @@ -1109,8 +1109,10 @@ yield int(pid) -def pgrep(name): +def pgrep(statValue, statKey='name'): res = [] + statMap = {'name': 1, 'state': 2, 'ppid': 3} + statIndex = statMap.get(statKey, 1) for pid in iteratePids(): try: pid = int(pid) @@ -1118,8 +1120,8 @@ continue try: - procName = utils.pidStat(pid)[1] - if procName == name: + watchedStat = utils.pidStat(pid)[statIndex] + if watchedStat == statValue: res.append(pid) except (OSError, IOError): continue -- To view, visit http://gerrit.ovirt.org/11033 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2034b04a4787d2f222cb98dfe3ffcdbb2a94ebb8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

8 18

Change in vdsm[master]: move get-conf-item to vdsm-tool
by shaohef＠linux.vnet.ibm.com 02 Dec '13

02 Dec '13

ShaoHe Feng has uploaded a new change for review. Change subject: move get-conf-item to vdsm-tool ...................................................................... move get-conf-item to vdsm-tool many items in init scripts and spec file. move the complexity from out of the init scripts and spec file and into vdsm-tool Change-Id: I5c64de097bbaea6a8cf862b43243377e10e00391 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M vdsm-tool/Makefile.am A vdsm-tool/vdsm_conf_item.py.in M vdsm.spec.in M vdsm/vdsmd.init.in 4 files changed, 86 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/95/7695/1 diff --git a/vdsm-tool/Makefile.am b/vdsm-tool/Makefile.am index 6ec0198..dcbf7e1 100644 --- a/vdsm-tool/Makefile.am +++ b/vdsm-tool/Makefile.am @@ -23,9 +23,11 @@ vdsm-tool EXTRA_DIST = \ - validate_ovirt_certs.py.in + validate_ovirt_certs.py.in \ + vdsm_conf_item.py.in dist_vdsmtool_DATA = \ __init__.py \ passwd.py \ - validate_ovirt_certs.py \ No newline at end of file + validate_ovirt_certs.py \ + vdsm_conf_item.py diff --git a/vdsm-tool/vdsm_conf_item.py.in b/vdsm-tool/vdsm_conf_item.py.in new file mode 100644 index 0000000..ed268ff --- /dev/null +++ b/vdsm-tool/vdsm_conf_item.py.in @@ -0,0 +1,72 @@ +# Copyright 2012 IBM, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +# Access python's config files from bash. +# Usage: get-conf-itemname + +from vdsm.tool import expose +try: + from vdsm.config import config + vdsmconf = config +except: + import ConfigParser + vdsmconf = ConfigParser.ConfigParser() +from functools import partial + + +CONF_FILE = "@CONFDIR@/vdsm.conf" + + +def _get_conf_item(file, section, item, default): + vdsmconf.read(file) + try: + return vdsmconf.get(section, item) + except: + return default + + +def print_conf_item(file, section, item, default): + val = _get_conf_item(file, section, item, default) + if (val.lower() == "false" or val.lower() == "true"): + print val.lower() + else: + print val + + +# there are so many item in vdsm.conf so list all the item +# then iterate the list and expose every item in list +# item in tuple: (file, section, value, default) +get_item_list = ( + (CONF_FILE, 'addresses', 'management_port', ''), + (CONF_FILE, 'addresses', 'management_ip', '0.0.0.0'), + (CONF_FILE, 'irs', 'repository', '/rhev/'), + (CONF_FILE, 'irs', 'vdsm_nice', '-5'), + (CONF_FILE, 'vars', 'core_dump_enable', 'false'), + (CONF_FILE, 'vars', 'ssl', 'true'), + (CONF_FILE, 'vars', 'libvirt_log_filters', '1:libvirt 1:remote'), + (CONF_FILE, 'vars', 'libvirt_log_outputs', + '1:file:/var/log/vdsm/libvirt.log'), + (CONF_FILE, 'vars', 'trust_store_path', '/etc/pki/vdsm')) + +for item in get_item_list: + f, s, v, d = item + alias = "get-conf-" + v + globals()[alias] = partial(print_conf_item, f, s, v, d) + globals()[alias].__doc__ = " get %s from vdsm configure file" % v + expose(alias)(globals()[alias]) diff --git a/vdsm.spec.in b/vdsm.spec.in index 0d1b195..e161a93 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -720,6 +720,7 @@ %{python_sitearch}/%{vdsm_name}/tool/__init__.py* %{python_sitearch}/%{vdsm_name}/tool/passwd.py* %{python_sitearch}/%{vdsm_name}/tool/validate_ovirt_certs.py* +%{python_sitearch}/%{vdsm_name}/tool/vdsm_conf_item.py* %files tests %doc %{_datadir}/%{vdsm_name}/tests/README diff --git a/vdsm/vdsmd.init.in b/vdsm/vdsmd.init.in index 860e301..b33e1be 100755 --- a/vdsm/vdsmd.init.in +++ b/vdsm/vdsmd.init.in @@ -40,7 +40,7 @@ LDCONF=/etc/sysconfig/libvirtd QLCONF=/etc/libvirt/qemu-sanlock.conf -is_coredump=`$GETCONFITEM $CONF_FILE vars core_dump_enable false | tr A-Z a-z` +is_coredump=`/usr/bin/vdsm-tool get-conf-core_dump_enable` [ $is_coredump != true ] && is_coredump=false SYSTEMCTL_SKIP_REDIRECT=true @@ -52,12 +52,12 @@ check_port_taken() { local MANAGEMENT_PORT MANAGEMENT_IP - MANAGEMENT_PORT=`$GETCONFITEM $CONF_FILE addresses management_port ""` + MANAGEMENT_PORT=`/usr/bin/vdsm-tool get-conf-management_port` if [ -z "$MANAGEMENT_PORT" ]; then log_failure_msg "$prog: management_port not found in $CONF_FILE" return 1 fi - MANAGEMENT_IP=`$GETCONFITEM $CONF_FILE addresses management_ip 0.0.0.0` + MANAGEMENT_IP=`/usr/bin/vdsm-tool get-conf-management_ip` netstat -ntl | grep -q "$MANAGEMENT_IP:$MANAGEMENT_PORT" RETVAL=$? if [ "$RETVAL" -eq 0 ]; then @@ -69,7 +69,7 @@ mk_data_center() { local dc - dc=`$GETCONFITEM $CONF_FILE irs repository /rhev/` + dc=`/usr/bin/vdsm-tool get-conf-repository` /bin/mkdir -p "$dc" /bin/chown vdsm.kvm "$dc" } @@ -110,7 +110,7 @@ test_conflicting_conf() { local listen_tcp auth_tcp ssl - ssl=`$GETCONFITEM $CONF_FILE vars ssl true | tr A-Z a-z` + ssl=`/usr/bin/vdsm-tool get-conf-ssl` [ "$ssl" == true ] && return 0 listen_tcp="`get_libvirt_conf_item $LCONF listen_tcp`" @@ -276,7 +276,7 @@ fi local lconf qconf ldconf - local ssl=`$GETCONFITEM $CONF_FILE vars ssl true | tr A-Z a-z` + local ssl=`/usr/bin/vdsm-tool get-conf-ssl` lconf="$2" qconf="$3" @@ -511,10 +511,10 @@ fi echo $"Starting up vdsm daemon: " - local vdsm_nice=`$GETCONFITEM $CONF_FILE vars vdsm_nice -5` + local vdsm_nice=`/usr/bin/vdsm-tool get-conf-vdsm_nice` - LIBVIRT_LOG_FILTERS=`$GETCONFITEM $CONF_FILE vars libvirt_log_filters "1:libvirt 1:remote"` \ - LIBVIRT_LOG_OUTPUTS=`$GETCONFITEM $CONF_FILE vars libvirt_log_outputs "1:file:/var/log/vdsm/libvirt.log"` \ + LIBVIRT_LOG_FILTERS=`/usr/bin/vdsm-tool get-conf-libvirt_log_filters` \ + LIBVIRT_LOG_OUTPUTS=`/usr/bin/vdsm-tool get-conf-libvirt_log_outputs` \ LC_ALL=C NICELEVEL=$vdsm_nice daemon --user=vdsm @VDSMDIR@/respawn --minlifetime 10 --daemon --masterpid $RESPAWNPIDFILE $VDSM_BIN RETVAL=$? [ "$RETVAL" -eq 0 ] && log_success_msg $"$prog start" || log_failure_msg $"$prog start" -- To view, visit http://gerrit.ovirt.org/7695 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5c64de097bbaea6a8cf862b43243377e10e00391 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

6 13

Change in vdsm[master]: Add shutdown based on qemu-ga(qemu guest agent) in vdsm
by shaohef＠linux.vnet.ibm.com 02 Dec '13

02 Dec '13

Hello Bing Bu Cao, I'd like you to do a code review. Please visit http://gerrit.ovirt.org/9655 to review the following change. Change subject: Add shutdown based on qemu-ga(qemu guest agent) in vdsm ...................................................................... Add shutdown based on qemu-ga(qemu guest agent) in vdsm As previously what we agreed, the agent-assisted shutdown and fsfreeze would be handled by the qemu guest agent while oVirt-specific functionality such as Single-Sign-On would continue to be managed by the ovirt guest agent. http://www.ovirt.org/wiki/Guest_agent_proposals This patch changes the shutdown verb, add an shutdown approach by qemu guest agent Change-Id: I86977c1b717d63de21ba4818c6b66e43976d65de Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> Signed-off-by: BingBu Cao <mars(a)linux.vnet.ibm.com> --- M vdsm/libvirtvm.py M vdsm/vm.py 2 files changed, 84 insertions(+), 28 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/9655/1 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index f20968f..f1d82d7 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -1222,6 +1222,8 @@ self._releaseLock = threading.Lock() self.saveState() + self.shutdownMethodUpdate() + def _buildLease(self, domainID, volumeID, leasePath, leaseOffset): """ Add a single SANLock lease. @@ -2326,9 +2328,45 @@ physical, alloc) self.extendDriveVolume(d) + def shutdownMethodUpdate(self): + self._shutdownMethods = [] + self._shutdownMethods.append({'qemugaShutdown': + self.qemugaShutdown}) + self._shutdownMethods.append({'guestAgentShutdown': + self.guestAgentShutdown}) + self._shutdownMethods.append({'acpiShutdown': + self.acpiShutdown}) + def _acpiShutdown(self): self._dom.shutdownFlags(libvirt.VIR_DOMAIN_SHUTDOWN_ACPI_POWER_BTN) + def acpiShutdown(self, now, *args): + if utils.tobool(self.conf.get('acpiEnable', 'true')): + self.log.debug("acpiShutdown vm") + self._guestEventTime = now + self._guestEvent = 'Powering down' + self._acpiShutdown() + else: + return None + + return {'status': {'code': doneCode['code'], + 'message': 'Machine shut down'}} + + def _qemugaShutdown(self): + self._dom.shutdownFlags(libvirt.VIR_DOMAIN_SHUTDOWN_GUEST_AGENT) + + def qemugaShutdown(self, now, *args): + if utils.tobool(self.conf.get('qgaEnable', 'true')): + self.log.debug("qemugaShutdown vm") + self._guestEventTime = now + self._guestEvent = 'Powering down' + self._qemugaShutdown() + else: + return None + + return {'status': {'code': doneCode['code'], + 'message': 'Machine shut down'}} + def _getPid(self): pid = '0' try: diff --git a/vdsm/vm.py b/vdsm/vm.py index cb0e552..bf6ee8b 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -364,6 +364,8 @@ CONTROLLER_DEVICES: [], GENERAL_DEVICES: [], BALLOON_DEVICES: [], REDIR_DEVICES: [], WATCHDOG_DEVICES: []} + self._shutdownMethods = [] + self.shutdownMethodUpdate() def _get_lastStatus(self): PAUSED_STATES = ('Powering down', 'RebootInProgress', 'Up') @@ -970,35 +972,51 @@ if not guestCpuLocked: self._guestCpuLock.release() + def shutdownMethodUpdate(self): + self._shutdownMethods = [] + self._shutdownMethods.append({'guestAgentShutdown': + self._guestAgentShutdown}) + def shutdown(self, timeout, message): - try: - now = time.time() - if self.lastStatus == 'Down': - return - if self.guestAgent and self.guestAgent.isResponsive(): - self._guestEventTime = now - self._guestEvent = 'Powering down' - self.log.debug('guestAgent shutdown called') - self.guestAgent.desktopShutdown(timeout, message) - agent_timeout = (int(timeout) + - config.getint('vars', 'sys_shutdown_timeout')) - timer = threading.Timer(agent_timeout, self._timedShutdown) - timer.start() - elif utils.tobool(self.conf.get('acpiEnable', 'true')): - self._guestEventTime = now - self._guestEvent = 'Powering down' - self._acpiShutdown() - # No tools, no ACPI - else: - return { - 'status': { - 'code': errCode['exist']['status']['code'], - 'message': 'VM without ACPI or active SolidICE tools. ' - 'Try Forced Shutdown.'}} - except: - self.log.error("Shutdown failed", exc_info=True) - return {'status': {'code': errCode['exist']['status']['code'], - 'message': 'Failed to shutdown VM. Try Forced Shutdown.'}} + if self.lastStatus == 'Down': + return + now = time.time() + + status = None + for method, func in self._shutdownMethods: + try: + status = func(now, timeout, message) + except: + self.log.error("%s failed", method, exc_info=True) + status = {'status': + {'code': errCode['exist']['status']['code'], + 'message': 'Failed to shutdown VM. ' + 'Try Forced Shutdown.'}} + # if one shutdown method failed, will try another method + # the loop will stop once a shutdown method is successful + if doneCode['code'] == status['status']['code']: + return status + if not status: + return {'status': + {'code': errCode['exist']['status']['code'], + 'message': 'VM without ACPI or active SolidICE tools. ' + 'Try Forced Shutdown.'}} + + return status + + def guestAgentShutdown(self, now, timeout, message): + if self.guestAgent and self.guestAgent.isResponsive(): + self._guestEventTime = time.time() + self._guestEvent = 'Powering down' + self.log.debug('guestAgent shutdown called') + self.guestAgent.desktopShutdown(timeout, message) + agent_timeout = (int(timeout) + + config.getint('vars', 'sys_shutdown_timeout')) + timer = threading.Timer(agent_timeout, self._timedShutdown) + timer.start() + else: + return None + return {'status': {'code': doneCode['code'], 'message': 'Machine shut down'}} -- To view, visit http://gerrit.ovirt.org/9655 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I86977c1b717d63de21ba4818c6b66e43976d65de Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> Gerrit-Reviewer: Bing Bu Cao <mars(a)linux.vnet.ibm.com>

6 11

← Newer
1
...
15
16
17
18
19
20
21
...
46
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches August 2013