July 2011 - vdsm-patches - Fedora mailing-lists

[ABANDONED PATCH] [WIP] BZ#719330 - No need to spew unmounted masters to the log. (via gerrit-bot)
by smizrahi＠redhat.com 18 Jul '11

18 Jul '11

Patch was abandoned by Saggi Mizrahi (smizrahi(a)redhat.com) because: Bug is non issue and refactoring can wait You can review this change at: http://gerrit.usersys.redhat.com/683

1 0

[NEW PATCH] [WIP] BZ#719330 - No need to spew unmounted masters to the log. (via gerrit-bot)
by smizrahi＠redhat.com 18 Jul '11

18 Jul '11

New patch submitted by Saggi Mizrahi (smizrahi(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/683 commit d334c99c34d3a922a89ff75280e927235b0984db Author: Saggi Mizrahi <smizrahi(a)redhat.com> Date: Mon Jul 11 12:15:48 2011 +0300 [WIP] BZ#719330 - No need to spew unmounted masters to the log. Change-Id: Ib0044075f10171bd37613e17b46b28cd7e81dff7 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index b0d5d21..623b09f 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -607,7 +607,7 @@ class HSM: for sdUUID in domList: vars.task.getExclusiveLock(STORAGE, sdUUID) - self.spm.detachAllDomains(pool) + self.spm.destroyPool(pool) return self._disconnectPool(pool, hostID, scsiKey, remove=True) diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 8aed3e6..e9f5c90 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -595,31 +595,11 @@ class StoragePool: self.log.error("Unexpected error", exc_info=True) - def __unmountLastMaster(self, sdUUID): - curmsd = SDF.produce(sdUUID) - # Check if it's last domain and allow it detaching - dl = self.getDomains(activeOnly=True) - domList = dl.keys() - if curmsd.sdUUID in domList: - domList.remove(curmsd.sdUUID) - for item in domList: - domain = SDF.produce(item) - if domain.isData(): - # Failure, we have at least one more data domain - # in the pool and one which can become 'master' - raise se.StoragePoolHasPotentialMaster(item) - curmsd.unmountMaster() - def masterMigrate(self, sdUUID, msdUUID, masterVersion): self.log.info("sdUUID=%s spUUID=%s msdUUID=%s", sdUUID, self.spUUID, msdUUID) - # Check if we are migrating to or just unmounting last master - if msdUUID != sd.BLANK_UUID: - self.__masterMigrate(sdUUID, msdUUID, masterVersion) - return False # not last master - - self.__unmountLastMaster(sdUUID) - return True # last master + self.__masterMigrate(sdUUID, msdUUID, masterVersion) + return False # not last master def attachSD(self, sdUUID): """ @@ -651,6 +631,21 @@ class StoragePool: dom.releaseClusterLock() self.updateMonitoringThreads() + def destroy(self): + curmsd = self.getMasterDomain() + # Check if there is only 1 domain and allow it detaching + domList = self.getDomains() + if len(domList) > 1: + raise se.StoragePoolDestroyingError() + + with self._metadata.transaction(): + spUUID = self.spUUID + self._metadata[PMDK_DOMAINS] = {} + self._metadata[PMDK_MASTER_VER] = 0 + curmsd.detachSD(spUUID) + + curmsd.unmountMaster() + def forcedDetachSD(self, sdUUID): self.log.warn("Force detaching domain `%s`", sdUUID) diff --git a/vdsm/storage/spm.py b/vdsm/storage/spm.py index b186d39..a80d03b 100644 --- a/vdsm/storage/spm.py +++ b/vdsm/storage/spm.py @@ -201,8 +201,6 @@ class SPM: blockSD.BlockStorageDomain.doUnmountMaster(master) except se.StorageDomainMasterUnmountError, e: misc.panic("unmount %s failed - %s" % (master, e)) - else: - cls.log.debug("master `%s` is not mounted, skipping", master) @classmethod def __cleanupSPMLinks(cls): @@ -1070,7 +1068,7 @@ class SPM: pool.detachSD(sdUUID, msdUUID, masterVersion) - def detachAllDomains(self, pool): + def destroyPool(self, pool): """ Detach all domains from pool before destroying pool """ @@ -1086,7 +1084,8 @@ class SPM: pool.detachSD(sdUUID=sdUUID, msdUUID=sd.BLANK_UUID, masterVersion=0) self._stop(pool.spUUID) # Forced detach 'master' domain after stopping SPM - pool.detachSD(mDom.sdUUID, sd.BLANK_UUID, 0) + pool.destroy() + return True # last master def public_attachStorageDomain(self, sdUUID, spUUID, options = None):

2 3

[NEW PATCH] BZ#720425 - No longer using the return value from wait, because there is None. (via gerrit-bot)
by erez＠redhat.com 17 Jul '11

17 Jul '11

New patch submitted by Erez Sh (erez(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/684 commit b4b7c5efc10d4c0702523c69d4ac54ffba2d87b1 Author: Erez Sh <erez(a)redhat.com> Date: Mon Jul 11 18:30:48 2011 +0300 BZ#720425 - No longer using the return value from wait, because there is None. Change-Id: I1f6f22fb4e047dc24335eeef7664c33e1076eb02 diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 0516c3b..89330c4 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -214,7 +214,8 @@ class MigrationMonitorThread(threading.Thread): def run(self): self._vm.log.debug('starting migration monitor thread') - while not self._stop.wait(self._MIGRATION_MONITOR_INTERVAL): + while not self._stop.isSet(): + self._stop.wait(self._MIGRATION_MONITOR_INTERVAL) jobType, timeElapsed, _, \ dataTotal, dataProcessed, _, \ memTotal, memProcessed, _, \

3 3

Change in vdsm[master]: BZ#720425 - No longer using the return value from wait, beca...
by erez＠redhat.com 17 Jul '11

17 Jul '11

Erez Sh has posted comments on this change. Change subject: BZ#720425 - No longer using the return value from wait, because there is None. ...................................................................... Patch Set 1: Verified Ran migration tests and it worked -- To view, visit http://gerrit.usersys/684 To unsubscribe, visit http://gerrit.usersys/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1f6f22fb4e047dc24335eeef7664c33e1076eb02 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Erez Sh <erez(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Erez Sh <erez(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>

1 0

[NEW PATCH] BZ#719857 - Add reboot message to certificate reject (via gerrit-bot)
by Yotam Oron 17 Jul '11

17 Jul '11

New patch submitted by Yotam Oron (yoron(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/715 commit 50f18d38e1ab7106def66a82147bd7590e8a5e70 Author: Yotam Oron <yoron(a)redhat.com> Date: Sun Jul 17 12:07:54 2011 +0300 BZ#719857 - Add reboot message to certificate reject When the user rejects the certificate's fingerprint, the host will reboot - added a message about that. Change-Id: I6b8a60eeb56e12627e1ad3213d453a7dfb03de0d diff --git a/vdsm_reg/rhevm.py b/vdsm_reg/rhevm.py index 5ff8be6..d32c639 100755 --- a/vdsm_reg/rhevm.py +++ b/vdsm_reg/rhevm.py @@ -143,8 +143,9 @@ class Plugin(PluginBase): if deployUtil.getRhevmCert(self.rhevm_server.value(), self.rhevm_server_port.value()): path, dontCare = deployUtil.certPaths('') fp = deployUtil.generateFingerPrint(path) - approval = ButtonChoiceWindow(self.ncs.screen, \ - "Certificate Fingerprint:", fp, buttons = ['Approve', 'Reject']) + approval = ButtonChoiceWindow(self.ncs.screen, + "Certificate Fingerprint (Rejecting the finrgerprint will reboot the host):", + fp, buttons = ['Approve', 'Reject']) if 'reject' == approval: out, err, rc = deployUtil._logExec(['/sbin/reboot']) if rc is not 0:

2 2

[NEW PATCH] Related to BZ#713173 - Add killProc rollback to deleteImage (via gerrit-bot)
by Igor Lvovsky 17 Jul '11

17 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/707 commit ee4f07f9ed34f6db2aacdbc2678b4f20900ad761 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 12 16:54:31 2011 +0300 Related to BZ#713173 - Add killProc rollback to deleteImage Change-Id: I1407626ab06583f5507f3aaa8ebf2a4e4edd60f9 diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index 3da76fa..6b04047 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -261,7 +261,8 @@ class BlockVolume(volume.Volume): # wipe out the whole volume idle = config.getfloat('irs', 'idle') try: - misc.ddWatchCopy("/dev/zero", vol_path, vars.task.aborting, idle, int(size)) + misc.ddWatchCopy("/dev/zero", vol_path, vars.task.aborting, idle, int(size), + recoveryCallback=volume.baseAsyncTasksRollback) except se.ActionStopped, e: raise e except Exception, e:

3 4

[NEW PATCH] Related to BZ#719255 - Release lock if prepare volume failed (via gerrit-bot)
by Igor Lvovsky 17 Jul '11

17 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/709 commit bb1cba217e45a05dedcdc689611842b4b64be607 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Thu Jul 14 15:14:47 2011 +0300 Related to BZ#719255 - Release lock if prepare volume failed Change-Id: I0eeeda07037824bb822b8d0d1fff869959bdfefc diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index b0d5d21..ccf93d5 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1671,8 +1671,8 @@ class HSM: vol.refreshVolume() vol.prepare(rw=rw) except: - imgResource.autoRelease = True self.log.error("Prepare volume %s in domain %s failed", volUUID, sdUUID, exc_info=True) + rmanager.releaseResource(imageResourcesNamespace, imgUUID) raise

3 8

[NEW PATCH] BZ#720670 vdsm-reg: rotate correct log file (via gerrit-bot)
by Dan Kenigsberg 14 Jul '11

14 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/694 commit b033abf7a59837d2e15d191049b0f6a23dc716cc Author: Robert Munilla <rmunilla(a)redhat.com> Date: Tue Jul 12 16:58:03 2011 +0300 BZ#720670 vdsm-reg: rotate correct log file Change-Id: Ic950727ba692317be5172053e5e62d01805b7ecd diff --git a/vdsm_reg/vdsm-reg-logrotate.conf b/vdsm_reg/vdsm-reg-logrotate.conf index 45572d7..3b590dd 100644 --- a/vdsm_reg/vdsm-reg-logrotate.conf +++ b/vdsm_reg/vdsm-reg-logrotate.conf @@ -4,7 +4,7 @@ maxage 30 } -/var/log/vdsm-reg/vds-reg.log { +/var/log/vdsm-reg/vdsm-reg.log { rotate 3 missingok size=10M

2 3

[NEW PATCH] BZ#717847 - Postpone VM's periodic tasks until all volumes will prepared (via gerrit-bot)
by Igor Lvovsky 13 Jul '11

13 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/697 commit 7c44b0af0303d7a4876091cc5bfc9783dbcb5ecb Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Wed Jul 13 15:14:19 2011 +0300 BZ#717847 - Postpone VM's periodic tasks until all volumes will prepared Change-Id: I9424087c3c9ef06a4a88921dc8ca14e890b7a44f diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 40567a5..79da4b2 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -55,6 +55,10 @@ class VmStatsThread(utils.AdvancedStatsThread): if self._vm._incomingMigrationPending(): return + if not self._vm._volumesPrepared: + # Avoid queries from storage during recovery process + return + for vmDrive in self._vm._drives: if vmDrive.blockDev and vmDrive.format == 'cow': capacity, alloc, physical = \ @@ -66,7 +70,7 @@ class VmStatsThread(utils.AdvancedStatsThread): self._vm._onHighWrite(vmDrive.name, alloc) def _updateVolumes(self): - if not self._vm.cif.irs.getConnectedStoragePoolsList()['poollist']: + if not self._vm._volumesPrepared: # Avoid queries from storage during recovery process return @@ -82,6 +86,10 @@ class VmStatsThread(utils.AdvancedStatsThread): return cpuTime / 1000**3 def _sampleDisk(self): + if not self._vm._volumesPrepared: + # Avoid queries from storage during recovery process + return + diskSamples = {} for vmDrive in self._vm._drives: diskSamples[vmDrive.name] = self._vm._dom.blockStats(vmDrive.name) diff --git a/vdsm/vm.py b/vdsm/vm.py index ecc8e70..5b30b1e 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -300,6 +300,7 @@ class Vm(object): self.conf.pop('elapsedTimeOffset', 0)) self._cdromPreparedPath = '' self._floppyPreparedPath = '' + self._volumesPrepared = False self._pathsPreparedEvent = threading.Event() self.saveState() @@ -418,6 +419,8 @@ class Vm(object): def preparePaths(self): for drive in self.conf.get('drives', []): drive['path'] = self._prepareVolumePath(drive) + # Now we got all needed locks + self._volumesPrepared = True try: self._cdromPreparedPath = self._prepareVolumePath(

3 4

[NEW PATCH] BZ#720919 spmprotect.sh: remove log noise (via gerrit-bot)
by Dan Kenigsberg 13 Jul '11

13 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/696 commit cfcabd3b7bba2d6513b65216017828f326f2f660 Author: Dan Kenigsberg <danken(a)redhat.com> Date: Wed Jul 13 14:23:45 2011 +0300 BZ#720919 spmprotect.sh: remove log noise check_renew may have been called after fence or release have removed RENEWDIR. We should not print nonsense to the log in that case. Change-Id: I7ce575b863fcbdf6b0bf35747a23882a53ae1a39 diff --git a/vdsm/storage/protect/spmprotect.sh b/vdsm/storage/protect/spmprotect.sh index 3548b17..f476446 100755 --- a/vdsm/storage/protect/spmprotect.sh +++ b/vdsm/storage/protect/spmprotect.sh @@ -135,7 +135,7 @@ function check_renew() { res=0 fi fi - if pushd "$RENEWDIR" > /dev/null; then + if pushd "$RENEWDIR" > /dev/null 2>&1 ; then rm -f $list popd > /dev/null fi

2 4

[NEW PATCH] libvirtconnection: Add "VIR_FROM_RPC" to trigger prepareForShutdown. (via gerrit-bot)
by David Naori 13 Jul '11

13 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/699 commit 1ce349a644d20dfd3d71f8b8f8eda30fab8108fc Author: David Naori <dnaori(a)redhat.com> Date: Wed Jul 13 16:34:04 2011 +0300 libvirtconnection: Add "VIR_FROM_RPC" to trigger prepareForShutdown. libvirt-0.9.3-2 has change the behavior of libvirt disconnection, we now should trigger prepareForShutdown in this case too. (Reference: BZ#681911) Change-Id: I2a85b01e5d9a32aab2e9399035043d4a08856d67 diff --git a/vdsm/libvirtconnection.py b/vdsm/libvirtconnection.py index ed1ae2e..48030e2 100644 --- a/vdsm/libvirtconnection.py +++ b/vdsm/libvirtconnection.py @@ -68,8 +68,8 @@ def get(cif=None): setattr(ret, name, wrapMethod(method)) return ret except libvirt.libvirtError, e: - if (e.get_error_domain() == libvirt.VIR_FROM_REMOTE and - e.get_error_code() == libvirt.VIR_ERR_SYSTEM_ERROR): + if (e.get_error_domain() in (libvirt.VIR_FROM_REMOTE, libvirt.VIR_FROM_RPC) + and e.get_error_code() == libvirt.VIR_ERR_SYSTEM_ERROR): cif.log.error('connection to libvirt broken. ' 'taking vdsm down.') cif.prepareForShutdown()

2 3

[NEW PATCH] BZ#712106 - Add lock for template during moveImage operation (via gerrit-bot)
by Igor Lvovsky 12 Jul '11

12 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/630 commit 74ec9aede8460a5cb69269ff5420bb183b3d3110 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jun 21 13:52:20 2011 +0300 BZ#712106 - Add lock for template during moveImage operation Change-Id: Ib87649cf0aebd5a35d56faa7e8f165d10d715331 diff --git a/vdsm/constants.py b/vdsm/constants.py index ae9a79b..3a800a4 100644 --- a/vdsm/constants.py +++ b/vdsm/constants.py @@ -139,6 +139,11 @@ CMD_LOWPRIO = [EXT_NICE, '-n', '19', EXT_IONICE, '-c', '2', '-n', '7'] # # Storage constants # + +LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' +IMAGE_NAMESPACE = 'imageNS' +VOLUME_NAMESPACE = 'volumeNS' + STRG_ISCSI_HOST = "iscsi_host/" STRG_SCSI_HOST = "scsi_host/" STRG_ISCSI_SESSION = "iscsi_session/" diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 214e8a4..01b2182 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -27,7 +27,7 @@ import safelease import blockVolume import multipath import resourceFactories -from resourceFactories import LVM_ACTIVATION_NAMESPACE +from constants import LVM_ACTIVATION_NAMESPACE from persistentDict import PersistentDict, DictValidator import iscsi import storage_exception as se diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index ae411be..cebdd6b 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -24,7 +24,7 @@ import lvm import resourceManager as rm from threadLocal import vars from sdf import StorageDomainFactory as SDF -from resourceFactories import LVM_ACTIVATION_NAMESPACE +from constants import LVM_ACTIVATION_NAMESPACE import fileUtils TAG_PREFIX_MD = "MD_" diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 2aaf822..c00cb70 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -38,7 +38,7 @@ from threadLocal import vars import constants from storageConstants import STORAGE from task import Job -from resourceFactories import IMAGE_NAMESPACE +from constants import IMAGE_NAMESPACE import resourceManager as rm import devicemapper diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..f655e44 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -22,6 +22,11 @@ from config import config import storage_exception as se import task from threadLocal import vars +from constants import IMAGE_NAMESPACE +import resourceManager as rm +from contextlib import contextmanager + +rmanager = rm.ResourceManager.getInstance() # Disk type UNKNOWN_DISK_TYPE = 0 @@ -315,7 +320,8 @@ class Image: srcVol = None if not srcVol: - raise se.ImageIsNotLegalChain("Image %s is not a leaf" % imgUUID) + self.log.error("There is no leaf in the image %s" % imgUUID) + raise se.ImageIsNotLegalChain(imgUUID) # Build up the sorted (parent->child) chain while not srcVol.isShared(): @@ -450,48 +456,29 @@ class Image: """ try: if srcVol: - srcVol.teardown(sdUUID=srcVol.sdUUID, volUUID=srcVol.volUUID, justme=True) + srcVol.teardown(sdUUID=srcVol.sdUUID, volUUID=srcVol.volUUID) if dstVol: - dstVol.teardown(sdUUID=dstVol.sdUUID, volUUID=dstVol.volUUID, justme=True) + dstVol.teardown(sdUUID=dstVol.sdUUID, volUUID=dstVol.volUUID) except Exception: self.log.error("Unexpected error", exc_info=True) - def move(self, srcSdUUID, dstSdUUID, imgUUID, vmUUID, op, postZero, force): - """ - Move/Copy image between storage domains within same storage pool - """ - self.log.info("srcSdUUID=%s dstSdUUID=%s "\ - "imgUUID=%s vmUUID=%s op=%s force=%s postZero=%s", - srcSdUUID, dstSdUUID, imgUUID, vmUUID, OP_TYPES[op], str(force), str(postZero)) - - srcVol = leafVol = dstVol = None + def _createTargetImage(self, destDom, srcSdUUID, imgUUID): + # Before actual data copying we need perform several operation + # such as: create all volumes, create fake template if needed, ... try: # Find all volumes of source image - chain = self.getChain(srcSdUUID, imgUUID) - leafVol = chain[-1] - except se.StorageException, e: + srcChain = self.getChain(srcSdUUID, imgUUID) + except se.StorageException: self.log.error("Unexpected error", exc_info=True) raise except Exception, e: self.log.error("Unexpected error", exc_info=True) raise se.SourceImageActionError(imgUUID, srcSdUUID, str(e)) - destDom = SDF.produce(dstSdUUID) + fakeTemplate = False pimg = volume.BLANK_UUID # standalone chain - # If image already exists check whether it illegal/fake, overwrite it - if not self.isLegal(dstSdUUID, imgUUID): - force = True - # We must first remove the previous instance of image (if exists) - # in destination domain, if we got the overwrite command - if force: - try: - self.log.info("delete image %s on domain %s before overwriting", imgUUID, dstSdUUID) - self.delete(dstSdUUID, imgUUID, postZero, force=True) - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise e # check if the chain is build above a template, or it is a standalone - pvol = chain[0].getParentVolume() + pvol = srcChain[0].getParentVolume() if pvol: # find out parent volume parameters volParams = pvol.getVolumeParams() @@ -500,81 +487,145 @@ class Image: # FIXME: This workaround help as copy VM to the backup domain without its template # We will create fake template for future VM creation and mark it as FAKE volume # This situation is relevant for backup domain only - self.createFakeTemplate(dstSdUUID, volParams) + fakeTemplate = True + + @contextmanager + def justLogIt(pimg): + self.log.debug("You don't really need lock for image %s", pimg) + yield + + dstTmplResourcesNamespace = sd.getNamespace(destDom.sdUUID, IMAGE_NAMESPACE) + # In destination domain we need to lock image's template if exists + with rmanager.acquireResource(dstTmplResourcesNamespace, pimg, rm.LockType.shared) \ + if pimg != volume.BLANK_UUID else justLogIt(pimg): + if fakeTemplate: + self.createFakeTemplate(destDom.sdUUID, volParams) + + dstChain = [] + for srcVol in srcChain: + # Create the dst volume + try: + # find out src volume parameters + volParams = srcVol.getVolumeParams(bs=1) + + # To avoid 'prezeroing' preallocated volume on NFS domain, + # we create the target volume with minimal size and after that w'll change + # its metadata back to the original size. + tmpSize = 20480 # in sectors (10M) + destDom.createVolume(imgUUID=imgUUID, size=tmpSize, + volFormat=volParams['volFormat'], preallocate=volParams['prealloc'], + diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], + srcImgUUID=pimg, srcVolUUID=volParams['parent']) + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) + # Extend volume (for LV only) size to the actual size + dstVol.extend((volParams['apparentsize'] + 511) / 512) + # Change destination volume metadata back to the original size. + dstVol.setSize(volParams['size']) + dstChain.append(dstVol) + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.DestImageActionError(imgUUID, destDom.sdUUID, str(e)) - for srcVol in chain: - # Create the dst volume - try: - # find out src volume parameters - volParams = srcVol.getVolumeParams(bs=1) + # only base may have a different parent image + pimg = imgUUID - # To avoid 'prezeroing' preallocated volume on NFS domain, - # we create the target volume with minimal size and after that w'll change - # its metadata back to the original size. - tmpSize = 20480 # in sectors (10M) - destDom.createVolume(imgUUID=imgUUID, size=tmpSize, - volFormat=volParams['volFormat'], preallocate=volParams['prealloc'], - diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], - srcImgUUID=pimg, srcVolUUID=volParams['parent']) - # Extend volume (for LV only) size to the actual size - dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) - dstVol.extend((volParams['apparentsize'] + 511) / 512) - # Change destination volume metadata back to the original size. - dstVol.setSize(volParams['size']) - except se.StorageException, e: - self.log.error("Unexpected error", exc_info=True) - raise - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise se.DestImageActionError(imgUUID, dstSdUUID, str(e)) + return {'srcChain':srcChain, 'dstChain':dstChain} - # Do the actual copy - try: + def _interImagesCopy(self, destDom, srcSdUUID, imgUUID, chains): + srcLeafVol = chains['srcChain'][-1] + dstLeafVol = chains['dstChain'][-1] + try: + # Prepare the whole chains before the copy + srcLeafVol.prepare(rw=False) + dstLeafVol.prepare(rw=True, chainrw=True, setrw=True) + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + # teardown volumes + self.__cleanupMove(srcLeafVol, dstLeafVol) + raise + + try: + for srcVol in chains['srcChain']: + # Do the actual copy try: + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) srcSize = srcVol.getVolumeSize(bs=1) - # Prepare relevant volumes before copy and teardown after that - srcVol.prepare(rw=False, justme=True) - dstVol.prepare(rw=True, justme=True, setrw=True) misc.ddWatchCopy(srcVol.getVolumePath(), dstVol.getVolumePath(), vars.task.aborting, self.idle, size=srcSize) - # In case of copying template, we should set the destination volume - # as SHARED (after copy because otherwise prepare as RW would fail) - if srcVol.isShared(): - dstVol.setShared() - elif srcVol.isInternal(): - dstVol.setInternal() - except se.ActionStopped, e: - raise e - except se.StorageException, e: + except se.ActionStopped: + raise + except se.StorageException: self.log.error("Unexpected error", exc_info=True) raise except Exception, e: self.log.error("Unexpected error", exc_info=True) - raise se.CopyImageError("image=%s, src domain=%s, dst domain=%s: msg %s" % (imgUUID, srcSdUUID, dstSdUUID, str(e))) - finally: - # teardown volumes - self.__cleanupMove(srcVol, dstVol) - srcVol = dstVol = None + raise se.CopyImageError("image=%s, src domain=%s, dst domain=%s: msg %s" % (imgUUID, srcSdUUID, destDom.sdUUID, str(e))) + finally: + # teardown volumes + self.__cleanupMove(srcLeafVol, dstLeafVol) - # only base may have a different parent image - pimg = imgUUID + def _finalizeDestinationImage(self, destDom, imgUUID, chains, force): + for srcVol in chains['srcChain']: + try: + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) + # In case of copying template, we should set the destination volume + # as SHARED (after copy because otherwise prepare as RW would fail) + if srcVol.isShared(): + dstVol.setShared() + elif srcVol.isInternal(): + dstVol.setInternal() + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.DestImageActionError(imgUUID, destDom.sdUUID, str(e)) if force: + leafVol = chains['dstChain'][-1] # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, imgUUID, leafVol.volUUID) + self.__templateRelink(destDom.sdUUID, imgUUID, leafVol.volUUID) # At this point we successfully finished the 'copy' part of the operation # and we can clear all recoveries. vars.task.clearRecoveries() + + def _removeImage(self, sdUUID, imgUUID, postZero, force): + try: + self.delete(sdUUID, imgUUID, postZero, force) + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.ImageDeleteError("%s: %s" % (imgUUID), str(e)) + + def move(self, srcSdUUID, dstSdUUID, imgUUID, vmUUID, op, postZero, force): + """ + Move/Copy image between storage domains within same storage pool + """ + self.log.info("srcSdUUID=%s dstSdUUID=%s "\ + "imgUUID=%s vmUUID=%s op=%s force=%s postZero=%s", + srcSdUUID, dstSdUUID, imgUUID, vmUUID, OP_TYPES[op], str(force), str(postZero)) + + destDom = SDF.produce(dstSdUUID) + # If image already exists check whether it illegal/fake, overwrite it + if not self.isLegal(destDom.sdUUID, imgUUID): + force = True + # We must first remove the previous instance of image (if exists) + # in destination domain, if we got the overwrite command + if force: + self.log.info("delete image %s on domain %s before overwriting", imgUUID, destDom.sdUUID) + self._removeImage(destDom.sdUUID, imgUUID, postZero, force=True) + + chains = self._createTargetImage(destDom, srcSdUUID, imgUUID) + self._interImagesCopy(destDom, srcSdUUID, imgUUID, chains) + self._finalizeDestinationImage(destDom, imgUUID, chains, force) # If it's 'move' operation, we should delete src image after copying if op == MOVE_OP: - try: - self.delete(srcSdUUID, imgUUID, postZero, force=True) - except se.StorageException, e: - self.log.error("Unexpected error", exc_info=True) - raise - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise se.ImageDeleteError("%s: %s" % (imgUUID), str(e)) + self._removeImage(srcSdUUID, imgUUID, postZero, force=True) self.log.info("%s task on image %s was successfully finished", OP_TYPES[op], imgUUID) return True diff --git a/vdsm/storage/resourceFactories.py b/vdsm/storage/resourceFactories.py index ce3ec70..4836217 100644 --- a/vdsm/storage/resourceFactories.py +++ b/vdsm/storage/resourceFactories.py @@ -16,10 +16,7 @@ import storage_exception as se from sdf import StorageDomainFactory as SDF import sd import image - -LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' -IMAGE_NAMESPACE = 'imageNS' -VOLUME_NAMESPACE = 'volumeNS' +from constants import VOLUME_NAMESPACE rmanager = rm.ResourceManager.getInstance() diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 325da5d..4cdaef9 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -17,7 +17,7 @@ from glob import glob import storage_exception as se import misc import resourceFactories -from resourceFactories import IMAGE_NAMESPACE, VOLUME_NAMESPACE +from constants import IMAGE_NAMESPACE, VOLUME_NAMESPACE import resourceManager as rm import constants import safelease diff --git a/vdsm/storage/spm.py b/vdsm/storage/spm.py index b186d39..6ed480b 100644 --- a/vdsm/storage/spm.py +++ b/vdsm/storage/spm.py @@ -36,7 +36,7 @@ import logging import storage_exception as se from threadLocal import vars from storageConstants import STORAGE -from resourceFactories import IMAGE_NAMESPACE +from constants import IMAGE_NAMESPACE import resourceManager as rm from contextlib import nested import fileUtils diff --git a/vdsm/storage/storage_exception.py b/vdsm/storage/storage_exception.py index b1cbcdf..c9df109 100644 --- a/vdsm/storage/storage_exception.py +++ b/vdsm/storage/storage_exception.py @@ -274,7 +274,7 @@ class ImageValidationError(StorageException): class ImageDeleteError(StorageException): code = 256 - message = "Could not remove all files" + message = "Could not remove all image's volumes" #REMOVED in 2.3. #This class was not in use in 2.2 nor in 2.3. diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py index 123ed37..0052e84 100644 --- a/vdsm/storage/volume.py +++ b/vdsm/storage/volume.py @@ -21,8 +21,9 @@ import misc import fileUtils import task from threadLocal import vars -import resourceFactories import resourceManager as rm +from constants import IMAGE_NAMESPACE + rmanager = rm.ResourceManager.getInstance() @@ -150,7 +151,7 @@ class Volume: cls.log.info("sdUUID=%s srcImg=%s srcVol=%s dstFormat=%s srcParent=%s", sdUUID, srcImg, srcVol, dstFormat, srcParent) - imageResourcesNamespace = sd.getNamespace(sdUUID, resourceFactories.IMAGE_NAMESPACE) + imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) with rmanager.acquireResource(imageResourcesNamespace, srcImg, rm.LockType.exclusive): try: vol = SDF.produce(sdUUID).produceVolume(imgUUID=srcImg, volUUID=srcVol)

3 9

[NEW PATCH] BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains (via gerrit-bot)
by Igor Lvovsky 12 Jul '11

12 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/646 commit e76296ddddfeecb08c56e2a3f1e9960751442b8a Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:00:21 2011 +0300 BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains Change-Id: I367e83a700e84be0e4cb0ca5b7a02416cb54ac25 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 51e237b..4fc6f5b 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -375,21 +375,26 @@ class Image: """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - # Avoid relink templates for SAN domains - if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: - vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) - for ch in chList: - # Remove hardlink of this template - v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) - v.delete(postZero=False, force=True) - - # Now we should re-link deleted hardlink, if exists - newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) - newVol.share(imageDir) - else: + # Avoid relink templates for non-NFS domains + if destDom.getStorageType() not in [ sd.NFS_DOMAIN ]: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) + return + + vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) + # Relink templates only + if not vol.isShared(): + self.log.debug("Doesn't relink regular volume %s of image %s", volUUID, imgUUID) + return + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) + for ch in chList: + # Remove hardlink of this template + v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) + v.delete(postZero=False, force=True) + + # Now we should re-link deleted hardlink, if exists + newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) + newVol.share(imageDir) def createFakeTemplate(self, sdUUID, volParams): """

3 5

[NEW PATCH] Related to BZ#717952 - Use 'with' instead of old lock.acquire and avoid unneeded SDF.produce(sdUUID) (via gerrit-bot)
by Igor Lvovsky 12 Jul '11

12 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/680 commit b25ca55f1282f1c5409bb4f8daf81efa9533a666 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:03:30 2011 +0300 Related to BZ#717952 - Use 'with' instead of old lock.acquire and avoid unneeded SDF.produce(sdUUID) Change-Id: Ia236b673cd1887b094e5a56cacaf09ee1430ffd4 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..51e237b 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -50,7 +50,7 @@ class Image: Consist from chain of volumes. """ log = logging.getLogger('Storage.Image') - _lock = threading.Lock() + _fakeTemplateLock = threading.Lock() @classmethod def createImageRollback(cls, taskObj, imageDir): @@ -371,15 +371,14 @@ class Image: self.log.error("Unexpected error", exc_info=True) raise se.CouldNotValideTemplateOnTargetDomain("Template %s Destination domain %s: %s" % (pimg, dstSdUUID, str(e))) - def __templateRelink(self, sdUUID, imgUUID, volUUID): + def __templateRelink(self, destDom, imgUUID, volUUID): """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - destDom = SDF.produce(sdUUID) # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, sdUUID, imgUUID, volUUID) + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) @@ -387,7 +386,7 @@ class Image: # Now we should re-link deleted hardlink, if exists newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(sdUUID, ch['imgUUID']) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) newVol.share(imageDir) else: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) @@ -396,8 +395,7 @@ class Image: """ Create fake template (relevant for Backup domain only) """ - try: - self._lock.acquire() + with self._fakeTemplateLock: try: destDom = SDF.produce(sdUUID) volclass = destDom.getVolumeClass() @@ -417,14 +415,12 @@ class Image: # Mark fake volume as shared vol.setShared() # Now we should re-link all hardlinks of this template in all VMs based on it - self.__templateRelink(sdUUID, volParams['imgUUID'], volParams['volUUID']) + self.__templateRelink(destDom, volParams['imgUUID'], volParams['volUUID']) self.log.debug("Succeeded to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID) except Exception: self.log.error("Failure to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID, exc_info=True) - finally: - self._lock.release() def isLegal(self, sdUUID, imgUUID): """ @@ -763,7 +759,7 @@ class Image: if force: # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, dstImgUUID, dstVolUUID) + self.__templateRelink(destDom, dstImgUUID, dstVolUUID) except se.StorageException, e: self.log.error("Unexpected error", exc_info=True) raise

3 3

[NEW PATCH] BZ#720359: Remove lockfile instead of stop libvirt-guests. (via gerrit-bot)
by David Naori 12 Jul '11

12 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/687 commit 3a787a84a78f1b9ea202e3c10df2985dc8382d3c Author: David Naori <dnaori(a)redhat.com> Date: Tue Jul 12 12:17:56 2011 +0300 BZ#720359: Remove lockfile instead of stop libvirt-guests. Stop libvirt-guests using Sys-V-Init script tries to access libvirtd which requires sasl authentication. Instead removing lockfile signifies to libvirt-guests that his stopped. Change-Id: Id74ce865ad06f7c3de86f937ff17f6f35affa818 diff --git a/vdsm/vdsmd b/vdsm/vdsmd index 6b35e23..d27fbaf 100755 --- a/vdsm/vdsmd +++ b/vdsm/vdsmd @@ -130,7 +130,11 @@ shutdown_conflicting_srv() { /sbin/chkconfig $srv off if /sbin/service $srv status > /dev/null 2>&1; then - /sbin/service $srv stop + if [ "$serv" == "libvirt-guests" ]; then + rm -f /var/lock/subsys/libvirt-guests + else + /sbin/service $srv stop + fi fi done return 0

2 7

Change in vdsm[master]: Related to BZ#719516: Make list nicer and include listNames ...
by Dan Kenigsberg 12 Jul '11

12 Jul '11

Dan Kenigsberg has submitted this change and it was merged. Change subject: Related to BZ#719516: Make list nicer and include listNames inside. ...................................................................... Related to BZ#719516: Make list nicer and include listNames inside. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c --- M vdsm_cli/vdsClient.py 1 file changed, 37 insertions(+), 44 deletions(-) Approvals: Dan Kenigsberg: Looks good to me, approved David Naori: Verified Erez Sh: Looks good to me, but someone else must approve -- To view, visit http://gerrit.usersys/676 To unsubscribe, visit http://gerrit.usersys/settings Gerrit-MessageType: merged Gerrit-Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: David Naori <dnaori(a)redhat.com> Gerrit-Reviewer: Ayal Baron Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: David Naori <dnaori(a)redhat.com> Gerrit-Reviewer: Erez Sh <erez(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>

1 0

Change in vdsm[master]: Related to BZ#719516: Make list nicer and include listNames ...
by Dan Kenigsberg 12 Jul '11

12 Jul '11

Dan Kenigsberg has posted comments on this change. Change subject: Related to BZ#719516: Make list nicer and include listNames inside. ...................................................................... Patch Set 7: Looks good to me, approved -- To view, visit http://gerrit.usersys/676 To unsubscribe, visit http://gerrit.usersys/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: David Naori <dnaori(a)redhat.com> Gerrit-Reviewer: Ayal Baron Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: David Naori <dnaori(a)redhat.com> Gerrit-Reviewer: Erez Sh <erez(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>

1 0

Change in vdsm[master]: BZ#720425 - No longer using the return value from wait, beca...
by Dan Kenigsberg 11 Jul '11

11 Jul '11

Dan Kenigsberg has posted comments on this change. Change subject: BZ#720425 - No longer using the return value from wait, because there is None. ...................................................................... Patch Set 1: Looks good to me, approved -- To view, visit http://gerrit.usersys/684 To unsubscribe, visit http://gerrit.usersys/settings Gerrit-MessageType: comment Gerrit-Change-Id: I1f6f22fb4e047dc24335eeef7664c33e1076eb02 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Erez Sh <erez(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>

1 0

[NEW PATCH] Related to BZ#719516: Make list nicer and include listNames inside. (via gerrit-bot)
by David Naori 11 Jul '11

11 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/676 commit 8e24c353d97f53ba139a504d11ac80891c3e4cff Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 16:21:15 2011 +0300 Related to BZ#719516: Make list nicer and include listNames inside. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 38640cc..32221a7 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,47 +173,40 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True - response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: - if table: + vmListViews = ['table', 'long', 'ids'] + view = 'long' #Default view + + if args: + view = args[0] + if view not in vmListViews: + raise ValueError('Invalid argument "%s".' % args[0]) + if view == 'table': allStats = {} for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - if id not in allStats: - continue - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) - sys.exit(response['status']['code']) + allStats[ s['vmId'] ] = s + + response = self.s.list(True) + for conf in response['vmList']: + if view == 'long': + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + + elif view == 'table': + vmId = conf['vmId'] + if vmId not in allStats: + continue + status = conf['status'] + if allStats[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, allStats[vmId].get('guestIPs', '') ) + + elif view == 'ids': + print conf['vmId'] - def do_listNames(self, args): - response = self.s.list() - if response['status']['code'] != 0: - print response['status']['message'] - else: - names = [] - for conf in response['vmList']: - names.append(conf['vmId']) - names.sort() - if names: - print '\n'.join(names) sys.exit(response['status']['code']) def do_destroy(self, args): @@ -1525,12 +1518,12 @@ if __name__ == '__main__': 'Stops the emulation and graceful shutdown the virtual machine.' )), 'list' : ( serv.do_list, - ('[table]', - 'Lists all available machines on the specified server and all available configuration info', - 'If table modifier added then show table with the fields: vmId vmName Status IP' - )), - 'listNames' : ( serv.do_listNames, - ('Lists all available machines on the specified server','' + ('[view]', + 'Lists all available machines on the specified server.', + 'Optional views:', + ' "long" all available configuration info (Default).', + ' "table" table output with the fields: vmId, vmName, Status and IP.', + ' "ids" all vmIds.' )), 'pause' : ( serv.do_pause, ('<vmId>',

1 1

[ABANDONED PATCH] BZ#699976 Related, use RHEVM leasetime and timeout (via gerrit-bot)
by Federico Simoncelli 11 Jul '11

11 Jul '11

Patch was abandoned by Federico Simoncelli (fsimonce(a)redhat.com) because: Abandoned in favor of: http://gerrit.usersys.redhat.com/677 You can review this change at: http://gerrit.usersys.redhat.com/652

1 0

Change in vdsm[master]: Related to BZ#719516: Make list nicer and include listNames ...
by erez＠redhat.com 11 Jul '11

11 Jul '11

Erez Sh has posted comments on this change. Change subject: Related to BZ#719516: Make list nicer and include listNames inside. ...................................................................... Patch Set 7: Looks good to me, but someone else must approve -- To view, visit http://gerrit.usersys/676 To unsubscribe, visit http://gerrit.usersys/settings Gerrit-MessageType: comment Gerrit-Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: David Naori <dnaori(a)redhat.com> Gerrit-Reviewer: Ayal Baron Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: David Naori <dnaori(a)redhat.com> Gerrit-Reviewer: Erez Sh <erez(a)redhat.com> Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>

1 0

[NEW PATCH] Related to BZ#719516: Make list nicer and include listNames inside. (via gerrit-bot)
by David Naori 11 Jul '11

11 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/676 commit b2bf639c6d47e89b4eef9c87345e63a7c142433b Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 16:21:15 2011 +0300 Related to BZ#719516: Make list nicer and include listNames inside. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 38640cc..3aa88ec 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,47 +173,40 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True - response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: - if table: + vmListViews = ['table', 'long', 'ids'] + view = 'long' #Default view + + if args: + view = args[0] + if view not in vmListViews: + raise ValueError() + if view == 'table': allStats = {} for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - if id not in allStats: - continue - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) - sys.exit(response['status']['code']) + allStats[ s['vmId'] ] = s + + response = self.s.list(True) + for conf in response['vmList']: + if view == 'long': + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + + elif view == 'table': + vmId = conf['vmId'] + if vmId not in allStats: + continue + status = conf['status'] + if allStats[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, allStats[vmId].get('guestIPs', '') ) + + elif view == 'ids': + print conf['vmId'] - def do_listNames(self, args): - response = self.s.list() - if response['status']['code'] != 0: - print response['status']['message'] - else: - names = [] - for conf in response['vmList']: - names.append(conf['vmId']) - names.sort() - if names: - print '\n'.join(names) sys.exit(response['status']['code']) def do_destroy(self, args): @@ -1525,12 +1518,12 @@ if __name__ == '__main__': 'Stops the emulation and graceful shutdown the virtual machine.' )), 'list' : ( serv.do_list, - ('[table]', - 'Lists all available machines on the specified server and all available configuration info', - 'If table modifier added then show table with the fields: vmId vmName Status IP' - )), - 'listNames' : ( serv.do_listNames, - ('Lists all available machines on the specified server','' + ('[view]', + 'Lists all available machines on the specified server.', + 'Optional views:', + ' "long" all available configuration info (Default).', + ' "table" table output with the fields: vmId, vmName, Status and IP.', + ' "ids" all vmIds.' )), 'pause' : ( serv.do_pause, ('<vmId>',

1 0

[NEW PATCH] Related to BZ#719516 vdsClient: Make list nicer. (via gerrit-bot)
by David Naori 11 Jul '11

11 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/676 commit f63d90a13ce03681ed284a632498f9345a8bbc75 Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 16:21:15 2011 +0300 Related to BZ#719516 vdsClient: Make list nicer. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 38640cc..f539fc2 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,47 +173,40 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True - response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: - if table: + vmListViews = ['table', 'long', 'ids'] + view = 'long' #Default view + + if args: + view = args[0] + if view not in vmListViews: + raise ValueError + if view == 'table': allStats = {} for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - if id not in allStats: - continue - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) - sys.exit(response['status']['code']) + allStats[ s['vmId'] ] = s + + response = self.s.list(True) + for conf in response['vmList']: + if view == 'long': + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + + elif view == 'table': + vmId = conf['vmId'] + if vmId not in allStats: + continue + status = conf['status'] + if allStats[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, allStats[vmId].get('guestIPs', '') ) + + elif view == 'ids': + print conf['vmId'] - def do_listNames(self, args): - response = self.s.list() - if response['status']['code'] != 0: - print response['status']['message'] - else: - names = [] - for conf in response['vmList']: - names.append(conf['vmId']) - names.sort() - if names: - print '\n'.join(names) sys.exit(response['status']['code']) def do_destroy(self, args): @@ -1525,12 +1518,12 @@ if __name__ == '__main__': 'Stops the emulation and graceful shutdown the virtual machine.' )), 'list' : ( serv.do_list, - ('[table]', - 'Lists all available machines on the specified server and all available configuration info', - 'If table modifier added then show table with the fields: vmId vmName Status IP' - )), - 'listNames' : ( serv.do_listNames, - ('Lists all available machines on the specified server','' + ('[view]', + 'Lists all available machines on the specified server.', + 'Optional views:', + ' "long" all available configuration info (Default).', + ' "table" table output with the fields: vmId, vmName, Status and IP.', + ' "ids" all vmIds.' )), 'pause' : ( serv.do_pause, ('<vmId>',

1 0

[MERGED PATCH] BZ#699976 Related, use RHEVM leasetime and timeout (via gerrit-bot)
by Dan Kenigsberg 11 Jul '11

11 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/677

1 0

[NEW PATCH] BZ#699976 Related, use RHEVM leasetime and timeout (via gerrit-bot)
by Dan Kenigsberg 11 Jul '11

11 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/677 commit 3aa8424b78c3cd4c591c0234836122dea053ef90 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Thu Jul 7 14:04:05 2011 +0000 BZ#699976 Related, use RHEVM leasetime and timeout Use the values provided by RHEV-M for LOCK_RENEWAL_INTERVAL, LEASE_TIME, LEASE_RETRIES and IO_OP_TIMEOUT. Change-Id: If326eb1955e771e303f4a39943142b3007187c6c diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 41363ba..4422487 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -192,7 +192,7 @@ def sizeStr2Int(size_str): def intOrDefault(default, val): try: - int(val) + return int(val) except ValueError: return default @@ -388,6 +388,12 @@ class StorageDomain: def acquireClusterLock(self, hostID): self.refresh() + self._clusterLock.setParams( + self.getMetaParam(DMDK_LOCK_RENEWAL_INTERVAL_SEC), + self.getMetaParam(DMDK_LEASE_TIME_SEC), + self.getMetaParam(DMDK_LEASE_RETRIES), + self.getMetaParam(DMDK_IO_OP_TIMEOUT_SEC) + ) self._clusterLock.acquire(hostID) def releaseClusterLock(self):

1 0

[NEW PATCH] BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains (via gerrit-bot)
by Igor Lvovsky 11 Jul '11

11 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/646 commit f8e9eccdad40b074e4eb9a24f038635bbba2c48b Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:00:21 2011 +0300 BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains Change-Id: I367e83a700e84be0e4cb0ca5b7a02416cb54ac25 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 51e237b..c1e3ec9 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -378,6 +378,10 @@ class Image: # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) + # Relink templates only + if not vol.isShared(): + self.log.debug("Doesn't relink regular volume %s of image %s", volUUID, imgUUID) + return chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template

1 0

[NEW PATCH] Related to BZ#717952 - Use 'with' instead of old lock.acquire and avoid unneeded SDF.produce(sdUUID) (via gerrit-bot)
by Igor Lvovsky 11 Jul '11

11 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/680 commit a0d0fb1a2d79c7f0b4ba15a888e1fc78cc8c751f Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:03:30 2011 +0300 Related to BZ#717952 - Use 'with' instead of old lock.acquire and avoid unneeded SDF.produce(sdUUID) Change-Id: Ia236b673cd1887b094e5a56cacaf09ee1430ffd4 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..51e237b 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -50,7 +50,7 @@ class Image: Consist from chain of volumes. """ log = logging.getLogger('Storage.Image') - _lock = threading.Lock() + _fakeTemplateLock = threading.Lock() @classmethod def createImageRollback(cls, taskObj, imageDir): @@ -371,15 +371,14 @@ class Image: self.log.error("Unexpected error", exc_info=True) raise se.CouldNotValideTemplateOnTargetDomain("Template %s Destination domain %s: %s" % (pimg, dstSdUUID, str(e))) - def __templateRelink(self, sdUUID, imgUUID, volUUID): + def __templateRelink(self, destDom, imgUUID, volUUID): """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - destDom = SDF.produce(sdUUID) # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, sdUUID, imgUUID, volUUID) + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) @@ -387,7 +386,7 @@ class Image: # Now we should re-link deleted hardlink, if exists newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(sdUUID, ch['imgUUID']) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) newVol.share(imageDir) else: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) @@ -396,8 +395,7 @@ class Image: """ Create fake template (relevant for Backup domain only) """ - try: - self._lock.acquire() + with self._fakeTemplateLock: try: destDom = SDF.produce(sdUUID) volclass = destDom.getVolumeClass() @@ -417,14 +415,12 @@ class Image: # Mark fake volume as shared vol.setShared() # Now we should re-link all hardlinks of this template in all VMs based on it - self.__templateRelink(sdUUID, volParams['imgUUID'], volParams['volUUID']) + self.__templateRelink(destDom, volParams['imgUUID'], volParams['volUUID']) self.log.debug("Succeeded to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID) except Exception: self.log.error("Failure to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID, exc_info=True) - finally: - self._lock.release() def isLegal(self, sdUUID, imgUUID): """ @@ -763,7 +759,7 @@ class Image: if force: # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, dstImgUUID, dstVolUUID) + self.__templateRelink(destDom, dstImgUUID, dstVolUUID) except se.StorageException, e: self.log.error("Unexpected error", exc_info=True) raise

1 0

[NEW PATCH] Revert "BZ#701398 Set monitorResponse using controlInfo" (via gerrit-bot)
by Dan Kenigsberg 10 Jul '11

10 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/681 commit 18e6933884d7411a359d01fe58289ae528597df0 Author: Dan Kenigsberg <danken(a)redhat.com> Date: Sun Jul 10 19:13:01 2011 +0300 Revert "BZ#701398 Set monitorResponse using controlInfo" This reverts commit e1a615c9ad91fe5f7fe147cded028954e80ccf08. Libvirt-0.9.3 is not yet ready for our consumption; Furthermore, the reverted commit does not solve the case of blocked libvirtd. Change-Id: I1e6286db657b2532835f0d9cc82d6379b2b8aeee diff --git a/vdsm.spec.in b/vdsm.spec.in index 8f01acc..f805602 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -28,7 +28,7 @@ Requires: psmisc >= 22.6-15.el6_0.1 Requires: fence-agents Requires: bridge-utils Requires: sos -Requires: libvirt >= 0.9.3 +Requires: libvirt >= 0.8.7-5 Requires: libvirt-python Requires: dosfstools Requires: policycoreutils-python diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 0516c3b..40567a5 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -283,6 +283,35 @@ class MigrationSourceThread(vm.MigrationSourceThread): if MigrationMonitorThread._MIGRATION_MONITOR_INTERVAL: monitorThread.stop() +class TimeoutError(libvirt.libvirtError): pass + +class NotifyingVirDomain: + # virDomain wrapper that notifies vm when a method raises an exception with + # get_error_code() = VIR_ERR_OPERATION_TIMEOUT + + def __init__(self, dom, tocb): + self._dom = dom + self._cb = tocb + + def __getattr__(self, name): + attr = getattr(self._dom, name) + if not callable(attr): + return attr + def f(*args, **kwargs): + try: + ret = attr(*args, **kwargs) + self._cb(False) + return ret + except libvirt.libvirtError, e: + if e.get_error_code() == libvirt.VIR_ERR_OPERATION_TIMEOUT: + self._cb(True) + toe = TimeoutError(e.get_error_message()) + toe.err = e.err + raise toe + raise + return f + + class _DomXML: def __init__(self, conf, log): """ @@ -888,7 +917,9 @@ class LibvirtVm(vm.Vm): domxml = hooks.before_vm_start(self._buildCmdLine(), self.conf) self.log.debug(domxml) if 'recover' in self.conf: - self._dom = self._connection.lookupByUUIDString(self.id) + self._dom = NotifyingVirDomain( + self._connection.lookupByUUIDString(self.id), + self._timeoutExperienced) elif 'restoreState' in self.conf: hooks.before_vm_dehibernate(self.conf.pop('_srcDomXML'), self.conf) @@ -898,14 +929,18 @@ class LibvirtVm(vm.Vm): finally: self.cif._teardownVolumePath(self.conf['restoreState']) - self._dom = self._connection.lookupByUUIDString(self.id) + self._dom = NotifyingVirDomain( + self._connection.lookupByUUIDString(self.id), + self._timeoutExperienced) else: flags = libvirt.VIR_DOMAIN_NONE if 'launchPaused' in self.conf: flags |= libvirt.VIR_DOMAIN_START_PAUSED self.conf['pauseCode'] = 'NOERR' del self.conf['launchPaused'] - self._dom = self._connection.createXML(domxml, flags) + self._dom = NotifyingVirDomain( + self._connection.createXML(domxml, flags), + self._timeoutExperienced) if self._dom.UUIDString() != self.id: raise Exception('libvirt bug 603494') hooks.after_vm_start(self._dom.XMLDesc(0), self.conf) @@ -933,25 +968,11 @@ class LibvirtVm(vm.Vm): def _monitorDependentInit(self, timeout=None): self.log.warning('unsupported by libvirt vm') - def _getMonitorResponse(self): - if not self._dom: - return 0 - - state, details, stateTime = self._dom.controlInfo(0) - stateTime /= 1000.0 - - if (state == libvirt.VIR_DOMAIN_CONTROL_OCCUPIED and - stateTime > config.getint('vars', 'vm_command_timeout')): - return -1 - - return 0 - - def _setMonitorResponse(self, value): - if value != 0: - self.log.debug("failed to set monitorResponse to %s", - value, exc_info=True) - - _monitorResponse = property(_getMonitorResponse, _setMonitorResponse) + def _timeoutExperienced(self, timeout): + if timeout: + self._monitorResponse = -1 + else: + self._monitorResponse = 0 def _waitForIncomingMigrationFinish(self): if 'restoreState' in self.conf: @@ -965,7 +986,9 @@ class LibvirtVm(vm.Vm): try: # Would fail if migration isn't successful, # or restart vdsm if connection to libvirt was lost - self._dom = self._connection.lookupByUUIDString(self.id) + self._dom = NotifyingVirDomain( + self._connection.lookupByUUIDString(self.id), + self._timeoutExperienced) except Exception, e: # Improve description of exception if not self._incomingMigrationFinished.isSet(): diff --git a/vdsm/utils.py b/vdsm/utils.py index bd62925..89f786f 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -407,16 +407,20 @@ class StatsThread(threading.Thread): pass def run(self): + import libvirtvm try: # wait a bit before starting to sample time.sleep(self.SAMPLE_INTERVAL_SEC) while not self._stopEvent.isSet(): if not self._paused: - sample = self.sample() - self._samples.append(sample) - self._lastSampleTime = sample.timestamp - if len(self._samples) > self.AVERAGING_WINDOW: - self._samples.pop(0) + try: + sample = self.sample() + self._samples.append(sample) + self._lastSampleTime = sample.timestamp + if len(self._samples) > self.AVERAGING_WINDOW: + self._samples.pop(0) + except libvirtvm.TimeoutError: + self._log.error(traceback.format_exc()) self._stopEvent.wait(self.SAMPLE_INTERVAL_SEC) except: if not self._stopEvent.isSet(): diff --git a/vdsm/vm.py b/vdsm/vm.py index 3653d82..ecc8e70 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -796,6 +796,10 @@ class Vm(object): try: if self._vmStats: decStats = self._vmStats.get() + if (not self._migrationSourceThread.isAlive() + and decStats['statsAge'] > config.getint('vars', + 'vm_command_timeout')): + stats['monitorResponse'] = '-1' except: self.log.error("Error fetching vm stats", exc_info=True) for var in decStats:

1 0

[NEW PATCH] BZ#705058 - Don't start monitoring domains when reconstructing master (via gerrit-bot)
by Yotam Oron 10 Jul '11

10 Jul '11

New patch submitted by Yotam Oron (yoron(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/634 commit c4bac51ca8afabd41144097bed81068e76b75f44 Author: Yotam Oron <yoron(a)redhat.com> Date: Mon Jun 27 15:22:39 2011 +0300 BZ#705058 - Don't start monitoring domains when reconstructing master During master recostruction there is no point in updating the repo stats monitoring threads (this is cause a repo stats thread reference not to be inserted to the list of monitoring threads and hence it leaks) Change-Id: I05914b8c3a8f304e0acda5463e1ad00630c3233b diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index c377ee4..543984c 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1156,7 +1156,6 @@ class StoragePool: del domDict[sdUUID] self.setMetaParam(PMDK_DOMAINS, domDict) self.log.info("Set storage pool domains: %s", domDict) - self.updateMonitoringThreads() if refresh: self.refresh()

2 2

[NEW PATCH] BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains (via gerrit-bot)
by Igor Lvovsky 10 Jul '11

10 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/646 commit 64badb305015f011ee2f0d84daa39563f3f17db4 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:00:21 2011 +0300 BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains Change-Id: I367e83a700e84be0e4cb0ca5b7a02416cb54ac25 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 51e237b..c1e3ec9 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -378,6 +378,10 @@ class Image: # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) + # Relink templates only + if not vol.isShared(): + self.log.debug("Doesn't relink regular volume %s of image %s", volUUID, imgUUID) + return chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template

1 0

[NEW PATCH] Related to BZ#717952 - Use 'with' instead of old lock.acquire (via gerrit-bot)
by Igor Lvovsky 10 Jul '11

10 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/680 commit fbf7dc40ccf87f7c0d7246b9f0b20d4ff54a2b76 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:03:30 2011 +0300 Related to BZ#717952 - Use 'with' instead of old lock.acquire Change-Id: Ia236b673cd1887b094e5a56cacaf09ee1430ffd4 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..51e237b 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -50,7 +50,7 @@ class Image: Consist from chain of volumes. """ log = logging.getLogger('Storage.Image') - _lock = threading.Lock() + _fakeTemplateLock = threading.Lock() @classmethod def createImageRollback(cls, taskObj, imageDir): @@ -371,15 +371,14 @@ class Image: self.log.error("Unexpected error", exc_info=True) raise se.CouldNotValideTemplateOnTargetDomain("Template %s Destination domain %s: %s" % (pimg, dstSdUUID, str(e))) - def __templateRelink(self, sdUUID, imgUUID, volUUID): + def __templateRelink(self, destDom, imgUUID, volUUID): """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - destDom = SDF.produce(sdUUID) # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, sdUUID, imgUUID, volUUID) + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) @@ -387,7 +386,7 @@ class Image: # Now we should re-link deleted hardlink, if exists newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(sdUUID, ch['imgUUID']) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) newVol.share(imageDir) else: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) @@ -396,8 +395,7 @@ class Image: """ Create fake template (relevant for Backup domain only) """ - try: - self._lock.acquire() + with self._fakeTemplateLock: try: destDom = SDF.produce(sdUUID) volclass = destDom.getVolumeClass() @@ -417,14 +415,12 @@ class Image: # Mark fake volume as shared vol.setShared() # Now we should re-link all hardlinks of this template in all VMs based on it - self.__templateRelink(sdUUID, volParams['imgUUID'], volParams['volUUID']) + self.__templateRelink(destDom, volParams['imgUUID'], volParams['volUUID']) self.log.debug("Succeeded to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID) except Exception: self.log.error("Failure to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID, exc_info=True) - finally: - self._lock.release() def isLegal(self, sdUUID, imgUUID): """ @@ -763,7 +759,7 @@ class Image: if force: # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, dstImgUUID, dstVolUUID) + self.__templateRelink(destDom, dstImgUUID, dstVolUUID) except se.StorageException, e: self.log.error("Unexpected error", exc_info=True) raise

1 0

[NEW PATCH] BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains (via gerrit-bot)
by Igor Lvovsky 10 Jul '11

10 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/646 commit 06bc73dd65b05a9d76938e36c93e7f12bcae0b4a Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:00:21 2011 +0300 BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains Change-Id: I367e83a700e84be0e4cb0ca5b7a02416cb54ac25 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index ccad167..c1e3ec9 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -378,7 +378,11 @@ class Image: # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, sdUUID, imgUUID, volUUID) + # Relink templates only + if not vol.isShared(): + self.log.debug("Doesn't relink regular volume %s of image %s", volUUID, imgUUID) + return + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID)

1 0

[NEW PATCH] Related to BZ#717952 - Use 'with' instead of old lock.acquire (via gerrit-bot)
by Igor Lvovsky 10 Jul '11

10 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/680 commit eaa3d0cf52c9ae410b34c93a1cd75d4ed278e55d Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jul 5 16:03:30 2011 +0300 Related to BZ#717952 - Use 'with' instead of old lock.acquire Change-Id: Ia236b673cd1887b094e5a56cacaf09ee1430ffd4 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..ccad167 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -50,7 +50,7 @@ class Image: Consist from chain of volumes. """ log = logging.getLogger('Storage.Image') - _lock = threading.Lock() + _fakeTemplateLock = threading.Lock() @classmethod def createImageRollback(cls, taskObj, imageDir): @@ -371,11 +371,10 @@ class Image: self.log.error("Unexpected error", exc_info=True) raise se.CouldNotValideTemplateOnTargetDomain("Template %s Destination domain %s: %s" % (pimg, dstSdUUID, str(e))) - def __templateRelink(self, sdUUID, imgUUID, volUUID): + def __templateRelink(self, destDom, imgUUID, volUUID): """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - destDom = SDF.produce(sdUUID) # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) @@ -387,7 +386,7 @@ class Image: # Now we should re-link deleted hardlink, if exists newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(sdUUID, ch['imgUUID']) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) newVol.share(imageDir) else: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) @@ -396,8 +395,7 @@ class Image: """ Create fake template (relevant for Backup domain only) """ - try: - self._lock.acquire() + with self._fakeTemplateLock: try: destDom = SDF.produce(sdUUID) volclass = destDom.getVolumeClass() @@ -417,14 +415,12 @@ class Image: # Mark fake volume as shared vol.setShared() # Now we should re-link all hardlinks of this template in all VMs based on it - self.__templateRelink(sdUUID, volParams['imgUUID'], volParams['volUUID']) + self.__templateRelink(destDom, volParams['imgUUID'], volParams['volUUID']) self.log.debug("Succeeded to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID) except Exception: self.log.error("Failure to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID, exc_info=True) - finally: - self._lock.release() def isLegal(self, sdUUID, imgUUID): """ @@ -763,7 +759,7 @@ class Image: if force: # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, dstImgUUID, dstVolUUID) + self.__templateRelink(destDom, dstImgUUID, dstVolUUID) except se.StorageException, e: self.log.error("Unexpected error", exc_info=True) raise

1 0

[MERGED PATCH] BZ#719255 - Release lock if prepare volume failed (via gerrit-bot)
by Igor Lvovsky 10 Jul '11

10 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/665

1 0

[NEW PATCH] Related to BZ#719516 vdsClient: Make list nicer. (via gerrit-bot)
by David Naori 08 Jul '11

08 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/676 commit 514f444578aebfe3174225cb331e82ae9b3728c2 Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 16:21:15 2011 +0300 Related to BZ#719516 vdsClient: Make list nicer. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 38640cc..2a73785 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,34 +173,31 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True + table = {} + if args: + if args != ['table']: + raise ValueError + for s in self.s.getAllVmStats()['statsList']: + table[ s['vmId'] ] = s + response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: + for conf in response['vmList']: if table: - allStats = {} - for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - if id not in allStats: - continue - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) + vmId = conf['vmId'] + if vmId not in table: + continue + status = conf['status'] + if table[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, table[vmId].get('guestIPs', '') ) + else: + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + sys.exit(response['status']['code']) def do_listNames(self, args):

1 0

[NEW PATCH] BZ#694026 initctl libvirtd as part of its configuration vdsm use to control libvirtd using System V init script. The changes have been made so that vdsm start uses initctl to start libvirtd. If libvirtd is already using initctl, then it will start libvi (via gerrit-bot)
by Sanjay Mehrotra 08 Jul '11

08 Jul '11

New patch submitted by Sanjay Mehrotra (smehrotr(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/675 commit af9ee33d5c26113c4a09af45a865fecc268ece95 Author: Sanjay Mehrotra <smehrotr(a)redhat.com> Date: Thu Jul 7 16:09:39 2011 +0300 BZ#694026 initctl libvirtd as part of its configuration vdsm use to control libvirtd using System V init script. The changes have been made so that vdsm start uses initctl to start libvirtd. If libvirtd is already using initctl, then it will start libvirtd if it is not running. If initctl fails, it will default to sysv initscript. This is patch 1 of the changes. The second patch will change the libvirtd to its default sys v during uninstall of vdsm. Change-Id: I0ea7b64a710b6e2c48fb96da82ebea64fd0482a4 diff --git a/vdsm.spec.in b/vdsm.spec.in index 8f01acc..85a11ed 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -151,6 +151,12 @@ then /usr/sbin/saslpasswd2 -p -a libvirt vdsm@rhevh < "$pfile" fi +# copy the libvirtd upstart job control. +upstart=`rpm -ql libvirt | grep libvirtd.upstart | head --lines=1` +if [ -f "$upstart" ]; then + cp -f $upstart /etc/vdsm/libvirtd.upstart +fi + %preun if [ "$1" -eq 0 ] then @@ -180,6 +186,14 @@ _EOF if selinuxenabled; then setsebool virt_use_nfs off fi + + if initctl list | grep -q libvirtd; then + initctl stop libvirtd + rm -f /etc/init/libvirtd.conf + chkconfig libvirtd on + service libvirtd condrestart + rm -f /etc/vdsm/libvirtd.upstart + fi fi %postun diff --git a/vdsm/vdsmd b/vdsm/vdsmd index 6b35e23..53705b3 100755 --- a/vdsm/vdsmd +++ b/vdsm/vdsmd @@ -30,7 +30,7 @@ RESPAWNPIDFILE=@P_VDSM_RUN@/respawn.pid CORE_DUMP_PATH=/var/log/core/core.%p.%t.dump DOM_METADATA_BACKUP_DIR=/var/log/vdsm/backup CORE_PATTERN=/proc/sys/kernel/core_pattern -NEEDED_SERVICES="iscsid multipathd libvirtd" +NEEDED_SERVICES="iscsid multipathd" CONFLICTING_SERVICES="libvirt-guests" # TODO: Remove cpu cgroup disabling when BZ#623712 is resolved (kernel on # massively multicore platforms failing to scale with cgroups turned on) @@ -323,8 +323,6 @@ configure_libvirt() virt_use_nfs=1 _EOF /usr/sbin/setsebool virt_use_nfs on - - service libvirtd condrestart } RETVAL=0 @@ -343,26 +341,47 @@ reconfigure() { configure_libvirt $args } +start_libvirtd() { + if ! initctl list | grep -q libvirtd + then + chkconfig libvirtd off && service libvirtd stop >/dev/null 2>&1 + cp -f /etc/vdsm/libvirtd.upstart /etc/init/libvirtd.conf + initctl reload-configuration && initctl start libvirtd + fi + + initctl status libvirtd | grep -q "libvirtd start" && return 0; + + if initctl status libvirtd | grep -q "libvirtd stop" + then + initctl start libvirtd + else + service libvirtd condrestart + chkconfig libvirtd on + fi +} + start() { local ret_val python @P_VDSM@/hooks.pyc before_vdsm_start - reconfigure noforce + start_needed_srv ret_val=$? if [ $ret_val -ne 0 ] then - log_failure_msg "$prog: failed to reconfigure libvirt" - return $ret_val + log_failure_msg "$prog: one of the dependent services did not start, error code $ret_val" + return $ret_val fi - start_needed_srv + reconfigure noforce ret_val=$? if [ $ret_val -ne 0 ] then - log_failure_msg "$prog: one of the dependent services did not start, error code $ret_val" - return $ret_val + log_failure_msg "$prog: failed to reconfigure libvirt" + return $ret_val fi + start_libvirtd + shutdown_conflicting_srv @P_VDSM@/vdsm-restore-net-config load_needed_modules

1 0

[NEW PATCH] Related to BZ#719516 vdsClient: Make list nicer. (via gerrit-bot)
by David Naori 08 Jul '11

08 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/676 commit 25b0be006a5529281dd48c8acbdb7d264f057dcd Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 16:21:15 2011 +0300 Related to BZ#719516 vdsClient: Make list nicer. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 38640cc..9a63129 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,34 +173,34 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True + table = {} + if args: + if args != ['table']: + raise ValueError + for s in self.s.getAllVmStats()['statsList']: + table[ s['vmId'] ] = s + response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: + if response['status']['code']: + return response['status']['code'], response['status']['message'] + + for conf in response['vmList']: if table: - allStats = {} - for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - if id not in allStats: - continue - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) + vmId = conf['vmId'] + if vmId not in table: + continue + status = conf['status'] + if table[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, table[vmId].get('guestIPs', '') ) + else: + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + sys.exit(response['status']['code']) def do_listNames(self, args):

1 0

[MERGED PATCH] BZ#719516 vdsClient: avoid possible race in list table. (via gerrit-bot)
by David Naori 08 Jul '11

08 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/671

1 0

[NEW PATCH] BZ#719396 monitorResponse for non-existent vm's (via gerrit-bot)
by Federico Simoncelli 07 Jul '11

07 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/673 commit 4cbf0a0bf8d328ce7949d16e06d682e681437ff8 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Thu Jul 7 11:41:39 2011 +0000 BZ#719396 monitorResponse for non-existent vm's Change-Id: Ic81b8078e7f830b8f811249c8b0756907d5ecf7c diff --git a/vdsm/config.py b/vdsm/config.py index 488aa7b..a75d602 100644 --- a/vdsm/config.py +++ b/vdsm/config.py @@ -56,6 +56,7 @@ config.set('vars', 'vm_command_timeout', '60') # how often should we sample each vm for statistics (seconds) config.set('vars', 'vm_watermark_interval', '2') +config.set('vars', 'vm_monitor_interval', '5') config.set('vars', 'vm_sample_cpu_interval', '15') config.set('vars', 'vm_sample_cpu_window', '2') config.set('vars', 'vm_sample_disk_interval', '60') diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 0516c3b..93d667b 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -37,6 +37,8 @@ class VmStatsThread(utils.AdvancedStatsThread): config.getint('vars', 'vm_watermark_interval')) self.updateVolumes = utils.AdvancedStatsFunction(self._updateVolumes, config.getint('irs', 'vol_size_sample_interval')) + self.updateMonitor = utils.AdvancedStatsFunction(self._updateMonitor, + config.getint('vars', 'vm_monitor_interval')) self.sampleCpu = utils.AdvancedStatsFunction(self._sampleCpu, config.getint('vars', 'vm_sample_cpu_interval'), @@ -77,6 +79,16 @@ class VmStatsThread(utils.AdvancedStatsThread): vmDrive.truesize = int(volSize['truesize']) vmDrive.apparentsize = int(volSize['apparentsize']) + def _updateMonitor(self): + state, details, stateTime = self._vm._dom.controlInfo(0) + stateTime /= 1000.0 + + if (state == libvirt.VIR_DOMAIN_CONTROL_OCCUPIED and + stateTime > config.getint('vars', 'vm_command_timeout')): + self._vm._monitorResponse = -1 + + self._vm._monitorResponse = 0 + def _sampleCpu(self): state, maxMem, memory, nrVirtCpu, cpuTime = self._vm._dom.info() return cpuTime / 1000**3 @@ -933,26 +945,6 @@ class LibvirtVm(vm.Vm): def _monitorDependentInit(self, timeout=None): self.log.warning('unsupported by libvirt vm') - def _getMonitorResponse(self): - if not self._dom: - return 0 - - state, details, stateTime = self._dom.controlInfo(0) - stateTime /= 1000.0 - - if (state == libvirt.VIR_DOMAIN_CONTROL_OCCUPIED and - stateTime > config.getint('vars', 'vm_command_timeout')): - return -1 - - return 0 - - def _setMonitorResponse(self, value): - if value != 0: - self.log.debug("failed to set monitorResponse to %s", - value, exc_info=True) - - _monitorResponse = property(_getMonitorResponse, _setMonitorResponse) - def _waitForIncomingMigrationFinish(self): if 'restoreState' in self.conf: self.cont()

1 0

[NEW PATCH] BZ#699976 Related, use RHEVM leasetime and timeout (via gerrit-bot)
by Federico Simoncelli 07 Jul '11

07 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/677 commit 65f4e23172e8cb2b2a6ca505bdef8006c97b83b6 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Thu Jul 7 14:04:05 2011 +0000 BZ#699976 Related, use RHEVM leasetime and timeout Use the values provided by RHEV-M for LOCK_RENEWAL_INTERVAL, LEASE_TIME, LEASE_RETRIES and IO_OP_TIMEOUT. Change-Id: If326eb1955e771e303f4a39943142b3007187c6c diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 41363ba..4422487 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -192,7 +192,7 @@ def sizeStr2Int(size_str): def intOrDefault(default, val): try: - int(val) + return int(val) except ValueError: return default @@ -388,6 +388,12 @@ class StorageDomain: def acquireClusterLock(self, hostID): self.refresh() + self._clusterLock.setParams( + self.getMetaParam(DMDK_LOCK_RENEWAL_INTERVAL_SEC), + self.getMetaParam(DMDK_LEASE_TIME_SEC), + self.getMetaParam(DMDK_LEASE_RETRIES), + self.getMetaParam(DMDK_IO_OP_TIMEOUT_SEC) + ) self._clusterLock.acquire(hostID) def releaseClusterLock(self):

1 0

[NEW PATCH] BZ#719516 vdsClient: avoid possible race in list table. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/671 commit 07456abc7df872f6f910abd6ed3a95f321541eb3 Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 09:24:20 2011 +0300 BZ#719516 vdsClient: avoid possible race in list table. Change-Id: I7fc52ac0428fd0a2128376a97d56df0fc41ff8ac diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 903dc46..38640cc 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -188,6 +188,8 @@ class service: for conf in response['vmList']: if table: id = conf['vmId'] + if id not in allStats: + continue status = conf['status'] if allStats[id].get('monitorResponse') == '-1': status += '*' @@ -201,7 +203,6 @@ class service: printConf(conf) sys.exit(response['status']['code']) - def do_listNames(self, args): response = self.s.list() if response['status']['code'] != 0:

1 0

[ABANDONED PATCH] Related to BZ#719516 vdsClient: avoid possible race in list table. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

Patch was abandoned by David Naori (dnaori(a)redhat.com) because: Can't push the previous patch again. You can review this change at: http://gerrit.usersys.redhat.com/671

1 0

[NEW PATCH] Related to BZ#719516 vdsClient: Make list nicer. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/676 commit 4310154bc8b305ea7d7561c4b8e83b1a4a0767bf Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 16:21:15 2011 +0300 Related to BZ#719516 vdsClient: Make list nicer. Change-Id: I0753685d3032e79cc3f5de888b8ee7e865954d9c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 38640cc..a2c5d94 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,34 +173,36 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: - if table: - allStats = {} + if response['status']['code']: + return response['status']['code'], response['status']['message'] + + table = {} + + if args: + if args[0] == 'table': for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - if id not in allStats: - continue - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) + table[ s['vmId'] ] = s + else: + raise ValueError + + for conf in response['vmList']: + if table: + vmId = conf['vmId'] + if vmId not in table: + continue + status = conf['status'] + if table[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, table[vmId].get('guestIPs', '') ) + else: + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + sys.exit(response['status']['code']) def do_listNames(self, args):

1 0

[NEW PATCH] [WIP] BZ#694026 initctl libvirtd as part of its configuration (via gerrit-bot)
by Dan Kenigsberg 07 Jul '11

07 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/675 commit e5e01e1876704c5ff8cad77a6a8126a74b49b83c Author: Sanjay Mehrotra <smehrotr(a)redhat.com> Date: Thu Jul 7 16:09:39 2011 +0300 [WIP] BZ#694026 initctl libvirtd as part of its configuration Change-Id: I0ea7b64a710b6e2c48fb96da82ebea64fd0482a4 diff --git a/vdsm/vdsmd b/vdsm/vdsmd index 6b35e23..53705b3 100755 --- a/vdsm/vdsmd +++ b/vdsm/vdsmd @@ -30,7 +30,7 @@ RESPAWNPIDFILE=@P_VDSM_RUN@/respawn.pid CORE_DUMP_PATH=/var/log/core/core.%p.%t.dump DOM_METADATA_BACKUP_DIR=/var/log/vdsm/backup CORE_PATTERN=/proc/sys/kernel/core_pattern -NEEDED_SERVICES="iscsid multipathd libvirtd" +NEEDED_SERVICES="iscsid multipathd" CONFLICTING_SERVICES="libvirt-guests" # TODO: Remove cpu cgroup disabling when BZ#623712 is resolved (kernel on # massively multicore platforms failing to scale with cgroups turned on) @@ -323,8 +323,6 @@ configure_libvirt() virt_use_nfs=1 _EOF /usr/sbin/setsebool virt_use_nfs on - - service libvirtd condrestart } RETVAL=0 @@ -343,26 +341,47 @@ reconfigure() { configure_libvirt $args } +start_libvirtd() { + if ! initctl list | grep -q libvirtd + then + chkconfig libvirtd off && service libvirtd stop >/dev/null 2>&1 + cp -f /etc/vdsm/libvirtd.upstart /etc/init/libvirtd.conf + initctl reload-configuration && initctl start libvirtd + fi + + initctl status libvirtd | grep -q "libvirtd start" && return 0; + + if initctl status libvirtd | grep -q "libvirtd stop" + then + initctl start libvirtd + else + service libvirtd condrestart + chkconfig libvirtd on + fi +} + start() { local ret_val python @P_VDSM@/hooks.pyc before_vdsm_start - reconfigure noforce + start_needed_srv ret_val=$? if [ $ret_val -ne 0 ] then - log_failure_msg "$prog: failed to reconfigure libvirt" - return $ret_val + log_failure_msg "$prog: one of the dependent services did not start, error code $ret_val" + return $ret_val fi - start_needed_srv + reconfigure noforce ret_val=$? if [ $ret_val -ne 0 ] then - log_failure_msg "$prog: one of the dependent services did not start, error code $ret_val" - return $ret_val + log_failure_msg "$prog: failed to reconfigure libvirt" + return $ret_val fi + start_libvirtd + shutdown_conflicting_srv @P_VDSM@/vdsm-restore-net-config load_needed_modules

1 0

[NEW PATCH] [WIP] BZ#694026 initctl libvirtd as part of its configuration (via gerrit-bot)
by Dan Kenigsberg 07 Jul '11

07 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/675 commit 342005164c3145b89bbcf1a5668a569c4aa7b12f Author: Dan Kenigsberg <danken(a)redhat.com> Date: Thu Jul 7 16:09:39 2011 +0300 [WIP] BZ#694026 initctl libvirtd as part of its configuration Change-Id: I0ea7b64a710b6e2c48fb96da82ebea64fd0482a4 diff --git a/vdsm/vdsmd b/vdsm/vdsmd index 6b35e23..53705b3 100755 --- a/vdsm/vdsmd +++ b/vdsm/vdsmd @@ -30,7 +30,7 @@ RESPAWNPIDFILE=@P_VDSM_RUN@/respawn.pid CORE_DUMP_PATH=/var/log/core/core.%p.%t.dump DOM_METADATA_BACKUP_DIR=/var/log/vdsm/backup CORE_PATTERN=/proc/sys/kernel/core_pattern -NEEDED_SERVICES="iscsid multipathd libvirtd" +NEEDED_SERVICES="iscsid multipathd" CONFLICTING_SERVICES="libvirt-guests" # TODO: Remove cpu cgroup disabling when BZ#623712 is resolved (kernel on # massively multicore platforms failing to scale with cgroups turned on) @@ -323,8 +323,6 @@ configure_libvirt() virt_use_nfs=1 _EOF /usr/sbin/setsebool virt_use_nfs on - - service libvirtd condrestart } RETVAL=0 @@ -343,26 +341,47 @@ reconfigure() { configure_libvirt $args } +start_libvirtd() { + if ! initctl list | grep -q libvirtd + then + chkconfig libvirtd off && service libvirtd stop >/dev/null 2>&1 + cp -f /etc/vdsm/libvirtd.upstart /etc/init/libvirtd.conf + initctl reload-configuration && initctl start libvirtd + fi + + initctl status libvirtd | grep -q "libvirtd start" && return 0; + + if initctl status libvirtd | grep -q "libvirtd stop" + then + initctl start libvirtd + else + service libvirtd condrestart + chkconfig libvirtd on + fi +} + start() { local ret_val python @P_VDSM@/hooks.pyc before_vdsm_start - reconfigure noforce + start_needed_srv ret_val=$? if [ $ret_val -ne 0 ] then - log_failure_msg "$prog: failed to reconfigure libvirt" - return $ret_val + log_failure_msg "$prog: one of the dependent services did not start, error code $ret_val" + return $ret_val fi - start_needed_srv + reconfigure noforce ret_val=$? if [ $ret_val -ne 0 ] then - log_failure_msg "$prog: one of the dependent services did not start, error code $ret_val" - return $ret_val + log_failure_msg "$prog: failed to reconfigure libvirt" + return $ret_val fi + start_libvirtd + shutdown_conflicting_srv @P_VDSM@/vdsm-restore-net-config load_needed_modules

1 0

[NEW PATCH] BZ#719255 - Release lock if prepare volume failed (via gerrit-bot)
by Igor Lvovsky 07 Jul '11

07 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/665 commit 03c28923ea2866df84db866c90efd8a0768bdcba Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Wed Jul 6 13:52:45 2011 +0300 BZ#719255 - Release lock if prepare volume failed Change-Id: I57a78ff800f0b8f006ca9c4fed3bb6ac185a55ed diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 7081558..b0d5d21 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1663,12 +1663,17 @@ class HSM: imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) imgResource = rmanager.acquireResource(imageResourcesNamespace, imgUUID, rm.LockType.exclusive) imgResource.autoRelease = False - vol = SDF.produce(sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, volUUID=volUUID) - # NB We want to be sure that at this point HSM does not use stale LVM - # cache info, so we call refresh explicitely. We may want to remove - # this refresh later, when we come up with something better. - vol.refreshVolume() - vol.prepare(rw=rw) + try: + vol = SDF.produce(sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, volUUID=volUUID) + # NB We want to be sure that at this point HSM does not use stale LVM + # cache info, so we call refresh explicitely. We may want to remove + # this refresh later, when we come up with something better. + vol.refreshVolume() + vol.prepare(rw=rw) + except: + imgResource.autoRelease = True + self.log.error("Prepare volume %s in domain %s failed", volUUID, sdUUID, exc_info=True) + raise def public_teardownVolume(self, sdUUID, spUUID, imgUUID, volUUID, rw=False, options = None):

1 0

[NEW PATCH] Fix vgs reload for stale VGs. (via gerrit-bot)
by ewarszaw＠redhat.com 07 Jul '11

07 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/674 commit 705c8ae081f275c5d0f54d117d0addbb524a8403 Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Thu Jul 7 16:04:36 2011 +0300 Fix vgs reload for stale VGs. Stale VG: A VG that is in canche but not in lvm data. Change-Id: Icfdf59db6488a0f4544d4fc7ad77fc927c8fd23b diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index 8a89a01..ef60c02 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -436,7 +436,7 @@ class LVMInfo(object): for staleName in staleVGs: removeVgMapping(staleName) log.warning("Removing stale VG: %s", staleName) - self._lvs.pop((staleName), None) + self._vgs.pop((staleName), None) return updatedVGs

1 0

[NEW PATCH] BZ#719396 monitorResponse for non-existent vm's (via gerrit-bot)
by Federico Simoncelli 07 Jul '11

07 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/673 commit 8cee9f27c039453ec4fad5cb95b246099bdc10ed Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Thu Jul 7 11:41:39 2011 +0000 BZ#719396 monitorResponse for non-existent vm's Change-Id: Ic81b8078e7f830b8f811249c8b0756907d5ecf7c diff --git a/vdsm/libvirtvm.py b/vdsm/libvirtvm.py index 0516c3b..02bebdd 100644 --- a/vdsm/libvirtvm.py +++ b/vdsm/libvirtvm.py @@ -934,10 +934,14 @@ class LibvirtVm(vm.Vm): self.log.warning('unsupported by libvirt vm') def _getMonitorResponse(self): - if not self._dom: + try: + state, details, stateTime = self._dom.controlInfo(0) + except AttributeError: return 0 + except libvirt.libvirtError, e: + if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: + return 0 - state, details, stateTime = self._dom.controlInfo(0) stateTime /= 1000.0 if (state == libvirt.VIR_DOMAIN_CONTROL_OCCUPIED and

1 0

[NEW PATCH] Related to BZ#719516 vdsClient: avoid possible race in list table. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/671 commit adf44e8372158e45481bb1e8015e32edb9e69991 Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 09:24:20 2011 +0300 Related to BZ#719516 vdsClient: avoid possible race in list table. Change-Id: I7fc52ac0428fd0a2128376a97d56df0fc41ff8ac diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 903dc46..a2c5d94 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,34 +173,37 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: - if table: - allStats = {} + if response['status']['code']: + return response['status']['code'], response['status']['message'] + + table = {} + + if args: + if args[0] == 'table': for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) - sys.exit(response['status']['code']) + table[ s['vmId'] ] = s + else: + raise ValueError + for conf in response['vmList']: + if table: + vmId = conf['vmId'] + if vmId not in table: + continue + status = conf['status'] + if table[vmId].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( vmId, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, table[vmId].get('guestIPs', '') ) + else: + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + + sys.exit(response['status']['code']) def do_listNames(self, args): response = self.s.list()

1 0

[NEW PATCH] BZ#719516 vdsClient: avoid possible race in list table. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/671 commit 1e17c8040d35507e796ff576c6f864a536b568ba Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 09:24:20 2011 +0300 BZ#719516 vdsClient: avoid possible race in list table. Change-Id: I7fc52ac0428fd0a2128376a97d56df0fc41ff8ac diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 903dc46..38640cc 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -188,6 +188,8 @@ class service: for conf in response['vmList']: if table: id = conf['vmId'] + if id not in allStats: + continue status = conf['status'] if allStats[id].get('monitorResponse') == '-1': status += '*' @@ -201,7 +203,6 @@ class service: printConf(conf) sys.exit(response['status']['code']) - def do_listNames(self, args): response = self.s.list() if response['status']['code'] != 0:

1 0

[MERGED PATCH] BZ#718993 - Remove vgchange --refresh (via gerrit-bot)
by ewarszaw＠redhat.com 07 Jul '11

07 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/662

1 0

[NEW PATCH] BZ#719516 vdsClient: avoid possible race in list table. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/671 commit 8f4d660ec410cfb9dbf55f3565111663306a6670 Author: David Naori <dnaori(a)redhat.com> Date: Thu Jul 7 09:24:20 2011 +0300 BZ#719516 vdsClient: avoid possible race in list table. Change-Id: I7fc52ac0428fd0a2128376a97d56df0fc41ff8ac diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 903dc46..ddada10 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -173,34 +173,35 @@ class service: return self.ExecAndExit(self.s.changeFloppy(vmId, file)) def do_list(self, args): - table=False - if len(args): - if args[0] == 'table': - table=True response = self.s.list(True) - if response['status']['code'] != 0: - print response['status']['message'] - else: - if table: - allStats = {} - for s in self.s.getAllVmStats()['statsList']: - allStats[s['vmId']] = s - for conf in response['vmList']: - if table: - id = conf['vmId'] - status = conf['status'] - if allStats[id].get('monitorResponse') == '-1': - status += '*' - print "%-36s %6s %-20s %-20s %-20s" % (id, - conf.get('pid', 'none'), - conf.get('vmName', '<< NO NAME >>'), - status, allStats[id].get('guestIPs', '') ) - else: - if 'sysprepInf' in conf: - conf['sysprepInf'] = '<<exists>>' - printConf(conf) - sys.exit(response['status']['code']) + if response['status']['code']: + print response['status']['message'] + return + + allStats = dict() + + if 'table' in args: + for s in self.s.getAllVmStats()['statsList']: + allStats[ s['vmId'] ] = s + + for conf in response['vmList']: + if allStats: + id = conf['vmId'] + if id not in allStats.keys(): + continue + status = conf['status'] + if allStats[id].get('monitorResponse') == '-1': + status += '*' + print "%-36s %6s %-20s %-20s %-20s" % ( id, + conf.get('pid', 'none'), + conf.get('vmName', '<< NO NAME >>'), + status, allStats[id].get('guestIPs', '') ) + else: + if 'sysprepInf' in conf: + conf['sysprepInf'] = '<<exists>>' + printConf(conf) + sys.exit(response['status']['code']) def do_listNames(self, args): response = self.s.list()

1 0

[MERGED PATCH] BZ#719301 clientIF: fix log typo. (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/667

1 0

[MERGED PATCH] BZ#719346 vdsClient: Remove unsupported "stopAllCopies" (via gerrit-bot)
by David Naori 07 Jul '11

07 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/670

1 0

[NEW PATCH] BZ#718993 - Remove vgchange --refresh (via gerrit-bot)
by ewarszaw＠redhat.com 07 Jul '11

07 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/662 commit 8e2a1b9332d66cde99b46de62f7ea5d4efad641d Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Tue Jul 5 18:05:39 2011 +0300 BZ#718993 - Remove vgchange --refresh vgchange --refresh: If any logical volume in the volume group is active, reload its metadata. vgchange --refresh may be too long when are many LVs in the VG. vgchange --refresh was NOT needed for refreshLV after migration in bug 651803, since the LV metadata is reloaded using lvchange --refresh. Change-Id: I123480a2f687bbfacaf8308fbfa3d0a7f5c29c47 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 9f744b2..0c00dd3 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -105,7 +105,7 @@ class VGTagMetadataRW(object): self._vgName = vgName def readlines(self): - lvm.refreshVG(self._vgName) + lvm.invalidateVG(self._vgName) vg = lvm.getVG(self._vgName) metadata = [] for tag in vg.tags: @@ -921,7 +921,7 @@ class BlockStorageDomain(sd.StorageDomain): def refresh(self): self.refreshDirTree() - lvm.refreshVG(self.sdUUID) + lvm.invalidateVG(self.sdUUID) self._metadata = selectMetadata(self.sdUUID) @staticmethod diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index d6159eb..8a89a01 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -881,14 +881,9 @@ def deactivateVG(vgName): _setVgAvailability(vgName, available="n") -def refreshVG(vgName): - #If any logical volume in the volume group is active, reload its metadata. - cmd = ['vgchange', '--refresh', vgName] - rc, out, err = _lvminfo.cmd(cmd) +def invalidateVG(vgName): _lvminfo._invalidatevgs(vgName) _lvminfo._invalidatelvs(vgName) - if rc != 0: - raise se.LogicalVolumeRefreshError("vgchange --refresh %s failed" % (vgName,)) #

1 0

[NEW PATCH] BZ#719346 vdsClient: Remove unsupported "stopAllCopies" (via gerrit-bot)
by David Naori 06 Jul '11

06 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/670 commit 467ed4a594d24309b376bd04a104a5965a67280d Author: David Naori <dnaori(a)redhat.com> Date: Wed Jul 6 19:45:23 2011 +0300 BZ#719346 vdsClient: Remove unsupported "stopAllCopies" Change-Id: I8c60725179eb9d2d75eaa62f796aec8b10be133c diff --git a/vdsm_cli/vdsClient.py b/vdsm_cli/vdsClient.py index 5239187..903dc46 100755 --- a/vdsm_cli/vdsClient.py +++ b/vdsm_cli/vdsClient.py @@ -1238,12 +1238,6 @@ class service: return image['status']['code'], image['status']['message'] return 0, '' - def stopAllCopies(self, args): - stats = self.s.stopAllCopies() - if stats['status']['code']: - return stats['status']['code'], stats['status']['message'] - return 0, '' - def prepareForShutdown(self, args): stats = self.s.prepareForShutdown() if stats['status']['code']: @@ -1866,9 +1860,6 @@ if __name__ == '__main__': ('<TaskID>', 'revert async task' )), - 'stopAllCopies': ( serv.stopAllCopies, - ('','' - )), 'prepareForShutdown': ( serv.prepareForShutdown, ('','' )),

1 0

[NEW PATCH] BZ#719255 - Release lock if prepare volume failed (via gerrit-bot)
by Igor Lvovsky 06 Jul '11

06 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/665 commit 700bc5898fa71ae91b8e7ac2cfdd4804d40cdab4 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Wed Jul 6 13:52:45 2011 +0300 BZ#719255 - Release lock if prepare volume failed Change-Id: I57a78ff800f0b8f006ca9c4fed3bb6ac185a55ed diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 7081558..14c281e 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1663,12 +1663,17 @@ class HSM: imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) imgResource = rmanager.acquireResource(imageResourcesNamespace, imgUUID, rm.LockType.exclusive) imgResource.autoRelease = False - vol = SDF.produce(sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, volUUID=volUUID) - # NB We want to be sure that at this point HSM does not use stale LVM - # cache info, so we call refresh explicitely. We may want to remove - # this refresh later, when we come up with something better. - vol.refreshVolume() - vol.prepare(rw=rw) + try: + vol = SDF.produce(sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, volUUID=volUUID) + # NB We want to be sure that at this point HSM does not use stale LVM + # cache info, so we call refresh explicitely. We may want to remove + # this refresh later, when we come up with something better. + vol.refreshVolume() + vol.prepare(rw=rw) + except Exception: + imgResource.autoRelease = True + self.log.error("Prepare volume %s in domain %s failed", volUUID, sdUUID, exc_info=True) + raise def public_teardownVolume(self, sdUUID, spUUID, imgUUID, volUUID, rw=False, options = None):

1 0

[NEW PATCH] BZ#719301 clientIF: fix log typo. (via gerrit-bot)
by David Naori 06 Jul '11

06 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/667 commit 62c25ccb5a6633e76289e60f926d507ec614ac11 Author: David Naori <dnaori(a)redhat.com> Date: Wed Jul 6 15:09:39 2011 +0300 BZ#719301 clientIF: fix log typo. Change-Id: Iae1a8ba75df6e85b22184fe0fd9e7ccbecbb9b90 diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py index c1219e4..c5169a4 100644 --- a/vdsm/clientIF.py +++ b/vdsm/clientIF.py @@ -433,7 +433,7 @@ class clientIF: Destroy the specified VM. """ self.vmContainerLock.acquire() - self.log.info("vmContainerLock aquired by vm %s", vmId) + self.log.info("vmContainerLock acquired by vm %s", vmId) try: v = self.vmContainer.get(vmId) if not v: @@ -639,7 +639,7 @@ class clientIF: vmParams['displayIp'] = self._getNetworkIp(vmParams.get( 'displayNetwork')) self.vmContainerLock.acquire() - self.log.info("vmContainerLock aquired by vm %s", vmParams['vmId']) + self.log.info("vmContainerLock acquired by vm %s", vmParams['vmId']) try: if 'recover' not in vmParams: if vmParams['vmId'] in self.vmContainer:

1 0

[NEW PATCH] Return 1 socket when "physical id" not present (via gerrit-bot)
by Federico Simoncelli 06 Jul '11

06 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/666 commit 532621aaab3c287a285100e94957db3f7d691331 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Wed Jul 6 11:52:09 2011 +0000 Return 1 socket when "physical id" not present When "physical id" is not present in /proc/cpuinfo return 1 socket. Change-Id: If25e97830d35286b32083f57156ebb97a0e1aeb0 diff --git a/vdsm/caps.py b/vdsm/caps.py index e943321..57a1bbf 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -44,7 +44,8 @@ class CpuInfo(object): return len(self._info) def sockets(self): - return len(set([ p['physical id'] for p in self._info.values() ])) + phys_ids = [ p.get('physical id', '0') for p in self._info.values() ] + return len(set(phys_ids)) def flags(self): return self._info.itervalues().next()['flags'].split()

1 0

[NEW PATCH] BZ#719255 - Release lock if prepare volume failed (via gerrit-bot)
by Igor Lvovsky 06 Jul '11

06 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/665 commit 25c86464dab07fd608ee326f59a253f93037672c Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Wed Jul 6 13:52:45 2011 +0300 BZ#719255 - Release lock if prepare volume failed Change-Id: I57a78ff800f0b8f006ca9c4fed3bb6ac185a55ed diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 7081558..e8194af 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1663,12 +1663,17 @@ class HSM: imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) imgResource = rmanager.acquireResource(imageResourcesNamespace, imgUUID, rm.LockType.exclusive) imgResource.autoRelease = False - vol = SDF.produce(sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, volUUID=volUUID) - # NB We want to be sure that at this point HSM does not use stale LVM - # cache info, so we call refresh explicitely. We may want to remove - # this refresh later, when we come up with something better. - vol.refreshVolume() - vol.prepare(rw=rw) + try: + vol = SDF.produce(sdUUID=sdUUID).produceVolume(imgUUID=imgUUID, volUUID=volUUID) + # NB We want to be sure that at this point HSM does not use stale LVM + # cache info, so we call refresh explicitely. We may want to remove + # this refresh later, when we come up with something better. + vol.refreshVolume() + vol.prepare(rw=rw) + except Exception: + imgResource.autoRelease = True + self.log.error("Prepare volume %s in domain %s failed", volUUID, sdUUID, exc_info=True) + raise def public_teardownVolume(self, sdUUID, spUUID, imgUUID, volUUID, rw=False, options = None): @@ -1691,10 +1696,15 @@ class HSM: self.validatePoolSD(spUUID, sdUUID) vars.task.getSharedLock(STORAGE, sdUUID) - volclass = SDF.produce(sdUUID).getVolumeClass() - volclass.teardown(sdUUID=sdUUID, volUUID=volUUID) - imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) - rmanager.releaseResource(imageResourcesNamespace, imgUUID) + try: + volclass = SDF.produce(sdUUID).getVolumeClass() + volclass.teardown(sdUUID=sdUUID, volUUID=volUUID) + except Exception: + self.log.error("Teardown volume %s in domain %s failed", volUUID, sdUUID, exc_info=True) + raise + finally: + imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) + rmanager.releaseResource(imageResourcesNamespace, imgUUID) def public_getVolumesList(self, sdUUID, spUUID, imgUUID=volume.BLANK_UUID, options = None):

1 0

[NEW PATCH] BZ#718993 - Remove vgchange --refresh (via gerrit-bot)
by ewarszaw＠redhat.com 05 Jul '11

05 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/662 commit d191e2ed17ca6cde26d25e020af32706df064fa5 Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Tue Jul 5 18:05:39 2011 +0300 BZ#718993 - Remove vgchange --refresh vgchange --refresh may be too long when are many LVs in the VG. Change-Id: I123480a2f687bbfacaf8308fbfa3d0a7f5c29c47 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 06e67b3..3e08aff 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -105,7 +105,7 @@ class VGTagMetadataRW(object): self._vgName = vgName def readlines(self): - lvm.refreshVG(self._vgName) + lvm.invalidateVG(self._vgName) vg = lvm.getVG(self._vgName) metadata = [] for tag in vg.tags: @@ -920,7 +920,7 @@ class BlockStorageDomain(sd.StorageDomain): def refresh(self): self.refreshDirTree() - lvm.refreshVG(self.sdUUID) + lvm.invalidateVG(self.sdUUID) self._metadata = selectMetadata(self.sdUUID) @staticmethod diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index d6159eb..8a89a01 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -881,14 +881,9 @@ def deactivateVG(vgName): _setVgAvailability(vgName, available="n") -def refreshVG(vgName): - #If any logical volume in the volume group is active, reload its metadata. - cmd = ['vgchange', '--refresh', vgName] - rc, out, err = _lvminfo.cmd(cmd) +def invalidateVG(vgName): _lvminfo._invalidatevgs(vgName) _lvminfo._invalidatelvs(vgName) - if rc != 0: - raise se.LogicalVolumeRefreshError("vgchange --refresh %s failed" % (vgName,)) #

1 0

[MERGED PATCH] BZ#716675 Remove timezone from kernelDict (via gerrit-bot)
by David Naori 05 Jul '11

05 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/660

1 0

[NEW PATCH] BZ#699976 Related, use RHEVM leasetime and timeout (via gerrit-bot)
by Federico Simoncelli 05 Jul '11

05 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/652 commit c51c0cb0f616fa4f2de10f9a28110a23ce4dd9b7 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Mon Jul 4 14:21:56 2011 +0000 BZ#699976 Related, use RHEVM leasetime and timeout Increase DMDK_IO_OP_TIMEOUT_SEC to 10 seconds and DMDK_LEASE_TIME_SEC to 60 seconds (RHEVM default values) to avoid spm self-fences when working on slow storages. Change-Id: I1d641290521379febca24331d1531f658996956b diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 41363ba..f40236c 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -108,9 +108,9 @@ DMDK_LEASE_RETRIES = 'LEASERETRIES' DEFAULT_LEASE_PARAMS = {DMDK_LOCK_POLICY : "ON", DMDK_LEASE_RETRIES : 3, - DMDK_LEASE_TIME_SEC : 30, + DMDK_LEASE_TIME_SEC : 60, DMDK_LOCK_RENEWAL_INTERVAL_SEC : 5, - DMDK_IO_OP_TIMEOUT_SEC : 1} + DMDK_IO_OP_TIMEOUT_SEC : 10} MASTER_FS_DIR = 'master' VMS_DIR = 'vms'

1 0

[NEW PATCH] BZ#716675 Remove timezone from kernelDict (via gerrit-bot)
by David Naori 05 Jul '11

05 Jul '11

New patch submitted by David Naori (dnaori(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/660 commit 8e3022ff133233d27af10b7bb5d0f3316b90ca9e Author: David Naori <dnaori(a)redhat.com> Date: Tue Jul 5 09:26:13 2011 +0300 BZ#716675 Remove timezone from kernelDict Change-Id: I472261ec7ff75d5d785f71dfc908442d7eb88d90 diff --git a/vdsm/caps.py b/vdsm/caps.py index 37af2f5..e943321 100644 --- a/vdsm/caps.py +++ b/vdsm/caps.py @@ -200,8 +200,9 @@ def _getKeyPackages(): logging.error('kernel release not found', exc_info=True) ver, rel = '0', '0' try: - t = file('/proc/sys/kernel/version').read().strip().split(None, 2)[2] - t = time.mktime(time.strptime(t, '%a %b %d %H:%M:%S %Z %Y')) + t = file('/proc/sys/kernel/version').read().split()[2:] + del t[4] # Delete timezone + t = time.mktime(time.strptime(' '.join(t))) except: logging.error('kernel build time not found', exc_info=True) t = '0'

1 0

[NEW PATCH] formatStorageDomain removes the VG too. (via gerrit-bot)
by ewarszaw＠redhat.com 05 Jul '11

05 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/659 commit 59a64954b5dcab5e37d762452bc0eb18722268dc Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Tue Jul 5 15:32:53 2011 +0300 formatStorageDomain removes the VG too. Change-Id: Ie993abd2a812f1ecec127cc637794994591e48ea diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 9f744b2..06e67b3 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -679,8 +679,7 @@ class BlockStorageDomain(sd.StorageDomain): except se.CannotRemoveLogicalVolume, e: cls.log.warning("Remove logical volume failed %s/%s %s", sdUUID, lv.name, str(e)) - # Remove SD tag - lvm.replaceVGTag(sdUUID, STORAGE_DOMAIN_TAG, STORAGE_UNREADY_DOMAIN_TAG) + lvm.removeVG(sdUUID) return True def getInfo(self): diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 7081558..11c32dc 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -828,6 +828,7 @@ class HSM: return dict(uuid=lvm.getVG(vgname).uuid) + #DEPRECATED: formatSD effectively removes the VG. def public_removeVG(self, vgUUID, options = None): """ Removes a volume group. @@ -835,12 +836,13 @@ class HSM: :param vgUUID: The UUID of the VG you want removed. :type vgUUID: UUID :param options: ? - - :raises: :exc:`storage_exception.VolumeGroupDoesNotExist` if no VG with this UUID exists. """ vars.task.setDefaultException(se.VolumeGroupActionError("%s" % str(vgUUID))) #getSharedLock(connectionsResource...) - lvm.removeVGbyUUID(vgUUID) + try: + lvm.removeVGbyUUID(vgUUID) + except se.VolumeGroupDoesNotExist: + pass def public_getTaskStatus(self, taskID, spUUID=None, options = None):

1 0

[NEW PATCH] _initpv should only throw PhysDevInitializationError exceptions, however when a bad device argument is passed to _initpv (say /dev/null) it throws OSError. This patch is fixing this scenario (via gerrit-bot)
by Yotam Oron 05 Jul '11

05 Jul '11

New patch submitted by Yotam Oron (yoron(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/653 commit 1d3cb8cbf8a43ec7d110980227b0a9d24deb3c1d Author: Yotam Oron <yoron(a)redhat.com> Date: Tue Jul 5 12:14:53 2011 +0300 _initpv should only throw PhysDevInitializationError exceptions, however when a bad device argument is passed to _initpv (say /dev/null) it throws OSError. This patch is fixing this scenario Change-Id: I05e55b361df1523d970428d5453f36e98b2c9a2b diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index d6159eb..b3d7221 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -676,7 +676,13 @@ def _initpv(device, withmetadata=False): if isPvPartOfVg(device): raise se.PhysDevInitializationError(device) - devicemapper.removeMappingsHoldingDevice(os.path.basename(device)) + try: + devicemapper.removeMappingsHoldingDevice(os.path.basename(device)) + except OSError: + log.warning("removeMappingsHoldingDevice() failed with %s" % device, + exc_info=True) + raise se.PhysDevInitializationError(device) + rc, out, err = _lvminfo.cmd(cmd) if rc != 0: raise se.PhysDevInitializationError(device)

1 0

[NEW PATCH] BZ#699976 Related, set IO_OP_TIMEOUT to 10 seconds (via gerrit-bot)
by Federico Simoncelli 05 Jul '11

05 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/652 commit 98f130e72800bd20c8db2ab24687f9bee156f5f9 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Mon Jul 4 14:21:56 2011 +0000 BZ#699976 Related, set IO_OP_TIMEOUT to 10 seconds Increase DMDK_IO_OP_TIMEOUT_SEC to 10 seconds (RHEVM default) to avoid spm self-fences when working on slow storages. Change-Id: I1d641290521379febca24331d1531f658996956b diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 325da5d..e9b6932 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -108,9 +108,9 @@ DMDK_LEASE_RETRIES = 'LEASERETRIES' DEFAULT_LEASE_PARAMS = {DMDK_LOCK_POLICY : "ON", DMDK_LEASE_RETRIES : 3, - DMDK_LEASE_TIME_SEC : 30, + DMDK_LEASE_TIME_SEC : 60, DMDK_LOCK_RENEWAL_INTERVAL_SEC : 5, - DMDK_IO_OP_TIMEOUT_SEC : 1} + DMDK_IO_OP_TIMEOUT_SEC : 10} MASTER_FS_DIR = 'master' VMS_DIR = 'vms'

1 0

[NEW PATCH] _initpv should only throw PhysDevInitializationError exceptions, however when a bad device is received (say /dev/null) it throws OSError. This patch is fixing this scenario (via gerrit-bot)
by Yotam Oron 05 Jul '11

05 Jul '11

New patch submitted by Yotam Oron (yoron(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/653 commit 46c8d0077575cdcc7353b802b9bd0c8f68136c6e Author: Yotam Oron <yoron(a)redhat.com> Date: Tue Jul 5 12:14:53 2011 +0300 _initpv should only throw PhysDevInitializationError exceptions, however when a bad device is received (say /dev/null) it throws OSError. This patch is fixing this scenario Change-Id: I05e55b361df1523d970428d5453f36e98b2c9a2b diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index d6159eb..f87bf6b 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -676,7 +676,11 @@ def _initpv(device, withmetadata=False): if isPvPartOfVg(device): raise se.PhysDevInitializationError(device) - devicemapper.removeMappingsHoldingDevice(os.path.basename(device)) + try: + devicemapper.removeMappingsHoldingDevice(os.path.basename(device)) + except OSError: + raise se.PhysDevInitializationError(device) + rc, out, err = _lvminfo.cmd(cmd) if rc != 0: raise se.PhysDevInitializationError(device)

1 0

[NEW PATCH] BZ#717658 - Do lvm cache refresh in SDF.refreshStorage (via gerrit-bot)
by ewarszaw＠redhat.com 04 Jul '11

04 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/651 commit 0a44575a18343fd2a41c2aff8cb0076191dc25d3 Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Mon Jul 4 13:44:55 2011 +0300 BZ#717658 - Do lvm cache refresh in SDF.refreshStorage Change-Id: Ib51dc550def7713eaa23def41a7133bcff2ee46b diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 0e7631b..7081558 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1327,6 +1327,7 @@ class HSM: :rtype: dict """ vars.task.setDefaultException(se.StorageDomainActionError("spUUID: %s" % str(spUUID))) + SDF.refreshStorage() if spUUID and spUUID != volume.BLANK_UUID: domList = self.getPool(spUUID).getDomains() domains = domList.keys() diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index 93529fd..245c5bf 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -44,6 +44,7 @@ class StorageDomainCache: def refreshStorage(self): multipath.rescan() lvm.updateLvmConf() + lvm.getAllVGs() self.storageStale = False def _getDomainFromCache(self, sdUUID):

1 0

[NEW PATCH] BZ#699976 Related, set IO_OP_TIMEOUT to 5 seconds (via gerrit-bot)
by Federico Simoncelli 04 Jul '11

04 Jul '11

New patch submitted by Federico Simoncelli (fsimonce(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/652 commit 984037b465c1171c21713232bbb7c5f98cc4566e Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Mon Jul 4 14:21:56 2011 +0000 BZ#699976 Related, set IO_OP_TIMEOUT to 5 seconds Increase DMDK_IO_OP_TIMEOUT_SEC to 5 seconds to avoid spm self-fences when working on slow storages. Change-Id: I1d641290521379febca24331d1531f658996956b diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 325da5d..cd0caf1 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -110,7 +110,7 @@ DEFAULT_LEASE_PARAMS = {DMDK_LOCK_POLICY : "ON", DMDK_LEASE_RETRIES : 3, DMDK_LEASE_TIME_SEC : 30, DMDK_LOCK_RENEWAL_INTERVAL_SEC : 5, - DMDK_IO_OP_TIMEOUT_SEC : 1} + DMDK_IO_OP_TIMEOUT_SEC : 5} MASTER_FS_DIR = 'master' VMS_DIR = 'vms'

1 0

[NEW PATCH] BZ#717658 - Remove unnecessary SDF.refreshStorage(). (via gerrit-bot)
by ewarszaw＠redhat.com 04 Jul '11

04 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/636 commit d6c647833d6c6d44cff5ab9411b8caad624c0bf9 Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Wed Jun 29 18:52:31 2011 +0300 BZ#717658 - Remove unnecessary SDF.refreshStorage(). Change-Id: I7de05de83f17a14895c150c331e971f35d7ae547 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 2aaf822..0e7631b 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -756,15 +756,6 @@ class HSM: """ vars.task.setDefaultException(se.BlockDeviceActionError("GUID: %s" % str(guid))) #getSharedLock(connectionsResource...) - rescan = True - try: - if options: - rescan = options.get("rescan", True) - except: - self.log.warning("ignore options %s" % (str(options))) - if rescan: - SDF.refreshStorage() - try: devInfo = self._getDeviceList(guids=[guid])[0] for p in devInfo["pathstatus"]:

1 0

[NEW PATCH] BZ#717658 - Remove StorageDomainCache.invalidate(). (via gerrit-bot)
by ewarszaw＠redhat.com 04 Jul '11

04 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/639 commit d11b018e257c612685dd518389e9974a64672cdb Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Thu Jun 30 06:09:56 2011 +0300 BZ#717658 - Remove StorageDomainCache.invalidate(). Change-Id: If6ee4fb5bd115d258c3027fe5e810f21e138985e diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index a3ea775..93529fd 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -45,13 +45,6 @@ class StorageDomainCache: multipath.rescan() lvm.updateLvmConf() self.storageStale = False - self.invalidate() - - def invalidate(self): - """ - """ - # TODO : Remove all calls to this - pass def _getDomainFromCache(self, sdUUID): try:

1 0

[NEW PATCH] BZ#717658 - Do lvm cache refresh in SDF.refreshStorage (via gerrit-bot)
by ewarszaw＠redhat.com 04 Jul '11

04 Jul '11

New patch submitted by Eduardo Warszawski (ewarszaw(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/651 commit e99595f1982af1bb505df640df1c698e6103430e Author: Eduardo Warszawski <ewarszaw(a)redhat.com> Date: Mon Jul 4 13:44:55 2011 +0300 BZ#717658 - Do lvm cache refresh in SDF.refreshStorage Change-Id: Ib51dc550def7713eaa23def41a7133bcff2ee46b diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 0e7631b..a864a06 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1326,6 +1326,7 @@ class HSM: :returns: a dict containing list of storage domains. :rtype: dict """ + SDF.refreshStorage() vars.task.setDefaultException(se.StorageDomainActionError("spUUID: %s" % str(spUUID))) if spUUID and spUUID != volume.BLANK_UUID: domList = self.getPool(spUUID).getDomains() diff --git a/vdsm/storage/sdc.py b/vdsm/storage/sdc.py index 93529fd..245c5bf 100644 --- a/vdsm/storage/sdc.py +++ b/vdsm/storage/sdc.py @@ -44,6 +44,7 @@ class StorageDomainCache: def refreshStorage(self): multipath.rescan() lvm.updateLvmConf() + lvm.getAllVGs() self.storageStale = False def _getDomainFromCache(self, sdUUID):

1 0

[MERGED PATCH] BZ#718488 deployUtil: run on python2.4 (via gerrit-bot)
by Dan Kenigsberg 03 Jul '11

03 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/647

1 0

[NEW PATCH] BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains (via gerrit-bot)
by Igor Lvovsky 03 Jul '11

03 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/646 commit 98ae8a04f3c055d648527e885a683ee79bd6d089 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Thu Jun 30 18:01:27 2011 +0300 BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains Change-Id: I367e83a700e84be0e4cb0ca5b7a02416cb54ac25 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 714f725..1e39052 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -55,7 +55,7 @@ class Image: Consist from chain of volumes. """ log = logging.getLogger('Storage.Image') - _lock = threading.Lock() + _fakeTemplateLock = threading.Lock() @classmethod def createImageRollback(cls, taskObj, imageDir): @@ -384,15 +384,18 @@ class Image: self.log.error("Unexpected error", exc_info=True) raise se.CouldNotValideTemplateOnTargetDomain("Template %s Destination domain %s: %s" % (pimg, dstSdUUID, str(e))) - def __templateRelink(self, sdUUID, imgUUID, volUUID): + def __templateRelink(self, destDom, imgUUID, volUUID): """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - destDom = SDF.produce(sdUUID) # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, sdUUID, imgUUID, volUUID) + # Relink templates only + if not vol.isShared(): + self.log.debug("Doesn't relink regular volume %s of image %s", volUUID, imgUUID) + return + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) @@ -400,7 +403,7 @@ class Image: # Now we should re-link deleted hardlink, if exists newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(sdUUID, ch['imgUUID']) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) newVol.share(imageDir) else: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) @@ -409,8 +412,7 @@ class Image: """ Create fake template (relevant for Backup domain only) """ - try: - self._lock.acquire() + with self._fakeTemplateLock: try: destDom = SDF.produce(sdUUID) volclass = destDom.getVolumeClass() @@ -430,14 +432,12 @@ class Image: # Mark fake volume as shared vol.setShared() # Now we should re-link all hardlinks of this template in all VMs based on it - self.__templateRelink(sdUUID, volParams['imgUUID'], volParams['volUUID']) + self.__templateRelink(destDom, volParams['imgUUID'], volParams['volUUID']) self.log.debug("Succeeded to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID) except Exception: self.log.error("Failure to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID, exc_info=True) - finally: - self._lock.release() def isLegal(self, sdUUID, imgUUID): """ @@ -806,7 +806,7 @@ class Image: if force: # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, dstImgUUID, dstVolUUID) + self.__templateRelink(destDom, dstImgUUID, dstVolUUID) except se.StorageException, e: self.log.error("Unexpected error", exc_info=True) raise

1 0

[NEW PATCH] BZ#712106 - Add lock for template during moveImage operation (via gerrit-bot)
by Igor Lvovsky 03 Jul '11

03 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/630 commit d1f09f4060092b3596328ad3346fe3d449720a10 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jun 21 13:52:20 2011 +0300 BZ#712106 - Add lock for template during moveImage operation Change-Id: Ib87649cf0aebd5a35d56faa7e8f165d10d715331 diff --git a/vdsm/constants.py b/vdsm/constants.py index ae9a79b..3a800a4 100644 --- a/vdsm/constants.py +++ b/vdsm/constants.py @@ -139,6 +139,11 @@ CMD_LOWPRIO = [EXT_NICE, '-n', '19', EXT_IONICE, '-c', '2', '-n', '7'] # # Storage constants # + +LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' +IMAGE_NAMESPACE = 'imageNS' +VOLUME_NAMESPACE = 'volumeNS' + STRG_ISCSI_HOST = "iscsi_host/" STRG_SCSI_HOST = "scsi_host/" STRG_ISCSI_SESSION = "iscsi_session/" diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 214e8a4..01b2182 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -27,7 +27,7 @@ import safelease import blockVolume import multipath import resourceFactories -from resourceFactories import LVM_ACTIVATION_NAMESPACE +from constants import LVM_ACTIVATION_NAMESPACE from persistentDict import PersistentDict, DictValidator import iscsi import storage_exception as se diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index 3da76fa..cac25e8 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -24,7 +24,7 @@ import lvm import resourceManager as rm from threadLocal import vars from sdf import StorageDomainFactory as SDF -from resourceFactories import LVM_ACTIVATION_NAMESPACE +from constants import LVM_ACTIVATION_NAMESPACE import fileUtils TAG_PREFIX_MD = "MD_" diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 2aaf822..c00cb70 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -38,7 +38,7 @@ from threadLocal import vars import constants from storageConstants import STORAGE from task import Job -from resourceFactories import IMAGE_NAMESPACE +from constants import IMAGE_NAMESPACE import resourceManager as rm import devicemapper diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..714f725 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -22,6 +22,11 @@ from config import config import storage_exception as se import task from threadLocal import vars +from constants import IMAGE_NAMESPACE +import resourceManager as rm +from contextlib import contextmanager + +rmanager = rm.ResourceManager.getInstance() # Disk type UNKNOWN_DISK_TYPE = 0 @@ -137,7 +142,14 @@ class Image: vars.task.pushRecovery(task.Recovery(name, "image", "Image", "deleteRecover", [self.repoPath, sdUUID, imgUUID, str(postZero), str(force)])) - self._delete(sdUUID, imgUUID, postZero, force) + try: + self._delete(sdUUID, imgUUID, postZero, force) + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.ImageDeleteError("%s: %s" % (imgUUID), str(e)) @classmethod def deleteRecover(cls, taskObj, repoPath, sdUUID, imgUUID, postZero, force): @@ -315,7 +327,8 @@ class Image: srcVol = None if not srcVol: - raise se.ImageIsNotLegalChain("Image %s is not a leaf" % imgUUID) + self.log.error("There is no leaf in the image %s" % imgUUID) + raise se.ImageIsNotLegalChain(imgUUID) # Build up the sorted (parent->child) chain while not srcVol.isShared(): @@ -450,48 +463,29 @@ class Image: """ try: if srcVol: - srcVol.teardown(sdUUID=srcVol.sdUUID, volUUID=srcVol.volUUID, justme=True) + srcVol.teardown(sdUUID=srcVol.sdUUID, volUUID=srcVol.volUUID) if dstVol: - dstVol.teardown(sdUUID=dstVol.sdUUID, volUUID=dstVol.volUUID, justme=True) + dstVol.teardown(sdUUID=dstVol.sdUUID, volUUID=dstVol.volUUID) except Exception: self.log.error("Unexpected error", exc_info=True) - def move(self, srcSdUUID, dstSdUUID, imgUUID, vmUUID, op, postZero, force): - """ - Move/Copy image between storage domains within same storage pool - """ - self.log.info("srcSdUUID=%s dstSdUUID=%s "\ - "imgUUID=%s vmUUID=%s op=%s force=%s postZero=%s", - srcSdUUID, dstSdUUID, imgUUID, vmUUID, OP_TYPES[op], str(force), str(postZero)) - - srcVol = leafVol = dstVol = None + def _createTargetImage(self, destDom, srcSdUUID, imgUUID): + # Before actual data copying we need perform several operation + # such as: create all volumes, create fake template if needed, ... try: # Find all volumes of source image - chain = self.getChain(srcSdUUID, imgUUID) - leafVol = chain[-1] - except se.StorageException, e: + srcChain = self.getChain(srcSdUUID, imgUUID) + except se.StorageException: self.log.error("Unexpected error", exc_info=True) raise except Exception, e: self.log.error("Unexpected error", exc_info=True) raise se.SourceImageActionError(imgUUID, srcSdUUID, str(e)) - destDom = SDF.produce(dstSdUUID) + fakeTemplate = False pimg = volume.BLANK_UUID # standalone chain - # If image already exists check whether it illegal/fake, overwrite it - if not self.isLegal(dstSdUUID, imgUUID): - force = True - # We must first remove the previous instance of image (if exists) - # in destination domain, if we got the overwrite command - if force: - try: - self.log.info("delete image %s on domain %s before overwriting", imgUUID, dstSdUUID) - self.delete(dstSdUUID, imgUUID, postZero, force=True) - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise e # check if the chain is build above a template, or it is a standalone - pvol = chain[0].getParentVolume() + pvol = srcChain[0].getParentVolume() if pvol: # find out parent volume parameters volParams = pvol.getVolumeParams() @@ -500,81 +494,134 @@ class Image: # FIXME: This workaround help as copy VM to the backup domain without its template # We will create fake template for future VM creation and mark it as FAKE volume # This situation is relevant for backup domain only - self.createFakeTemplate(dstSdUUID, volParams) + fakeTemplate = True + + @contextmanager + def justLogIt(img): + self.log.debug("You don't really need lock parent of image %s", img) + yield + + dstImageResourcesNamespace = sd.getNamespace(destDom.sdUUID, IMAGE_NAMESPACE) + # In destination domain we need to lock image's template if exists + with rmanager.acquireResource(dstImageResourcesNamespace, pimg, rm.LockType.shared) \ + if pimg != volume.BLANK_UUID else justLogIt(imgUUID): + if fakeTemplate: + self.createFakeTemplate(destDom.sdUUID, volParams) + + dstChain = [] + for srcVol in srcChain: + # Create the dst volume + try: + # find out src volume parameters + volParams = srcVol.getVolumeParams(bs=1) + + # To avoid 'prezeroing' preallocated volume on NFS domain, + # we create the target volume with minimal size and after that w'll change + # its metadata back to the original size. + tmpSize = 20480 # in sectors (10M) + destDom.createVolume(imgUUID=imgUUID, size=tmpSize, + volFormat=volParams['volFormat'], preallocate=volParams['prealloc'], + diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], + srcImgUUID=pimg, srcVolUUID=volParams['parent']) + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) + # Extend volume (for LV only) size to the actual size + dstVol.extend((volParams['apparentsize'] + 511) / 512) + # Change destination volume metadata back to the original size. + dstVol.setSize(volParams['size']) + dstChain.append(dstVol) + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.DestImageActionError(imgUUID, destDom.sdUUID, str(e)) - for srcVol in chain: - # Create the dst volume - try: - # find out src volume parameters - volParams = srcVol.getVolumeParams(bs=1) + # only base may have a different parent image + pimg = imgUUID - # To avoid 'prezeroing' preallocated volume on NFS domain, - # we create the target volume with minimal size and after that w'll change - # its metadata back to the original size. - tmpSize = 20480 # in sectors (10M) - destDom.createVolume(imgUUID=imgUUID, size=tmpSize, - volFormat=volParams['volFormat'], preallocate=volParams['prealloc'], - diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], - srcImgUUID=pimg, srcVolUUID=volParams['parent']) - # Extend volume (for LV only) size to the actual size - dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) - dstVol.extend((volParams['apparentsize'] + 511) / 512) - # Change destination volume metadata back to the original size. - dstVol.setSize(volParams['size']) - except se.StorageException, e: - self.log.error("Unexpected error", exc_info=True) - raise - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise se.DestImageActionError(imgUUID, dstSdUUID, str(e)) + return {'srcChain':srcChain, 'dstChain':dstChain} - # Do the actual copy - try: + def _interImagesCopy(self, destDom, srcSdUUID, imgUUID, chains): + srcLeafVol = chains['srcChain'][-1] + dstLeafVol = chains['dstChain'][-1] + try: + # Prepare the whole chains before the copy + srcLeafVol.prepare(rw=False) + dstLeafVol.prepare(rw=True, chainrw=True, setrw=True) + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + # teardown volumes + self.__cleanupMove(srcLeafVol, dstLeafVol) + raise + + try: + for srcVol in chains['srcChain']: + # Do the actual copy try: + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) srcSize = srcVol.getVolumeSize(bs=1) - # Prepare relevant volumes before copy and teardown after that - srcVol.prepare(rw=False, justme=True) - dstVol.prepare(rw=True, justme=True, setrw=True) misc.ddWatchCopy(srcVol.getVolumePath(), dstVol.getVolumePath(), vars.task.aborting, self.idle, size=srcSize) - # In case of copying template, we should set the destination volume - # as SHARED (after copy because otherwise prepare as RW would fail) - if srcVol.isShared(): - dstVol.setShared() - elif srcVol.isInternal(): - dstVol.setInternal() - except se.ActionStopped, e: - raise e - except se.StorageException, e: + except se.ActionStopped: + raise + except se.StorageException: self.log.error("Unexpected error", exc_info=True) raise except Exception, e: self.log.error("Unexpected error", exc_info=True) - raise se.CopyImageError("image=%s, src domain=%s, dst domain=%s: msg %s" % (imgUUID, srcSdUUID, dstSdUUID, str(e))) - finally: - # teardown volumes - self.__cleanupMove(srcVol, dstVol) - srcVol = dstVol = None + raise se.CopyImageError("image=%s, src domain=%s, dst domain=%s: msg %s" % (imgUUID, srcSdUUID, destDom.sdUUID, str(e))) + finally: + # teardown volumes + self.__cleanupMove(srcLeafVol, dstLeafVol) + + def _finalizeDestinationImage(self, destDom, imgUUID, chains, force): + for srcVol in chains['srcChain']: + try: + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) + # In case of copying template, we should set the destination volume + # as SHARED (after copy because otherwise prepare as RW would fail) + if srcVol.isShared(): + dstVol.setShared() + elif srcVol.isInternal(): + dstVol.setInternal() + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.DestImageActionError(imgUUID, destDom.sdUUID, str(e)) + + def move(self, srcSdUUID, dstSdUUID, imgUUID, vmUUID, op, postZero, force): + """ + Move/Copy image between storage domains within same storage pool + """ + self.log.info("srcSdUUID=%s dstSdUUID=%s "\ + "imgUUID=%s vmUUID=%s op=%s force=%s postZero=%s", + srcSdUUID, dstSdUUID, imgUUID, vmUUID, OP_TYPES[op], str(force), str(postZero)) - # only base may have a different parent image - pimg = imgUUID + destDom = SDF.produce(dstSdUUID) + # If image already exists check whether it illegal/fake, overwrite it + if not self.isLegal(destDom.sdUUID, imgUUID): + force = True + # We must first remove the previous instance of image (if exists) + # in destination domain, if we got the overwrite command + if force: + self.log.info("delete image %s on domain %s before overwriting", imgUUID, destDom.sdUUID) + self.delete(destDom.sdUUID, imgUUID, postZero, force=True) + chains = self._createTargetImage(destDom, srcSdUUID, imgUUID) + self._interImagesCopy(destDom, srcSdUUID, imgUUID, chains) + self._finalizeDestinationImage(destDom, imgUUID, chains, force) if force: + leafVol = chains['dstChain'][-1] # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, imgUUID, leafVol.volUUID) + self.__templateRelink(destDom, imgUUID, leafVol.volUUID) # At this point we successfully finished the 'copy' part of the operation # and we can clear all recoveries. vars.task.clearRecoveries() # If it's 'move' operation, we should delete src image after copying if op == MOVE_OP: - try: - self.delete(srcSdUUID, imgUUID, postZero, force=True) - except se.StorageException, e: - self.log.error("Unexpected error", exc_info=True) - raise - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise se.ImageDeleteError("%s: %s" % (imgUUID), str(e)) + self.delete(srcSdUUID, imgUUID, postZero, force=True) self.log.info("%s task on image %s was successfully finished", OP_TYPES[op], imgUUID) return True @@ -587,7 +634,7 @@ class Image: try: self.delete(sdUUID, imgUUID, postZero, force=True) except Exception: - self.log.error("Unexpected error", exc_info=True) + pass def multiMove(self, srcSdUUID, dstSdUUID, imgDict, vmUUID, force): """ @@ -616,8 +663,8 @@ class Image: for (imgUUID, postZero) in imgDict.iteritems(): try: self.delete(srcSdUUID, imgUUID, postZero, force=True) - except Exception, e: - self.log.error("Unexpected error", exc_info=True) + except Exception: + pass def __cleanupCopy(self, srcVol, dstVol): """ @@ -708,12 +755,8 @@ class Image: # We must first remove the previous instance of image (if exists) # in destination domain, if we got the overwrite command if force: - try: - self.log.info("delete image %s on domain %s before overwriting", dstImgUUID, dstSdUUID) - self.delete(dstSdUUID, dstImgUUID, postZero, force=True) - except Exception, e: - self.log.error(e, exc_info=True) - raise se.OverwriteImageError(dstImgUUID, dstSdUUID) + self.log.info("delete image %s on domain %s before overwriting", dstImgUUID, dstSdUUID) + self.delete(dstSdUUID, dstImgUUID, postZero, force=True) # To avoid 'prezeroing' preallocated volume on NFS domain, # we create the target volume with minimal size and after that w'll change diff --git a/vdsm/storage/resourceFactories.py b/vdsm/storage/resourceFactories.py index ce3ec70..4836217 100644 --- a/vdsm/storage/resourceFactories.py +++ b/vdsm/storage/resourceFactories.py @@ -16,10 +16,7 @@ import storage_exception as se from sdf import StorageDomainFactory as SDF import sd import image - -LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' -IMAGE_NAMESPACE = 'imageNS' -VOLUME_NAMESPACE = 'volumeNS' +from constants import VOLUME_NAMESPACE rmanager = rm.ResourceManager.getInstance() diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 325da5d..4cdaef9 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -17,7 +17,7 @@ from glob import glob import storage_exception as se import misc import resourceFactories -from resourceFactories import IMAGE_NAMESPACE, VOLUME_NAMESPACE +from constants import IMAGE_NAMESPACE, VOLUME_NAMESPACE import resourceManager as rm import constants import safelease diff --git a/vdsm/storage/spm.py b/vdsm/storage/spm.py index b186d39..6ed480b 100644 --- a/vdsm/storage/spm.py +++ b/vdsm/storage/spm.py @@ -36,7 +36,7 @@ import logging import storage_exception as se from threadLocal import vars from storageConstants import STORAGE -from resourceFactories import IMAGE_NAMESPACE +from constants import IMAGE_NAMESPACE import resourceManager as rm from contextlib import nested import fileUtils diff --git a/vdsm/storage/storage_exception.py b/vdsm/storage/storage_exception.py index b1cbcdf..c9df109 100644 --- a/vdsm/storage/storage_exception.py +++ b/vdsm/storage/storage_exception.py @@ -274,7 +274,7 @@ class ImageValidationError(StorageException): class ImageDeleteError(StorageException): code = 256 - message = "Could not remove all files" + message = "Could not remove all image's volumes" #REMOVED in 2.3. #This class was not in use in 2.2 nor in 2.3. diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py index 123ed37..0052e84 100644 --- a/vdsm/storage/volume.py +++ b/vdsm/storage/volume.py @@ -21,8 +21,9 @@ import misc import fileUtils import task from threadLocal import vars -import resourceFactories import resourceManager as rm +from constants import IMAGE_NAMESPACE + rmanager = rm.ResourceManager.getInstance() @@ -150,7 +151,7 @@ class Volume: cls.log.info("sdUUID=%s srcImg=%s srcVol=%s dstFormat=%s srcParent=%s", sdUUID, srcImg, srcVol, dstFormat, srcParent) - imageResourcesNamespace = sd.getNamespace(sdUUID, resourceFactories.IMAGE_NAMESPACE) + imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) with rmanager.acquireResource(imageResourcesNamespace, srcImg, rm.LockType.exclusive): try: vol = SDF.produce(sdUUID).produceVolume(imgUUID=srcImg, volUUID=srcVol)

1 0

[NEW PATCH] BZ#718488 deployUtil: run on python2.4 (via gerrit-bot)
by Dan Kenigsberg 03 Jul '11

03 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/647 commit 727a1491b0c5391d1b7715215307edfdd123a1bd Author: Dan Kenigsberg <danken(a)redhat.com> Date: Sun Jul 3 11:06:26 2011 +0300 BZ#718488 deployUtil: run on python2.4 Change-Id: I5d48514b06107afc19e959a4d692c4e047f965cf diff --git a/vdsm_reg/deployUtil.py b/vdsm_reg/deployUtil.py index d6a3b9b..7564ed0 100644 --- a/vdsm_reg/deployUtil.py +++ b/vdsm_reg/deployUtil.py @@ -1316,8 +1316,11 @@ def getRhevmCert(IP, port): dirName = os.path.dirname(CACERT) if not os.path.exists(dirName): os.makedirs(dirName) - with file(CACERT, "w+") as crt: + crt = file(CACERT, "w+") + try: crt.write(rhevmCert) + finally: + crt.close() return True else: return False

1 0

[NEW PATCH] BZ#712106 - Add lock for template during moveImage operation (via gerrit-bot)
by Igor Lvovsky 03 Jul '11

03 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/630 commit fea663462e0da5f717bba339a9299b54ee2ee18b Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Tue Jun 21 13:52:20 2011 +0300 BZ#712106 - Add lock for template during moveImage operation Change-Id: Ib87649cf0aebd5a35d56faa7e8f165d10d715331 diff --git a/vdsm/constants.py b/vdsm/constants.py index ae9a79b..3a800a4 100644 --- a/vdsm/constants.py +++ b/vdsm/constants.py @@ -139,6 +139,11 @@ CMD_LOWPRIO = [EXT_NICE, '-n', '19', EXT_IONICE, '-c', '2', '-n', '7'] # # Storage constants # + +LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' +IMAGE_NAMESPACE = 'imageNS' +VOLUME_NAMESPACE = 'volumeNS' + STRG_ISCSI_HOST = "iscsi_host/" STRG_SCSI_HOST = "scsi_host/" STRG_ISCSI_SESSION = "iscsi_session/" diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 214e8a4..01b2182 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -27,7 +27,7 @@ import safelease import blockVolume import multipath import resourceFactories -from resourceFactories import LVM_ACTIVATION_NAMESPACE +from constants import LVM_ACTIVATION_NAMESPACE from persistentDict import PersistentDict, DictValidator import iscsi import storage_exception as se diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py index 3da76fa..cac25e8 100644 --- a/vdsm/storage/blockVolume.py +++ b/vdsm/storage/blockVolume.py @@ -24,7 +24,7 @@ import lvm import resourceManager as rm from threadLocal import vars from sdf import StorageDomainFactory as SDF -from resourceFactories import LVM_ACTIVATION_NAMESPACE +from constants import LVM_ACTIVATION_NAMESPACE import fileUtils TAG_PREFIX_MD = "MD_" diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 2aaf822..c00cb70 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -38,7 +38,7 @@ from threadLocal import vars import constants from storageConstants import STORAGE from task import Job -from resourceFactories import IMAGE_NAMESPACE +from constants import IMAGE_NAMESPACE import resourceManager as rm import devicemapper diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 40a82c9..85f9561 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -22,6 +22,11 @@ from config import config import storage_exception as se import task from threadLocal import vars +from constants import IMAGE_NAMESPACE +import resourceManager as rm +from contextlib import contextmanager + +rmanager = rm.ResourceManager.getInstance() # Disk type UNKNOWN_DISK_TYPE = 0 @@ -137,7 +142,14 @@ class Image: vars.task.pushRecovery(task.Recovery(name, "image", "Image", "deleteRecover", [self.repoPath, sdUUID, imgUUID, str(postZero), str(force)])) - self._delete(sdUUID, imgUUID, postZero, force) + try: + self._delete(sdUUID, imgUUID, postZero, force) + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.ImageDeleteError("%s: %s" % (imgUUID), str(e)) @classmethod def deleteRecover(cls, taskObj, repoPath, sdUUID, imgUUID, postZero, force): @@ -315,7 +327,8 @@ class Image: srcVol = None if not srcVol: - raise se.ImageIsNotLegalChain("Image %s is not a leaf" % imgUUID) + self.log.error("There is no leaf in the image %s" % imgUUID) + raise se.ImageIsNotLegalChain(imgUUID) # Build up the sorted (parent->child) chain while not srcVol.isShared(): @@ -450,48 +463,29 @@ class Image: """ try: if srcVol: - srcVol.teardown(sdUUID=srcVol.sdUUID, volUUID=srcVol.volUUID, justme=True) + srcVol.teardown(sdUUID=srcVol.sdUUID, volUUID=srcVol.volUUID) if dstVol: - dstVol.teardown(sdUUID=dstVol.sdUUID, volUUID=dstVol.volUUID, justme=True) + dstVol.teardown(sdUUID=dstVol.sdUUID, volUUID=dstVol.volUUID) except Exception: self.log.error("Unexpected error", exc_info=True) - def move(self, srcSdUUID, dstSdUUID, imgUUID, vmUUID, op, postZero, force): - """ - Move/Copy image between storage domains within same storage pool - """ - self.log.info("srcSdUUID=%s dstSdUUID=%s "\ - "imgUUID=%s vmUUID=%s op=%s force=%s postZero=%s", - srcSdUUID, dstSdUUID, imgUUID, vmUUID, OP_TYPES[op], str(force), str(postZero)) - - srcVol = leafVol = dstVol = None + def _createTargetImage(self, destDom, srcSdUUID, imgUUID): + # Before actual data copying we need perform several operation + # such as: create all volumes, create fake template if needed, ... try: # Find all volumes of source image - chain = self.getChain(srcSdUUID, imgUUID) - leafVol = chain[-1] - except se.StorageException, e: + srcChain = self.getChain(srcSdUUID, imgUUID) + except se.StorageException: self.log.error("Unexpected error", exc_info=True) raise except Exception, e: self.log.error("Unexpected error", exc_info=True) raise se.SourceImageActionError(imgUUID, srcSdUUID, str(e)) - destDom = SDF.produce(dstSdUUID) + fakeTemplate = False pimg = volume.BLANK_UUID # standalone chain - # If image already exists check whether it illegal/fake, overwrite it - if not self.isLegal(dstSdUUID, imgUUID): - force = True - # We must first remove the previous instance of image (if exists) - # in destination domain, if we got the overwrite command - if force: - try: - self.log.info("delete image %s on domain %s before overwriting", imgUUID, dstSdUUID) - self.delete(dstSdUUID, imgUUID, postZero, force=True) - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise e # check if the chain is build above a template, or it is a standalone - pvol = chain[0].getParentVolume() + pvol = srcChain[0].getParentVolume() if pvol: # find out parent volume parameters volParams = pvol.getVolumeParams() @@ -500,81 +494,135 @@ class Image: # FIXME: This workaround help as copy VM to the backup domain without its template # We will create fake template for future VM creation and mark it as FAKE volume # This situation is relevant for backup domain only - self.createFakeTemplate(dstSdUUID, volParams) + fakeTemplate = True + + @contextmanager + def justLogIt(pimg): + self.log.debug("You don't really need lock for image %s", pimg) + yield + + dstTmplResourcesNamespace = sd.getNamespace(destDom.sdUUID, IMAGE_NAMESPACE) + # In destination domain we need to lock image's template if exists + with rmanager.acquireResource(dstTmplResourcesNamespace, pimg, rm.LockType.shared) \ + if pimg != volume.BLANK_UUID else justLogIt(pimg): + if fakeTemplate: + self.createFakeTemplate(destDom.sdUUID, volParams) + + dstChain = [] + for srcVol in srcChain: + # Create the dst volume + try: + # find out src volume parameters + volParams = srcVol.getVolumeParams(bs=1) + + # To avoid 'prezeroing' preallocated volume on NFS domain, + # we create the target volume with minimal size and after that w'll change + # its metadata back to the original size. + tmpSize = 20480 # in sectors (10M) + destDom.createVolume(imgUUID=imgUUID, size=tmpSize, + volFormat=volParams['volFormat'], preallocate=volParams['prealloc'], + diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], + srcImgUUID=pimg, srcVolUUID=volParams['parent']) + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) + # Extend volume (for LV only) size to the actual size + dstVol.extend((volParams['apparentsize'] + 511) / 512) + # Change destination volume metadata back to the original size. + dstVol.setSize(volParams['size']) + dstChain.append(dstVol) + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.DestImageActionError(imgUUID, destDom.sdUUID, str(e)) - for srcVol in chain: - # Create the dst volume - try: - # find out src volume parameters - volParams = srcVol.getVolumeParams(bs=1) + # only base may have a different parent image + pimg = imgUUID - # To avoid 'prezeroing' preallocated volume on NFS domain, - # we create the target volume with minimal size and after that w'll change - # its metadata back to the original size. - tmpSize = 20480 # in sectors (10M) - destDom.createVolume(imgUUID=imgUUID, size=tmpSize, - volFormat=volParams['volFormat'], preallocate=volParams['prealloc'], - diskType=volParams['disktype'], volUUID=srcVol.volUUID, desc=volParams['descr'], - srcImgUUID=pimg, srcVolUUID=volParams['parent']) - # Extend volume (for LV only) size to the actual size - dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) - dstVol.extend((volParams['apparentsize'] + 511) / 512) - # Change destination volume metadata back to the original size. - dstVol.setSize(volParams['size']) - except se.StorageException, e: - self.log.error("Unexpected error", exc_info=True) - raise - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise se.DestImageActionError(imgUUID, dstSdUUID, str(e)) + return {'srcChain':srcChain, 'dstChain':dstChain} - # Do the actual copy - try: + def _interImagesCopy(self, destDom, srcSdUUID, imgUUID, chains): + srcLeafVol = chains['srcChain'][-1] + dstLeafVol = chains['dstChain'][-1] + try: + # Prepare the whole chains before the copy + srcLeafVol.prepare(rw=False) + dstLeafVol.prepare(rw=True, chainrw=True, setrw=True) + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + # teardown volumes + self.__cleanupMove(srcLeafVol, dstLeafVol) + raise + + try: + for srcVol in chains['srcChain']: + # Do the actual copy try: + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) srcSize = srcVol.getVolumeSize(bs=1) - # Prepare relevant volumes before copy and teardown after that - srcVol.prepare(rw=False, justme=True) - dstVol.prepare(rw=True, justme=True, setrw=True) misc.ddWatchCopy(srcVol.getVolumePath(), dstVol.getVolumePath(), vars.task.aborting, self.idle, size=srcSize) - # In case of copying template, we should set the destination volume - # as SHARED (after copy because otherwise prepare as RW would fail) - if srcVol.isShared(): - dstVol.setShared() - elif srcVol.isInternal(): - dstVol.setInternal() - except se.ActionStopped, e: - raise e - except se.StorageException, e: + except se.ActionStopped: + raise + except se.StorageException: self.log.error("Unexpected error", exc_info=True) raise except Exception, e: self.log.error("Unexpected error", exc_info=True) - raise se.CopyImageError("image=%s, src domain=%s, dst domain=%s: msg %s" % (imgUUID, srcSdUUID, dstSdUUID, str(e))) - finally: - # teardown volumes - self.__cleanupMove(srcVol, dstVol) - srcVol = dstVol = None + raise se.CopyImageError("image=%s, src domain=%s, dst domain=%s: msg %s" % (imgUUID, srcSdUUID, destDom.sdUUID, str(e))) + finally: + # teardown volumes + self.__cleanupMove(srcLeafVol, dstLeafVol) - # only base may have a different parent image - pimg = imgUUID + def _finalizeDestinationImage(self, destDom, imgUUID, chains, force): + for srcVol in chains['srcChain']: + try: + dstVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=srcVol.volUUID) + # In case of copying template, we should set the destination volume + # as SHARED (after copy because otherwise prepare as RW would fail) + if srcVol.isShared(): + dstVol.setShared() + elif srcVol.isInternal(): + dstVol.setInternal() + except se.StorageException: + self.log.error("Unexpected error", exc_info=True) + raise + except Exception, e: + self.log.error("Unexpected error", exc_info=True) + raise se.DestImageActionError(imgUUID, destDom.sdUUID, str(e)) if force: + leafVol = chains['dstChain'][-1] # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, imgUUID, leafVol.volUUID) + self.__templateRelink(destDom.sdUUID, imgUUID, leafVol.volUUID) # At this point we successfully finished the 'copy' part of the operation # and we can clear all recoveries. vars.task.clearRecoveries() + + def move(self, srcSdUUID, dstSdUUID, imgUUID, vmUUID, op, postZero, force): + """ + Move/Copy image between storage domains within same storage pool + """ + self.log.info("srcSdUUID=%s dstSdUUID=%s "\ + "imgUUID=%s vmUUID=%s op=%s force=%s postZero=%s", + srcSdUUID, dstSdUUID, imgUUID, vmUUID, OP_TYPES[op], str(force), str(postZero)) + + destDom = SDF.produce(dstSdUUID) + # If image already exists check whether it illegal/fake, overwrite it + if not self.isLegal(destDom.sdUUID, imgUUID): + force = True + # We must first remove the previous instance of image (if exists) + # in destination domain, if we got the overwrite command + if force: + self.log.info("delete image %s on domain %s before overwriting", imgUUID, destDom.sdUUID) + self.delete(destDom.sdUUID, imgUUID, postZero, force=True) + + chains = self._createTargetImage(destDom, srcSdUUID, imgUUID) + self._interImagesCopy(destDom, srcSdUUID, imgUUID, chains) + self._finalizeDestinationImage(destDom, imgUUID, chains, force) # If it's 'move' operation, we should delete src image after copying if op == MOVE_OP: - try: - self.delete(srcSdUUID, imgUUID, postZero, force=True) - except se.StorageException, e: - self.log.error("Unexpected error", exc_info=True) - raise - except Exception, e: - self.log.error("Unexpected error", exc_info=True) - raise se.ImageDeleteError("%s: %s" % (imgUUID), str(e)) + self.delete(srcSdUUID, imgUUID, postZero, force=True) self.log.info("%s task on image %s was successfully finished", OP_TYPES[op], imgUUID) return True @@ -587,7 +635,7 @@ class Image: try: self.delete(sdUUID, imgUUID, postZero, force=True) except Exception: - self.log.error("Unexpected error", exc_info=True) + pass def multiMove(self, srcSdUUID, dstSdUUID, imgDict, vmUUID, force): """ @@ -616,8 +664,8 @@ class Image: for (imgUUID, postZero) in imgDict.iteritems(): try: self.delete(srcSdUUID, imgUUID, postZero, force=True) - except Exception, e: - self.log.error("Unexpected error", exc_info=True) + except Exception: + pass def __cleanupCopy(self, srcVol, dstVol): """ @@ -708,12 +756,8 @@ class Image: # We must first remove the previous instance of image (if exists) # in destination domain, if we got the overwrite command if force: - try: - self.log.info("delete image %s on domain %s before overwriting", dstImgUUID, dstSdUUID) - self.delete(dstSdUUID, dstImgUUID, postZero, force=True) - except Exception, e: - self.log.error(e, exc_info=True) - raise se.OverwriteImageError(dstImgUUID, dstSdUUID) + self.log.info("delete image %s on domain %s before overwriting", dstImgUUID, dstSdUUID) + self.delete(dstSdUUID, dstImgUUID, postZero, force=True) # To avoid 'prezeroing' preallocated volume on NFS domain, # we create the target volume with minimal size and after that w'll change diff --git a/vdsm/storage/resourceFactories.py b/vdsm/storage/resourceFactories.py index ce3ec70..4836217 100644 --- a/vdsm/storage/resourceFactories.py +++ b/vdsm/storage/resourceFactories.py @@ -16,10 +16,7 @@ import storage_exception as se from sdf import StorageDomainFactory as SDF import sd import image - -LVM_ACTIVATION_NAMESPACE = 'lvmActivationNS' -IMAGE_NAMESPACE = 'imageNS' -VOLUME_NAMESPACE = 'volumeNS' +from constants import VOLUME_NAMESPACE rmanager = rm.ResourceManager.getInstance() diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 325da5d..4cdaef9 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -17,7 +17,7 @@ from glob import glob import storage_exception as se import misc import resourceFactories -from resourceFactories import IMAGE_NAMESPACE, VOLUME_NAMESPACE +from constants import IMAGE_NAMESPACE, VOLUME_NAMESPACE import resourceManager as rm import constants import safelease diff --git a/vdsm/storage/spm.py b/vdsm/storage/spm.py index b186d39..6ed480b 100644 --- a/vdsm/storage/spm.py +++ b/vdsm/storage/spm.py @@ -36,7 +36,7 @@ import logging import storage_exception as se from threadLocal import vars from storageConstants import STORAGE -from resourceFactories import IMAGE_NAMESPACE +from constants import IMAGE_NAMESPACE import resourceManager as rm from contextlib import nested import fileUtils diff --git a/vdsm/storage/storage_exception.py b/vdsm/storage/storage_exception.py index b1cbcdf..c9df109 100644 --- a/vdsm/storage/storage_exception.py +++ b/vdsm/storage/storage_exception.py @@ -274,7 +274,7 @@ class ImageValidationError(StorageException): class ImageDeleteError(StorageException): code = 256 - message = "Could not remove all files" + message = "Could not remove all image's volumes" #REMOVED in 2.3. #This class was not in use in 2.2 nor in 2.3. diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py index 123ed37..0052e84 100644 --- a/vdsm/storage/volume.py +++ b/vdsm/storage/volume.py @@ -21,8 +21,9 @@ import misc import fileUtils import task from threadLocal import vars -import resourceFactories import resourceManager as rm +from constants import IMAGE_NAMESPACE + rmanager = rm.ResourceManager.getInstance() @@ -150,7 +151,7 @@ class Volume: cls.log.info("sdUUID=%s srcImg=%s srcVol=%s dstFormat=%s srcParent=%s", sdUUID, srcImg, srcVol, dstFormat, srcParent) - imageResourcesNamespace = sd.getNamespace(sdUUID, resourceFactories.IMAGE_NAMESPACE) + imageResourcesNamespace = sd.getNamespace(sdUUID, IMAGE_NAMESPACE) with rmanager.acquireResource(imageResourcesNamespace, srcImg, rm.LockType.exclusive): try: vol = SDF.produce(sdUUID).produceVolume(imgUUID=srcImg, volUUID=srcVol)

1 0

[NEW PATCH] BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains (via gerrit-bot)
by Igor Lvovsky 03 Jul '11

03 Jul '11

New patch submitted by Igor Lvovsky (ilvovsky(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/646 commit 462529ce907980458db5a24849107a85ccbce9e7 Author: Igor Lvovsky <ilvovsky(a)redhat.com> Date: Thu Jun 30 18:01:27 2011 +0300 BZ#717952 - Fix race in moveImage with overwrite of VM with several disks on NFS domains Change-Id: I367e83a700e84be0e4cb0ca5b7a02416cb54ac25 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 85f9561..2eef6a5 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -55,7 +55,7 @@ class Image: Consist from chain of volumes. """ log = logging.getLogger('Storage.Image') - _lock = threading.Lock() + _fakeTemplateLock = threading.Lock() @classmethod def createImageRollback(cls, taskObj, imageDir): @@ -384,15 +384,18 @@ class Image: self.log.error("Unexpected error", exc_info=True) raise se.CouldNotValideTemplateOnTargetDomain("Template %s Destination domain %s: %s" % (pimg, dstSdUUID, str(e))) - def __templateRelink(self, sdUUID, imgUUID, volUUID): + def __templateRelink(self, destDom, imgUUID, volUUID): """ Relink all hardlinks of the template 'volUUID' in all VMs based on it """ - destDom = SDF.produce(sdUUID) # Avoid relink templates for SAN domains if destDom.getStorageType() in [ sd.NFS_DOMAIN ]: vol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - chList = vol.getAllChildrenList(self.repoPath, sdUUID, imgUUID, volUUID) + # Relink templates only + if not vol.isShared(): + self.log.debug("Doesn't relink regular volume %s of image %s", volUUID, imgUUID) + return + chList = vol.getAllChildrenList(self.repoPath, destDom.sdUUID, imgUUID, volUUID) for ch in chList: # Remove hardlink of this template v = destDom.produceVolume(imgUUID=ch['imgUUID'], volUUID=volUUID) @@ -400,7 +403,7 @@ class Image: # Now we should re-link deleted hardlink, if exists newVol = destDom.produceVolume(imgUUID=imgUUID, volUUID=volUUID) - imageDir = self.getImageDir(sdUUID, ch['imgUUID']) + imageDir = self.getImageDir(destDom.sdUUID, ch['imgUUID']) newVol.share(imageDir) else: self.log.debug("Doesn't relink templates non-NFS domain %s", destDom.sdUUID) @@ -409,8 +412,7 @@ class Image: """ Create fake template (relevant for Backup domain only) """ - try: - self._lock.acquire() + with self._fakeTemplateLock: try: destDom = SDF.produce(sdUUID) volclass = destDom.getVolumeClass() @@ -430,14 +432,12 @@ class Image: # Mark fake volume as shared vol.setShared() # Now we should re-link all hardlinks of this template in all VMs based on it - self.__templateRelink(sdUUID, volParams['imgUUID'], volParams['volUUID']) + self.__templateRelink(destDom, volParams['imgUUID'], volParams['volUUID']) self.log.debug("Succeeded to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID) except Exception: self.log.error("Failure to create fake image %s in domain %s", volParams['imgUUID'], destDom.sdUUID, exc_info=True) - finally: - self._lock.release() def isLegal(self, sdUUID, imgUUID): """ @@ -593,7 +593,7 @@ class Image: if force: leafVol = chains['dstChain'][-1] # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(destDom.sdUUID, imgUUID, leafVol.volUUID) + self.__templateRelink(destDom, imgUUID, leafVol.volUUID) # At this point we successfully finished the 'copy' part of the operation # and we can clear all recoveries. @@ -807,7 +807,7 @@ class Image: if force: # Now we should re-link all deleted hardlinks, if exists - self.__templateRelink(dstSdUUID, dstImgUUID, dstVolUUID) + self.__templateRelink(destDom, dstImgUUID, dstVolUUID) except se.StorageException, e: self.log.error("Unexpected error", exc_info=True) raise

1 0

[MERGED PATCH] Related to BZ#716692: avoid pyflakes error (via gerrit-bot)
by Dan Kenigsberg 03 Jul '11

03 Jul '11

Patch merged into repository You can review this change at: http://gerrit.usersys.redhat.com/645

1 0

[NEW PATCH] Related to BZ#716692: avoid pyflakes error (via gerrit-bot)
by Dan Kenigsberg 03 Jul '11

03 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/645 commit dc821613120e7c42874ab607ac5536346739d826 Author: Dan Kenigsberg <danken(a)redhat.com> Date: Sun Jul 3 11:12:42 2011 +0300 Related to BZ#716692: avoid pyflakes error Change-Id: I0a3d179549aa744ac47749fbcac236ef2211cbe9 diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index c26bc25..e1bbf90 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -147,7 +147,7 @@ def getRoutes(): gateways = dict() with open("/proc/net/route") as route_file: - head = route_file.readline() + route_file.readline() # skip header line for route_line in route_file.xreadlines(): route_parm = route_line.rstrip().split('\t')

1 0

[NEW PATCH] BZ#716692 Report the real gateway for the networks (via gerrit-bot)
by Dan Kenigsberg 01 Jul '11

01 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/631 commit 0c4496b13119f7480843e4a2daff90d614b1f26d Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Tue Jun 28 14:00:31 2011 +0000 BZ#716692 Report the real gateway for the networks Change-Id: Ifcafa3617bcab3a720f229bfce45d49c9109b37d diff --git a/vdsm/netinfo.py b/vdsm/netinfo.py index d2d0795..c26bc25 100644 --- a/vdsm/netinfo.py +++ b/vdsm/netinfo.py @@ -131,6 +131,33 @@ def getVlanBondingNic(bridge): nics = [iface] return vlan, bonding, nics +def intToAddress(ip_num): + "Convert an integer to the corresponding ip address in the dot-notation" + ip_address = [] + + for i in xrange(4): + ip_num, ip_val = divmod(ip_num, 256) + ip_address.append(str(ip_val)) + + return '.'.join(ip_address) + +def getRoutes(): + """Return the interface gateway or None if not found.""" + + gateways = dict() + + with open("/proc/net/route") as route_file: + head = route_file.readline() + + for route_line in route_file.xreadlines(): + route_parm = route_line.rstrip().split('\t') + + if route_parm[2] != '00000000': + ip_num = int(route_parm[2], 16) + gateways[route_parm[0]] = intToAddress(ip_num) + + return gateways + def getIfaceCfg(iface): d = {} try: @@ -161,11 +188,13 @@ def permAddr(): def get(): d = {} ifaces = ifconfig() + routes = getRoutes() # FIXME handle bridge/nic missing from ifconfig d['networks'] = dict([ (bridge, {'ports': ports(bridge), 'stp': bridge_stp_state(bridge), 'addr': ifaces[bridge]['addr'], 'netmask': ifaces[bridge]['netmask'], + 'gateway': routes.get(bridge, '0.0.0.0'), 'cfg': getIfaceCfg(bridge)}) for bridge in bridges() ]) d['nics'] = dict([ (nic, {'speed': speed(nic),

1 1

[NEW PATCH] BZ#716962 logrotate: compress logs using the xz format (via gerrit-bot)
by Dan Kenigsberg 01 Jul '11

01 Jul '11

New patch submitted by Dan Kenigsberg (danken(a)redhat.com) You can review this change at: http://gerrit.usersys.redhat.com/632 commit f7d79ce0791e8cb8281b0d5974c2667ce2b38c49 Author: Federico Simoncelli <fsimonce(a)redhat.com> Date: Tue Jun 28 14:12:04 2011 +0000 BZ#716962 logrotate: compress logs using the xz format Change-Id: I736c5e625e4d4fba8d5d7d220ee86df9f01cd7fb diff --git a/vdsm/vdsm-logrotate b/vdsm/vdsm-logrotate index 4c84d7e..efa9e09 100755 --- a/vdsm/vdsm-logrotate +++ b/vdsm/vdsm-logrotate @@ -7,7 +7,6 @@ if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]" fi -/usr/bin/find /var/log/core -type f -name '*gz' -mtime +7 -exec /bin/rm -f '{}' \; EXITVALUE=$? if [ $EXITVALUE != 0 ]; then /usr/bin/logger -t logrotate "ALERT clean old core files exited abnormally with [$EXITVALUE]" diff --git a/vdsm/vdsm-logrotate.conf b/vdsm/vdsm-logrotate.conf index 2962ce3..12e16ad 100644 --- a/vdsm/vdsm-logrotate.conf +++ b/vdsm/vdsm-logrotate.conf @@ -4,12 +4,21 @@ copytruncate size 15M compress + compresscmd /usr/bin/xz + uncompresscmd /usr/bin/unxz + compressext .xz } /var/log/core/*.dump { rotate 1 missingok size 0 compress + compresscmd /usr/bin/xz + uncompresscmd /usr/bin/unxz + compressext .xz +} +/var/log/core/*.dump.xz { + maxage 7 } /var/log/libvirtd*.log { rotate 100 @@ -17,4 +26,7 @@ copytruncate size 15M compress + compresscmd /usr/bin/xz + uncompresscmd /usr/bin/unxz + compressext .xz }

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches July 2011