October 2016 - vdsm-patches - Fedora mailing-lists

Change in vdsm[master]: sp: Remove vol_extend_policy option
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Remove vol_extend_policy option ...................................................................... sp: Remove vol_extend_policy option We had a vol_extend_policy option disabling the mailbox. This option is not useful for anything but causing vms using thin block disks to pause without a way to recover. This option was added in commit 73adfc489e6e (Add spmProtect, release lock and toggle lock functionality. Add toggle lvextend functionality) in 2009. I did not find why this option was added. Change-Id: Ib067c09a985a633a452c476e21d8c2c073e6ca50 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M lib/vdsm/config.py.in M vdsm/storage/sp.py 2 files changed, 2 insertions(+), 8 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/53/65053/1 diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in index 94640b2..55ce6f3 100644 --- a/lib/vdsm/config.py.in +++ b/lib/vdsm/config.py.in @@ -329,8 +329,6 @@ 'NFS mount options, comma-separated list (NB: no white space ' 'allowed!)'), - ('vol_extend_policy', 'ON', None), - ('lock_util_path', '@LIBEXECDIR@', None), ('lock_cmd', 'spmprotect.sh', None), diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 8f98205..febd444 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -68,7 +68,6 @@ log = logging.getLogger('storage.StoragePool') storage_repository = config.get('irs', 'repository') - lvExtendPolicy = config.get('irs', 'vol_extend_policy') def __init__(self, spUUID, domainMonitor, taskManager): self._secured = threading.Event() @@ -315,9 +314,7 @@ # commands are allowed to run to prevent a race between the # mailbox and the "self._setSecure() call" - # FIXME : Use a system wide grouping mechanism - if (self.lvExtendPolicy == "ON" and - self.masterDomain.supportsMailbox): + if self.masterDomain.supportsMailbox: self.masterDomain.prepareMailbox() self.spmMailer = storage_mailbox.SPM_MailMonitor(self, maxHostID) @@ -450,8 +447,7 @@ if self.hsmMailer: return - if (self.lvExtendPolicy == "ON" and - self.masterDomain.supportsMailbox): + if self.masterDomain.supportsMailbox: self.hsmMailer = storage_mailbox.HSM_Mailbox(self.id, self.spUUID) self.log.debug("HSM mailbox ready for pool %s on master " "domain %s", self.spUUID, self.masterDomain.sdUUID) -- To view, visit https://gerrit.ovirt.org/65053 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib067c09a985a633a452c476e21d8c2c073e6ca50 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 17

Change in vdsm[master]: sp: Sort dict literal by key
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Sort dict literal by key ...................................................................... sp: Sort dict literal by key With more then few items, it is easier to work with sorted items, and people are less likely to add duplicate keys at the end, as see in commit f9cf58b7bced. Change-Id: I128ed804fcdbdf7b6b562f5cef662ae2ba2c9014 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 6 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/52/65052/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index dbc89c9..8f98205 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1360,16 +1360,16 @@ self.masterDomain.sdUUID) poolInfo = { - 'type': msdInfo['type'], - 'name': '', 'domains': '', + 'isoprefix': '', + 'lver': LVER_INVALID, 'master_uuid': self.masterDomain.sdUUID, 'master_ver': self._backend.getMasterVersion(), - 'lver': LVER_INVALID, - 'spm_id': SPM_ID_FREE, - 'version': str(msdInfo['version']), - 'isoprefix': '', + 'name': '', 'pool_status': 'connected', + 'spm_id': SPM_ID_FREE, + 'type': msdInfo['type'], + 'version': str(msdInfo['version']), } poolInfo.update(self._backend.getInfo()) -- To view, visit https://gerrit.ovirt.org/65052 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I128ed804fcdbdf7b6b562f5cef662ae2ba2c9014 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

5 17

Change in vdsm[master]: sp: Remove double dict key
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Remove double dict key ...................................................................... sp: Remove double dict key Commit f9cf58b7bced (Make getRepoStats() a hsm method) added a second "pool_status" key in the same dictionary. Turns out that this is well defined, and the second key overrides the first one. Remove the old unused key. For more info see: https://docs.python.org/2/reference/expressions.html#dictionary-displays Change-Id: I05f6aeda1858e85b8660d972593e82bcb7e3bff3 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 0 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/51/65051/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index dabe892..dbc89c9 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1367,7 +1367,6 @@ 'master_ver': self._backend.getMasterVersion(), 'lver': LVER_INVALID, 'spm_id': SPM_ID_FREE, - 'pool_status': 'uninitialized', 'version': str(msdInfo['version']), 'isoprefix': '', 'pool_status': 'connected', -- To view, visit https://gerrit.ovirt.org/65051 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I05f6aeda1858e85b8660d972593e82bcb7e3bff3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

5 17

Change in vdsm[master]: sp: pep8 1.6.2 fix
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: pep8 1.6.2 fix ...................................................................... sp: pep8 1.6.2 fix Both pep8 tool and me do not like line breaks before binary operators. Change-Id: Ic8e561da219c31aa26d32dcb615eacf223050f5c Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/50/65050/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index b4b1731..dabe892 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -316,8 +316,8 @@ # mailbox and the "self._setSecure() call" # FIXME : Use a system wide grouping mechanism - if (self.lvExtendPolicy == "ON" - and self.masterDomain.supportsMailbox): + if (self.lvExtendPolicy == "ON" and + self.masterDomain.supportsMailbox): self.masterDomain.prepareMailbox() self.spmMailer = storage_mailbox.SPM_MailMonitor(self, maxHostID) -- To view, visit https://gerrit.ovirt.org/65050 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic8e561da219c31aa26d32dcb615eacf223050f5c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 14

Change in vdsm[master]: sp: Streamline usage of image.Image class
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Streamline usage of image.Image class ...................................................................... sp: Streamline usage of image.Image class We used to create an image.Image an call a method in one line, making the code hard to read for no benefit. Separate creation of the image from the call. Change-Id: Icc9de180542cd81a6d563d52b4b30b9520046469 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 16 insertions(+), 14 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/49/65049/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 3297588..b317e7f 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1528,7 +1528,8 @@ with nested(rm.acquireResource(src_img_ns, srcImgUUID, rm.SHARED), rm.acquireResource(dst_img_ns, dstImgUUID, rm.EXCLUSIVE) ): - dstUUID = image.Image(self.poolPath).copyCollapsed( + img = image.Image(self.poolPath) + dstUUID = img.copyCollapsed( sdUUID, vmUUID, srcImgUUID, srcVolUUID, dstImgUUID, dstVolUUID, descr, dstSdUUID, volType, volFormat, preallocate, postZero, force) @@ -1571,8 +1572,9 @@ with nested(rm.acquireResource(src_img_ns, imgUUID, srcLock), rm.acquireResource(dst_img_ns, imgUUID, rm.EXCLUSIVE)): - image.Image(self.poolPath).move(srcDomUUID, dstDomUUID, imgUUID, - vmUUID, op, postZero, force) + img = image.Image(self.poolPath) + img.move(srcDomUUID, dstDomUUID, imgUUID, vmUUID, op, postZero, + force) def sparsifyImage(self, tmpSdUUID, tmpImgUUID, tmpVolUUID, dstSdUUID, dstImgUUID, dstVolUUID): @@ -1610,9 +1612,9 @@ with nested( rm.acquireResource(srcNamespace, tmpImgUUID, rm.EXCLUSIVE), rm.acquireResource(dstNamespace, dstImgUUID, rm.EXCLUSIVE)): - image.Image(self.poolPath).sparsify( - tmpSdUUID, tmpImgUUID, tmpVolUUID, dstSdUUID, dstImgUUID, - dstVolUUID) + img = image.Image(self.poolPath) + img.sparsify(tmpSdUUID, tmpImgUUID, tmpVolUUID, dstSdUUID, + dstImgUUID, dstVolUUID) def cloneImageStructure(self, sdUUID, imgUUID, dstSdUUID): """ @@ -1638,8 +1640,8 @@ ))) with nested(*resList): - image.Image(self.poolPath).cloneStructure( - sdUUID, imgUUID, dstSdUUID) + img = image.Image(self.poolPath) + img.cloneStructure( sdUUID, imgUUID, dstSdUUID) def syncImageData(self, sdUUID, imgUUID, dstSdUUID, syncType): """ @@ -1666,8 +1668,8 @@ ))) with nested(*resList): - image.Image(self.poolPath).syncData( - sdUUID, imgUUID, dstSdUUID, syncType) + img = image.Image(self.poolPath) + img.syncData(sdUUID, imgUUID, dstSdUUID, syncType) def uploadImage(self, methodArgs, sdUUID, imgUUID, volUUID=None): """ @@ -1752,8 +1754,8 @@ dst_img_ns, imgUUID, rm.EXCLUSIVE)) with nested(*resourceList): - image.Image(self.poolPath).multiMove( - srcDomUUID, dstDomUUID, imgDict, vmUUID, force) + img = image.Image(self.poolPath) + img.multiMove( srcDomUUID, dstDomUUID, imgDict, vmUUID, force) def reconcileVolumeChain(self, sdUUID, imgUUID, leafVolUUID): """ @@ -1798,8 +1800,8 @@ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): - image.Image(self.poolPath).merge( - sdUUID, vmUUID, imgUUID, ancestor, successor, postZero) + img = image.Image(self.poolPath) + img.merge(sdUUID, vmUUID, imgUUID, ancestor, successor, postZero) def createVolume(self, sdUUID, imgUUID, size, volFormat, preallocate, diskType, volUUID=None, desc="", -- To view, visit https://gerrit.ovirt.org/65049 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icc9de180542cd81a6d563d52b4b30b9520046469 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 16

Change in vdsm[master]: sp: Fix positinal args calling convention
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Fix positinal args calling convention ...................................................................... sp: Fix positinal args calling convention StorageDomain.setDescription and Volume.setLegality accept one positional argument. Using kwarg calling style is pointless. Change-Id: I2d73df3daa59efb512bfdc9f6e400bacb3c08c49 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 3 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/48/65048/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 78562c9..3297588 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1942,14 +1942,14 @@ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): vol = sdCache.produce(sdUUID).produceVolume(imgUUID, volUUID) - vol.setDescription(descr=description) + vol.setDescription(description) def setVolumeLegality(self, sdUUID, imgUUID, volUUID, legality): self.validatePoolSD(sdUUID) img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): vol = sdCache.produce(sdUUID).produceVolume(imgUUID, volUUID) - vol.setLegality(legality=legality) + vol.setLegality(legality) def getVmsList(self, sdUUID): self.validatePoolSD(sdUUID) @@ -1972,7 +1972,7 @@ def setSDDescription(self, sd, description): self.validatePoolSD(sd.sdUUID) - sd.setDescription(descr=description) + sd.setDescription(description) def getAllTasksStatuses(self): return self.taskMng.getAllTasksStatuses("spm") -- To view, visit https://gerrit.ovirt.org/65048 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2d73df3daa59efb512bfdc9f6e400bacb3c08c49 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 16

Change in vdsm[master]: sp: Streamline calls to volume methods
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Streamline calls to volume methods ...................................................................... sp: Streamline calls to volume methods When calling volume methods, we typically produce the domain, produce the volume, and calling the method. Adding a temporary variable for the volume makes the code easier to read. Change-Id: I8bbdfbd5cf6a4a7345c350359e76c2156720c683 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 6 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/47/65047/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 759ea00..78562c9 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1891,8 +1891,8 @@ with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): dom = sdCache.produce(sdUUID) for volUUID in volumes: - dom.produceVolume(imgUUID, volUUID).delete( - postZero=postZero, force=force) + vol = dom.produceVolume(imgUUID, volUUID) + vol.delete(postZero=postZero, force=force) def purgeImage(self, sdUUID, imgUUID, volsByImg): """ @@ -1941,15 +1941,15 @@ self.validatePoolSD(sdUUID) img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): - sdCache.produce(sdUUID).produceVolume( - imgUUID, volUUID).setDescription(descr=description) + vol = sdCache.produce(sdUUID).produceVolume(imgUUID, volUUID) + vol.setDescription(descr=description) def setVolumeLegality(self, sdUUID, imgUUID, volUUID, legality): self.validatePoolSD(sdUUID) img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): - sdCache.produce(sdUUID).produceVolume( - imgUUID, volUUID).setLegality(legality=legality) + vol = sdCache.produce(sdUUID).produceVolume(imgUUID, volUUID) + vol.setLegality(legality=legality) def getVmsList(self, sdUUID): self.validatePoolSD(sdUUID) -- To view, visit https://gerrit.ovirt.org/65047 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8bbdfbd5cf6a4a7345c350359e76c2156720c683 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 14

Change in vdsm[master]: sp: Fix domain.produceVolume calling convention
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Fix domain.produceVolume calling convention ...................................................................... sp: Fix domain.produceVolume calling convention As a general rule positional args should be called as positional args (arg), and kwargs as kwargs (arg=value). In some cases using kwarg call style for positional argument makes the code more clear, but not in the case of domain.procudeVolume. Change-Id: Id804b4754507f1c6bf4e85f006f6db6fdda2e330 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 3 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/65046/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 2e50a98..759ea00 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1848,8 +1848,7 @@ if imgUUID != srcImgUUID and srcImgUUID != sc.BLANK_UUID: srcDom = sdCache.produce(sdUUID) - srcVol = srcDom.produceVolume(imgUUID=srcImgUUID, - volUUID=srcVolUUID) + srcVol = srcDom.produceVolume(srcImgUUID, srcVolUUID) if not srcVol.isShared(): if srcVol.getParent() == sc.BLANK_UUID: @@ -1943,16 +1942,14 @@ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): sdCache.produce(sdUUID).produceVolume( - imgUUID=imgUUID, - volUUID=volUUID).setDescription(descr=description) + imgUUID, volUUID).setDescription(descr=description) def setVolumeLegality(self, sdUUID, imgUUID, volUUID, legality): self.validatePoolSD(sdUUID) img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): sdCache.produce(sdUUID).produceVolume( - imgUUID=imgUUID, - volUUID=volUUID).setLegality(legality=legality) + imgUUID, volUUID).setLegality(legality=legality) def getVmsList(self, sdUUID): self.validatePoolSD(sdUUID) -- To view, visit https://gerrit.ovirt.org/65046 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id804b4754507f1c6bf4e85f006f6db6fdda2e330 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 15

Change in vdsm[master]: sp: Simplify long lines using continuation \
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Simplify long lines using continuation \ ...................................................................... sp: Simplify long lines using continuation \ Eliminate fragile and hard to read continuation \ with temporary variable. Change-Id: I51fc21b452f2f1d920ad1353d2b14067d432a837 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 10 insertions(+), 10 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/45/65045/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index ed32432..2e50a98 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1340,8 +1340,8 @@ def extendVolumeSize(self, sdUUID, imgUUID, volUUID, newSize): img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): - return sdCache.produce(sdUUID) \ - .produceVolume(imgUUID, volUUID).extendSize(int(newSize)) + vol = sdCache.produce(sdUUID).produceVolume(imgUUID, volUUID) + return vol.extendSize(int(newSize)) @unsecured def getVersion(self): @@ -1676,8 +1676,8 @@ """ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.SHARED): - return image.Image(self.poolPath) \ - .upload(methodArgs, sdUUID, imgUUID, volUUID) + img = image.Image(self.poolPath) + return img.upload(methodArgs, sdUUID, imgUUID, volUUID) def downloadImage(self, methodArgs, sdUUID, imgUUID, volUUID=None): """ @@ -1686,8 +1686,8 @@ """ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): - return image.Image(self.poolPath) \ - .download(methodArgs, sdUUID, imgUUID, volUUID) + img = image.Image(self.poolPath) + return img.download(methodArgs, sdUUID, imgUUID, volUUID) def uploadImageToStream(self, methodArgs, callback, startEvent, sdUUID, imgUUID, volUUID=None): @@ -1701,8 +1701,8 @@ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.SHARED): try: - return image.Image(self.poolPath) \ - .copyFromImage(methodArgs, sdUUID, imgUUID, volUUID) + img = image.Image(self.poolPath) + return img.copyFromImage(methodArgs, sdUUID, imgUUID, volUUID) finally: callback() @@ -1714,8 +1714,8 @@ img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): try: - return image.Image(self.poolPath) \ - .copyToImage(methodArgs, sdUUID, imgUUID, volUUID) + img = image.Image(self.poolPath) + return img.copyToImage(methodArgs, sdUUID, imgUUID, volUUID) finally: callback() -- To view, visit https://gerrit.ovirt.org/65045 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I51fc21b452f2f1d920ad1353d2b14067d432a837 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 13

Change in vdsm[master]: sp: Remove useless continuation to a commented line
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Remove useless continuation to a commented line ...................................................................... sp: Remove useless continuation to a commented line Change-Id: I7b8a0f989fbd55280163c54481f1748bcfff7b6a Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 1 insertion(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/44/65044/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index ef2e9a28..ed32432 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1232,8 +1232,7 @@ fileUtils.createdir(os.path.join(domaindir, sd.DOMAIN_META_DATA)) fileUtils.createdir(os.path.join(domaindir, sd.DOMAIN_IMAGES)) # Add the file domains - for domUUID, domaindir in fileSD.scanDomains(): \ - # [(fileDomUUID, file_domaindir)] + for domUUID, domaindir in fileSD.scanDomains(): if domUUID in domUUIDs: domDirs[domUUID] = domaindir -- To view, visit https://gerrit.ovirt.org/65044 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7b8a0f989fbd55280163c54481f1748bcfff7b6a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 13

Change in vdsm[master]: sp: Streamline building domains list for upgrade
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Streamline building domains list for upgrade ...................................................................... sp: Streamline building domains list for upgrade Instead of creating a list of domain uuids, and removing the master domain uuid, remove the master the domain uuid from the domains dict. This avoids noisy exception handler and using \ continuation. While touching this code, also fix the log, we are not marking *all* domains for upgrade but only active domains. Change-Id: I2be0a4816d733fdae13bcb933201b1ede795ca68 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 4 insertions(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/43/65043/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 8e0b4bd..ef2e9a28 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -426,13 +426,10 @@ self._convertDomain(self.masterDomain, str(targetDomVersion)) - self.log.debug("Marking all domains for upgrade") - self._domainsToUpgrade = self.getDomains(activeOnly=True)\ - .keys() - try: - self._domainsToUpgrade.remove(self.masterDomain.sdUUID) - except ValueError: - pass + self.log.debug("Marking active domains for upgrade") + domains = self.getDomains(activeOnly=True) + domains.pop(self.masterDomain.sdUUID, None) + self._domainsToUpgrade = domains.keys() self.log.debug("Registering with state change event") self.domainMonitor.onDomainStateChange.register( -- To view, visit https://gerrit.ovirt.org/65043 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2be0a4816d733fdae13bcb933201b1ede795ca68 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 15

Change in vdsm[master]: sp: Do not use rm.acquireResource return value
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Do not use rm.acquireResource return value ...................................................................... sp: Do not use rm.acquireResource return value resourceManager.acquireResource is returning now a Resource object, which can be used as a context manager. This usage is deprecated, as we want to replace resourceManager with something much simpler. Replace such code with usage of rm.acquireResource as a context manger. This usage will be the only usage allowed in the future. Change-Id: I8ff5653f0b15861d2034011f7fccc76bf25788f6 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 8 insertions(+), 16 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/42/65042/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 11b3287..8e0b4bd 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1678,10 +1678,8 @@ Upload an image to a remote endpoint using the specified method and methodArgs. """ - imgResourceLock = rm.acquireResource( - sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID), imgUUID, rm.SHARED) - - with imgResourceLock: + img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) + with rm.acquireResource(img_ns, imgUUID, rm.SHARED): return image.Image(self.poolPath) \ .upload(methodArgs, sdUUID, imgUUID, volUUID) @@ -1690,10 +1688,8 @@ Download an image from a remote endpoint using the specified method and methodArgs. """ - imgResourceLock = rm.acquireResource( - sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID), imgUUID, rm.EXCLUSIVE) - - with imgResourceLock: + img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) + with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): return image.Image(self.poolPath) \ .download(methodArgs, sdUUID, imgUUID, volUUID) @@ -1706,10 +1702,8 @@ while not startEvent.is_set(): startEvent.wait() - imgResourceLock = rm.acquireResource( - sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID), imgUUID, rm.SHARED) - - with imgResourceLock: + img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) + with rm.acquireResource(img_ns, imgUUID, rm.SHARED): try: return image.Image(self.poolPath) \ .copyFromImage(methodArgs, sdUUID, imgUUID, volUUID) @@ -1721,10 +1715,8 @@ """ Download an image from a stream. """ - imgResourceLock = rm.acquireResource( - sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID), imgUUID, rm.EXCLUSIVE) - - with imgResourceLock: + img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) + with rm.acquireResource(img_ns, imgUUID, rm.EXCLUSIVE): try: return image.Image(self.poolPath) \ .copyToImage(methodArgs, sdUUID, imgUUID, volUUID) -- To view, visit https://gerrit.ovirt.org/65042 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I8ff5653f0b15861d2034011f7fccc76bf25788f6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 14

Change in vdsm[master]: sp: Streamline acquiring of multiple images locks
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: sp: Streamline acquiring of multiple images locks ...................................................................... sp: Streamline acquiring of multiple images locks Replace srcImageResourcesNamespace and dstImageResourcesNamespace with src_img_ns and dst_img_ns, streamlining code creating the namespace and using them. Change-Id: I20bcebeb11f3ccbe10b029a424f4c27a890eea8c Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 13 insertions(+), 23 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/41/65041/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index eb0ca1d..11b3287 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -1523,18 +1523,14 @@ :returns: a dict containing the UUID of the newly created image. :rtype: dict """ - srcImageResourcesNamespace = sd.getNamespace(sc.IMAGE_NAMESPACE, - sdUUID) + src_img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, sdUUID) if dstSdUUID not in (sdUUID, sd.BLANK_UUID): - dstImageResourcesNamespace = sd.getNamespace(sc.IMAGE_NAMESPACE, - dstSdUUID) + dst_img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, dstSdUUID) else: - dstImageResourcesNamespace = srcImageResourcesNamespace + dst_img_ns = src_img_ns - with nested(rm.acquireResource(srcImageResourcesNamespace, - srcImgUUID, rm.SHARED), - rm.acquireResource(dstImageResourcesNamespace, - dstImgUUID, rm.EXCLUSIVE) + with nested(rm.acquireResource(src_img_ns, srcImgUUID, rm.SHARED), + rm.acquireResource(dst_img_ns, dstImgUUID, rm.EXCLUSIVE) ): dstUUID = image.Image(self.poolPath).copyCollapsed( sdUUID, vmUUID, srcImgUUID, srcVolUUID, dstImgUUID, @@ -1566,10 +1562,8 @@ :param force: Should the operation be forced. :type force: bool """ - srcImageResourcesNamespace = sd.getNamespace(sc.IMAGE_NAMESPACE, - srcDomUUID) - dstImageResourcesNamespace = sd.getNamespace(sc.IMAGE_NAMESPACE, - dstDomUUID) + src_img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, srcDomUUID) + dst_img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, dstDomUUID) # For MOVE_OP acquire exclusive lock # For COPY_OP shared lock is enough if op == image.MOVE_OP: @@ -1579,10 +1573,8 @@ else: raise se.MoveImageError(imgUUID) - with nested(rm.acquireResource(srcImageResourcesNamespace, - imgUUID, srcLock), - rm.acquireResource(dstImageResourcesNamespace, - imgUUID, rm.EXCLUSIVE)): + with nested(rm.acquireResource(src_img_ns, imgUUID, srcLock), + rm.acquireResource(dst_img_ns, imgUUID, rm.EXCLUSIVE)): image.Image(self.poolPath).move(srcDomUUID, dstDomUUID, imgUUID, vmUUID, op, postZero, force) @@ -1758,10 +1750,8 @@ :param force: Should the operation be forced. :type force: bool """ - srcImageResourcesNamespace = sd.getNamespace(sc.IMAGE_NAMESPACE, - srcDomUUID) - dstImageResourcesNamespace = sd.getNamespace(sc.IMAGE_NAMESPACE, - dstDomUUID) + src_img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, srcDomUUID) + dst_img_ns = sd.getNamespace(sc.IMAGE_NAMESPACE, dstDomUUID) imgList = imgDict.keys() imgList.sort() @@ -1769,9 +1759,9 @@ resourceList = [] for imgUUID in imgList: resourceList.append(rm.acquireResource( - srcImageResourcesNamespace, imgUUID, rm.EXCLUSIVE)) + src_img_ns, imgUUID, rm.EXCLUSIVE)) resourceList.append(rm.acquireResource( - dstImageResourcesNamespace, imgUUID, rm.EXCLUSIVE)) + dst_img_ns, imgUUID, rm.EXCLUSIVE)) with nested(*resourceList): image.Image(self.poolPath).multiMove( -- To view, visit https://gerrit.ovirt.org/65041 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I20bcebeb11f3ccbe10b029a424f4c27a890eea8c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 14

Change in vdsm[master]: tests: Add tests for poll and wait failures
by automation＠ovirt.org 10 Oct '16

10 Oct '16

gerrit-hooks has posted comments on this change. Change subject: tests: Add tests for poll and wait failures ...................................................................... Patch Set 5: * Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/65294 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8674c9c3c2118041c74213cd8ce0d383086d6cbf Gerrit-PatchSet: 5 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: tests: Add tests for poll and wait failures
by automation＠ovirt.org 10 Oct '16

10 Oct '16

gerrit-hooks has posted comments on this change. Change subject: tests: Add tests for poll and wait failures ...................................................................... Patch Set 4: * Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/65294 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8674c9c3c2118041c74213cd8ce0d383086d6cbf Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: tests: Add tests for poll and wait failures
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has posted comments on this change. Change subject: tests: Add tests for poll and wait failures ...................................................................... Patch Set 3: Verified+1 Verified by the tests. -- To view, visit https://gerrit.ovirt.org/65294 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8674c9c3c2118041c74213cd8ce0d383086d6cbf Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: tests: Add tests for poll and wait failures
by automation＠ovirt.org 10 Oct '16

10 Oct '16

gerrit-hooks has posted comments on this change. Change subject: tests: Add tests for poll and wait failures ...................................................................... Patch Set 3: * Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/65294 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8674c9c3c2118041c74213cd8ce0d383086d6cbf Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: hsm:Use sd compat instead of qemu conf compat.
by Maor Lipchuk 10 Oct '16

10 Oct '16

Maor Lipchuk has posted comments on this change. Change subject: hsm:Use sd compat instead of qemu conf compat. ...................................................................... Patch Set 9: Verified+1 -- To view, visit https://gerrit.ovirt.org/64951 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie4b19a99e5d9a73c011bf6d8079e3855298561b9 Gerrit-PatchSet: 9 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Amit Aviram <aaviram(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: hsm:Use sd compat instead of qemu conf compat.
by automation＠ovirt.org 10 Oct '16

10 Oct '16

gerrit-hooks has posted comments on this change. Change subject: hsm:Use sd compat instead of qemu conf compat. ...................................................................... Patch Set 9: * Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/64951 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie4b19a99e5d9a73c011bf6d8079e3855298561b9 Gerrit-PatchSet: 9 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Amit Aviram <aaviram(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: fix abort to raise
by alitke＠redhat.com 10 Oct '16

10 Oct '16

Adam Litke has uploaded a new change for review. Change subject: fix abort to raise ...................................................................... fix abort to raise Change-Id: I5efe2319d023bd813777c76a0bbaa0b9576281f6 Signed-off-by: Adam Litke <alitke(a)redhat.com> --- M lib/vdsm/jobs.py M tests/jobsTests.py 2 files changed, 21 insertions(+), 8 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/47/65147/1 diff --git a/lib/vdsm/jobs.py b/lib/vdsm/jobs.py index 2c6c27c..42f43bc 100644 --- a/lib/vdsm/jobs.py +++ b/lib/vdsm/jobs.py @@ -155,13 +155,24 @@ self._status = STATUS.RUNNING try: self._run() + except exception.ActionStopped: + # Once a job has been aborted we expect _run to raise ActionStopped + with self._status_lock: + if self.status != STATUS.ABORTING: + logging.warning("Unexpected ActionStopped exception in " + "job %r from state %r", + self.id, self.status) + self._status = STATUS.ABORTED + if self.autodelete: + self._autodelete() except Exception as e: with self._status_lock: if self.status == STATUS.ABORTING: + # Other exceptions while in aborting state mean that abort + # was not successful. We should not autodelete because + # there may still be an ongoing operation. logging.exception("Failed to abort job (id=%s desc=%s)", self.id, self.description) - # We do not autodelete failed aborting jobs because it is - # likely that something is still running on the system. else: self._status = STATUS.FAILED logging.exception("Job (id=%s desc=%s) failed", @@ -172,13 +183,12 @@ e = exception.GeneralException(str(e)) self._error = e else: + # We always mark the job done. Even if we were in aborting state + # _run might finish normally before the abort action executes. with self._status_lock: - if self.status == STATUS.ABORTING: - self._status = STATUS.ABORTED - else: - self._status = STATUS.DONE - if self.autodelete: - self._autodelete() + self._status = STATUS.DONE + if self.autodelete: + self._autodelete() def _abort(self): """ @@ -188,6 +198,8 @@ status to change to aborted. - Must raise if the job could not be aborted - Must not raise if the job was aborted + - A successful abort must cause the job's _run method to raise an + ActionStopped exception. """ raise AbortNotSupported() diff --git a/tests/jobsTests.py b/tests/jobsTests.py index 6c6d49a..4c83810 100644 --- a/tests/jobsTests.py +++ b/tests/jobsTests.py @@ -86,6 +86,7 @@ def _run(self): self.event_running.set() self.event_aborted.wait(1) + raise exception.ActionStopped() def _abort(self): self.event_aborted.set() -- To view, visit https://gerrit.ovirt.org/65147 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5efe2319d023bd813777c76a0bbaa0b9576281f6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <alitke(a)redhat.com>

3 5

Change in vdsm[master]: hsm:Use sd compat instead of qemu conf compat.
by automation＠ovirt.org 10 Oct '16

10 Oct '16

gerrit-hooks has posted comments on this change. Change subject: hsm:Use sd compat instead of qemu conf compat. ...................................................................... Patch Set 8: * Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/64951 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: Ie4b19a99e5d9a73c011bf6d8079e3855298561b9 Gerrit-PatchSet: 8 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Amit Aviram <aaviram(a)redhat.com> Gerrit-Reviewer: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[ovirt-4.0]: gluster: set selinux labels while creating bricks
by Ramesh Nachimuthu 10 Oct '16

10 Oct '16

Hello Piotr Kliczewski, Yaniv Bronhaim, Dan Kenigsberg, Francesco Romani, Sahina Bose, I'd like you to do a code review. Please visit https://gerrit.ovirt.org/64296 to review the following change. Change subject: gluster: set selinux labels while creating bricks ...................................................................... gluster: set selinux labels while creating bricks brick should have correct selinux labels on the brick mount points. But it missing in the createBrick vdsm verb. This patch sets the correct selinux lables on brick mount point using 'restorecon' and 'semanage' commands Change-Id: I1ca5fec80831073643635875095b88c1c4c2132e Bug-Url: https://bugzilla.redhat.com/1368474 Signed-off-by: Ramesh Nachimuthu <rnachimu(a)redhat.com> Reviewed-on: https://gerrit.ovirt.org/62773 Continuous-Integration: Jenkins CI Reviewed-by: Yaniv Bronhaim <ybronhei(a)redhat.com> Reviewed-by: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Reviewed-by: Sahina Bose <sabose(a)redhat.com> Reviewed-by: Francesco Romani <fromani(a)redhat.com> Reviewed-by: Dan Kenigsberg <danken(a)redhat.com> --- M vdsm/gluster/exception.py M vdsm/gluster/storagedev.py 2 files changed, 40 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/96/64296/1 diff --git a/vdsm/gluster/exception.py b/vdsm/gluster/exception.py index 1e1b961..369d6dd 100644 --- a/vdsm/gluster/exception.py +++ b/vdsm/gluster/exception.py @@ -477,6 +477,28 @@ message = "vgscan failed" +class GlusterHostFailedToSetSelinuxContext(GlusterHostException): + code = 4420 + + def __init__(self, brickMountPoint=None, rc=0, out=(), err=()): + self.rc = rc + self.out = out + self.err = err + self.message = "Failed to set selinux context on the brick : %s" \ + % brickMountPoint + + +class GlusterHostFailedToRunRestorecon(GlusterHostException): + code = 4421 + + def __init__(self, brickMountPoint=None, rc=0, out=(), err=()): + self.rc = rc + self.out = out + self.err = err + self.message = "Failed to run restorecon on the brick : %s" \ + % brickMountPoint + + # Hook class GlusterHookException(GlusterException): code = 4500 diff --git a/vdsm/gluster/storagedev.py b/vdsm/gluster/storagedev.py index ca1ee01..43f8c5f 100644 --- a/vdsm/gluster/storagedev.py +++ b/vdsm/gluster/storagedev.py @@ -21,6 +21,7 @@ import errno import logging import os +import selinux import blivet import blivet.formats @@ -54,6 +55,9 @@ _vgscanCommandPath = utils.CommandPath("vgscan", "/sbin/vgscan", "/usr/sbin/vgscan",) +_semanageCommandPath = utils.CommandPath("semanage", + "/sbin/semanage", + "/usr/sbin/semanage",) # All size are in MiB unless otherwise specified DEFAULT_CHUNK_SIZE_KB = 256 @@ -313,4 +317,18 @@ raise ge.GlusterHostStorageDeviceVGScanFailedException(rc, out, err) fstab.FsTab().add(thinlv.path, mountPoint, DEFAULT_FS_TYPE, mntOpts=[DEFAULT_MOUNT_OPTIONS]) + + # If selinux is enabled, set correct selinux labels on the brick. + if selinux.is_selinux_enabled(): + rc, out, err = commands.execCmd([_semanageCommandPath.cmd, + 'fcontext', '-a', '-t', + 'glusterd_brick_t', mountPoint]) + if rc: + raise ge.GlusterHostFailedToSetSelinuxContext(mountPoint, rc, + out, err) + try: + selinux.restorecon(mountPoint, recursive=True) + except OSError as e: + errMsg = "[Errno %s] %s: '%s'" % (e.errno, e.strerror, e.filename) + raise ge.GlusterHostFailedToRunRestorecon(mountPoint, err=errMsg) return _getDeviceDict(thinlv) -- To view, visit https://gerrit.ovirt.org/64296 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I1ca5fec80831073643635875095b88c1c4c2132e Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-4.0 Gerrit-Owner: Ramesh N <rnachimu(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: Sahina Bose <sabose(a)redhat.com> Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>

5 8

Change in vdsm[ovirt-4.0]: virt: Make boolean values from boolean migration options
by mzamazal＠redhat.com 10 Oct '16

10 Oct '16

Hello Nir Soffer, Francesco Romani, I'd like you to do a code review. Please visit https://gerrit.ovirt.org/65108 to review the following change. Change subject: virt: Make boolean values from boolean migration options ...................................................................... virt: Make boolean values from boolean migration options `compressed' and `autoConverge' migration options have boolean values in the string form. They must be converted to actual booleans before they are checked. Change-Id: I642eb607785a1b6f877092e187c91b7a065b38e1 Bug-Url: https://bugzilla.redhat.com/1380822 Backport-To: 4.0 Signed-off-by: Milan Zamazal <mzamazal(a)redhat.com> Reviewed-on: https://gerrit.ovirt.org/65007 Continuous-Integration: Jenkins CI Reviewed-by: Nir Soffer <nsoffer(a)redhat.com> Reviewed-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/migration.py 1 file changed, 4 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/08/65108/1 diff --git a/vdsm/virt/migration.py b/vdsm/virt/migration.py index e502756..edd2c79 100644 --- a/vdsm/virt/migration.py +++ b/vdsm/virt/migration.py @@ -100,6 +100,8 @@ self._dstparams = dstparams self._enableGuestEvents = kwargs.get('enableGuestEvents', False) self._machineParams = {} + # TODO: utils.tobool shouldn't be used in this constructor, the + # conversions should be handled properly in the API layer self._tunneled = utils.tobool(tunneled) self._abortOnError = utils.tobool(abortOnError) self._consoleAddress = consoleAddress @@ -110,8 +112,8 @@ kwargs.get('maxBandwidth') or config.getint('vars', 'migration_max_bandwidth') ) - self._autoConverge = autoConverge - self._compressed = compressed + self._autoConverge = utils.tobool(autoConverge) + self._compressed = utils.tobool(compressed) self._incomingLimit = kwargs.get('incomingLimit') self._outgoingLimit = kwargs.get('outgoingLimit') self.status = { -- To view, visit https://gerrit.ovirt.org/65108 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I642eb607785a1b6f877092e187c91b7a065b38e1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-4.0 Gerrit-Owner: Milan Zamazal <mzamazal(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>

4 6

Change in vdsm[ovirt-4.0]: Live Merge: Remove volume run link after live merge
by ahino＠redhat.com 10 Oct '16

10 Oct '16

Hello Adam Litke, Nir Soffer, I'd like you to do a code review. Please visit https://gerrit.ovirt.org/64986 to review the following change. Change subject: Live Merge: Remove volume run link after live merge ...................................................................... Live Merge: Remove volume run link after live merge When deleting a volume while the VM is running, volume teardown doesn't remove the volume run symbolic link: /run/vdsm/storage/sdUUID/volUUID. In patch Iec3b6a (Live Merge: teardown volume on HSM after live merge) we added volume teardown logic that, for block storage it deactivated the volume. In this patch we extend volume teardown logic to unlink volume run link. Note that this change isn't required for file storage as no symbolic links are created. Change-Id: Ib88bf92e702ac6c324b87c9459b01adf165eaca4 Bug-Url: https://bugzilla.redhat.com/1321018 Signed-off-by: Ala Hino <ahino(a)redhat.com> Reviewed-on: https://gerrit.ovirt.org/59725 Reviewed-by: Nir Soffer <nsoffer(a)redhat.com> Reviewed-by: Adam Litke <alitke(a)redhat.com> Continuous-Integration: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/blockSD.py 1 file changed, 15 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/86/64986/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 58d2507..f977150 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -813,6 +813,21 @@ def teardownVolume(self, imgUUID, volUUID): lvm.deactivateLVs(self.sdUUID, [volUUID]) + self.removeVolumeRunLink(imgUUID, volUUID) + + def removeVolumeRunLink(self, imgUUID, volUUID): + """ + Remove /run/vdsm/storage/sdUUID/imgUUID/volUUID + """ + vol_run_link = os.path.join(constants.P_VDSM_STORAGE, + self.sdUUID, imgUUID, volUUID) + self.log.info("Unlinking volme runtime link: %r", vol_run_link) + try: + os.unlink(vol_run_link) + except OSError as e: + if e.error != errno.ENOENT: + raise + self.log.debug("Volume run link %r does not exist", vol_run_link) class BlockStorageDomain(sd.StorageDomain): -- To view, visit https://gerrit.ovirt.org/64986 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib88bf92e702ac6c324b87c9459b01adf165eaca4 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-4.0 Gerrit-Owner: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>

5 11

Change in vdsm[ovirt-4.0]: Live Merge: Teardown volume on HSM after live merge
by ahino＠redhat.com 10 Oct '16

10 Oct '16

Hello Adam Litke, Nir Soffer, I'd like you to do a code review. Please visit https://gerrit.ovirt.org/64985 to review the following change. Change subject: Live Merge: Teardown volume on HSM after live merge ...................................................................... Live Merge: Teardown volume on HSM after live merge If a VM is running on HSM and live merge is performed, the LV isn't deactivated because the deactivation is done when deleting the volume. However, deleting the volume is done on SPM and this means that the LV is not deactivated on the HSM. In this patch, a logic to teardown the volume is added after live merge has completed. Change-Id: Iec3b6adb50293d8c98f5d8726d668eb272d16549 Bug-Url: https://bugzilla.redhat.com/1377849 Signed-off-by: Ala Hino <ahino(a)redhat.com> Reviewed-on: https://gerrit.ovirt.org/64301 Reviewed-by: Nir Soffer <nsoffer(a)redhat.com> Continuous-Integration: Nir Soffer <nsoffer(a)redhat.com> Continuous-Integration: Jenkins CI Reviewed-by: Adam Litke <alitke(a)redhat.com> --- M vdsm/storage/blockSD.py M vdsm/storage/sd.py M vdsm/virt/vm.py 3 files changed, 32 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/85/64985/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index 8de13b1..58d2507 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -799,6 +799,21 @@ if preallocate == sc.SPARSE_VOL and volFormat == sc.RAW_FORMAT: raise se.IncorrectFormat(sc.type2name(volFormat)) + def getVolumeLease(self, imgUUID, volUUID): + """ + Return the volume lease (leasePath, leaseOffset) + """ + if not self.hasVolumeLeases(): + return clusterlock.Lease(None, None, None) + # TODO: use the sanlock specific offset when present + slot = self.produceVolume(imgUUID, volUUID).getMetaOffset() + offset = ((slot + blockVolume.RESERVED_LEASES) * self.logBlkSize * + sd.LEASE_BLOCKS) + return clusterlock.Lease(volUUID, self.getLeasesFilePath(), offset) + + def teardownVolume(self, imgUUID, volUUID): + lvm.deactivateLVs(self.sdUUID, [volUUID]) + class BlockStorageDomain(sd.StorageDomain): manifestClass = BlockStorageDomainManifest diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 2b151a6..d004794 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -487,6 +487,14 @@ if preallocate is not None and preallocate not in sc.VOL_TYPE: raise se.IncorrectType(preallocate) + def teardownVolume(self, imgUUID, volUUID): + """ + Called when a volume is detached from a prepared image during live + merge flow. In this case, the volume will not be torn down when + the image is torn down. + This does nothing, subclass should override this if needed. + """ + class StorageDomain(object): log = logging.getLogger("Storage.StorageDomain") diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index a896f58..22e656c 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -66,6 +66,7 @@ from vdsm.virt.utils import isVdsmImage, cleanup_guest_socket from storage import outOfProcess as oop from storage import sd +from storage import sdc # local imports # In future those should be imported via .. @@ -4936,6 +4937,12 @@ self.drive.imageID, baseVolUUID, topVolInfo['capacity']) + def teardown_top_volume(self): + # TODO move this method to storage public API + sd_manifest = sdc.sdCache.produce_manifest(self.drive.domainID) + sd_manifest.teardownVolume(self.drive.imageID, + self.job['topVolume']) + @utils.traceback() def run(self): self.update_base_size() @@ -4946,6 +4953,8 @@ self.vm._syncVolumeChain(self.drive) if self.doPivot: self.vm.startDisksStatsCollection() + self.vm.enableDriveMonitor() + self.teardown_top_volume() self.success = True self.vm.log.info("Synchronization completed (job %s)", self.job['jobID']) -- To view, visit https://gerrit.ovirt.org/64985 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iec3b6adb50293d8c98f5d8726d668eb272d16549 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: ovirt-4.0 Gerrit-Owner: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>

5 15

Change in vdsm[master]: tests: Use same log format as vdsm
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: tests: Use same log format as vdsm ...................................................................... tests: Use same log format as vdsm Using the same format as in vdsm logs may make it easier to debug failing tests. In particular, the old format was missing the module and line number. Change-Id: Ia8a6c07442d3fb2dd86d08f5c71c3b640bbda43d Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/run_tests_local.sh.in 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/16/64216/1 diff --git a/tests/run_tests_local.sh.in b/tests/run_tests_local.sh.in index a2c38c5..121eb7c 100644 --- a/tests/run_tests_local.sh.in +++ b/tests/run_tests_local.sh.in @@ -5,7 +5,7 @@ export \ LC_ALL=C \ - NOSE_LOGFORMAT='%(asctime)s %(levelname)-7s [%(name)s] (%(threadName)s) %(message)s' \ + NOSE_LOGFORMAT='%(asctime)s %(levelname)-7s [%(name)s] (%(threadName)s) %(message)s (%(module)s:%(lineno)d)' \ NOSE_VERBOSE=${NOSE_VERBOSE:-3} \ PYTHONDONTWRITEBYTECODE=1 \ PYTHONPATH="@top_srcdir@/lib:@top_srcdir@/vdsm:@top_srcdir@/client:@top_srcdir@/vdsm_api:$PYTHONPATH" \ -- To view, visit https://gerrit.ovirt.org/64216 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia8a6c07442d3fb2dd86d08f5c71c3b640bbda43d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

6 12

Change in vdsm[master]: log: Nicer logging format
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: log: Nicer logging format ...................................................................... log: Nicer logging format Use nicer and more standard logging format, similar to engine log format. Having similar logging formats should make it easier to debug and support. Here is an example log using the new format: 2016-09-19 00:02:25,291 INFO [vds] (MainThread) (PID: 10281) I am the actual vdsm 4.18.999-578.git61bfe00.el7 voodoo6.tlv.redhat.com (3.10.0-489.el7.x86_64) 2016-09-19 00:02:25,291 INFO [vds] (MainThread) VDSM will run with cpu affinity: frozenset([1]) 2016-09-19 00:02:25,379 INFO [storage.check] (MainThread) Starting check service 2016-09-19 00:02:25,384 INFO [storage.Dispatcher] (MainThread) Starting StorageDispatcher... 2016-09-19 00:02:25,385 INFO [storage.asyncevent] (check/loop) Starting <EventLoop running=True closed=False at 0x33716496> 2016-09-19 00:02:25,482 INFO [dispatcher] (MainThread) Run and protect: registerDomainStateChangeCallback(callbackFunc=<functools.partial object at 0x288f260>) 2016-09-19 00:02:25,482 INFO [dispatcher] (MainThread) Run and protect: registerDomainStateChangeCallback, Return response: None 2016-09-19 00:02:25,485 INFO [MOM] (MainThread) Preparing MOM interface 2016-09-19 00:02:25,486 INFO [MOM] (MainThread) Using named unix socket /var/run/vdsm/mom-vdsm.sock 2016-09-19 00:02:25,487 INFO [root] (MainThread) Unregistering all secrets 2016-09-19 00:02:25,491 INFO [vds] (MainThread) Setting channels' timeout to 30 seconds. 2016-09-19 00:02:25,493 INFO [vds.MultiProtocolAcceptor] (MainThread) Listening at :::54321 2016-09-19 00:02:25,526 INFO [vds] (clientIFinit) recovery: completed in 0s 2016-09-19 00:02:25,560 WARNING [storage.Multipath] (hsm/init) Map '360014052c81462a280847e8a6e3af8cd' has no slaves 2016-09-19 00:02:25,560 WARNING [storage.Multipath] (hsm/init) Map '36001405292bb96f19064f81b884fa63f' has no slaves 2016-09-19 00:02:27,115 INFO [vds] (BindingXMLRPC) XMLRPC server running 2016-09-19 00:02:27,122 INFO [health] (MainThread) Starting health monitor (interval=60) 2016-09-19 00:02:40,256 INFO [ProtocolDetector.AcceptorImpl] (Reactor thread) Accepting connection from ::1:42888 2016-09-19 00:02:40,270 INFO [ProtocolDetector.Detector] (Reactor thread) Detected protocol xml from ::1:42888 2016-09-19 00:02:40,270 INFO [vds.XMLRPCServer] (BindingXMLRPC) Starting request handler for ::1:42888 2016-09-19 00:02:40,271 INFO [vds.XMLRPCServer] (Thread-12) Request handler for ::1:42888 started 2016-09-19 00:02:40,272 INFO [throttled] (Thread-12) Current getAllVmStats: {} 2016-09-19 00:02:40,272 INFO [vds] (Thread-12) RPC call getAllVmStats finished (code=0) in 0.00 seconds Unfinished: require updating all the tools parsing vdsm log. Change-Id: I640c3c83db3b5818a08b4984d5358b1e27bf0ef3 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M static/etc/vdsm/logger.conf.in 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/64113/1 diff --git a/static/etc/vdsm/logger.conf.in b/static/etc/vdsm/logger.conf.in index 34137ff..820fcb4 100644 --- a/static/etc/vdsm/logger.conf.in +++ b/static/etc/vdsm/logger.conf.in @@ -86,7 +86,7 @@ format: %(message)s [formatter_long] -format: %(threadName)s::%(levelname)s::%(asctime)s::%(module)s::%(lineno)d::%(name)s::(%(funcName)s) %(message)s +format: %(asctime)s %(levelname)-7s [%(name)s] (%(threadName)s) %(message)s [formatter_sysform] format= vdsm %(name)s %(levelname)s %(message)s -- To view, visit https://gerrit.ovirt.org/64113 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I640c3c83db3b5818a08b4984d5358b1e27bf0ef3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

9 25

Change in vdsm[master]: dockerfile: Minimize docker image
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: dockerfile: Minimize docker image ...................................................................... dockerfile: Minimize docker image Cleaning dnf caches remove 145MiB of useless data slowing down pushing and pulling the image: # du -sh /var/cache/* 145M /var/cache/dnf 20K /var/cache/ldconfig 0 /var/cache/libvirt # dnf clean all 72 files removed # du -sh /var/cache/* 12K /var/cache/dnf 20K /var/cache/ldconfig 0 /var/cache/libvirt Change-Id: I5e3d606b20a086ef0d2da443a35182d64c3b880c Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M contrib/Dockerfile.fedora 1 file changed, 2 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/65271/1 diff --git a/contrib/Dockerfile.fedora b/contrib/Dockerfile.fedora index 5eaa12d..6495167 100644 --- a/contrib/Dockerfile.fedora +++ b/contrib/Dockerfile.fedora @@ -41,4 +41,5 @@ which \ && \ easy_install pip && \ - pip install nose==1.3.7 tox==2.1.1 + pip install nose==1.3.7 tox==2.1.1 && \ + dnf clean all -- To view, visit https://gerrit.ovirt.org/65271 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I5e3d606b20a086ef0d2da443a35182d64c3b880c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 10

Change in vdsm[master]: dockerfile: Add Fedora dockerfile
by Nir Soffer 10 Oct '16

10 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: dockerfile: Add Fedora dockerfile ...................................................................... dockerfile: Add Fedora dockerfile The Fedora image should run the the python 3 tests that cannot run on CentOS because of missing packages. We probably want to have a matrix with two builds, one using CentOS image running only python 2 tests, and one using Fedora image running all tests. Change-Id: Ie14526fdda539a53785ec3c91fae23996f7b3bcf Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- A contrib/Dockerfile.fedora 1 file changed, 44 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/70/65270/1 diff --git a/contrib/Dockerfile.fedora b/contrib/Dockerfile.fedora new file mode 100644 index 0000000..5eaa12d --- /dev/null +++ b/contrib/Dockerfile.fedora @@ -0,0 +1,44 @@ +FROM fedora:24 +RUN dnf install -y http://resources.ovirt.org/pub/yum-repo/ovirt-release-master.rpm && \ + dnf update -y && \ + dnf install -y \ + PyYAML \ + autoconf \ + automake \ + bridge-utils \ + dosfstools \ + e2fsprogs \ + genisoimage \ + git \ + libselinux-python \ + libselinux-python3 \ + libvirt-python \ + libvirt-python3 \ + make \ + mom \ + openvswitch \ + psmisc \ + python-cpopen \ + python-decorator \ + python-devel \ + python-inotify \ + python-ioprocess \ + python-netaddr \ + python-pthreading \ + python-setuptools \ + python-six \ + python2-mock \ + python3 \ + python3-decorator \ + python3-netaddr \ + python3-nose \ + python3-six \ + python3-yaml \ + sanlock-python \ + sudo \ + systemd \ + systemd-udev \ + which \ + && \ + easy_install pip && \ + pip install nose==1.3.7 tox==2.1.1 -- To view, visit https://gerrit.ovirt.org/65270 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie14526fdda539a53785ec3c91fae23996f7b3bcf Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 10

Change in vdsm[master]: hooks: avoid redundant temporary list
by Dan Kenigsberg 10 Oct '16

10 Oct '16

Dan Kenigsberg has uploaded a new change for review. Change subject: hooks: avoid redundant temporary list ...................................................................... hooks: avoid redundant temporary list No functional change is expected. Change-Id: Id6b0505ebabef0ec33a738c3698dc446e8849a2a Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M lib/vdsm/hooks.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/73/65273/1 diff --git a/lib/vdsm/hooks.py b/lib/vdsm/hooks.py index 96969fd..777b4ee 100644 --- a/lib/vdsm/hooks.py +++ b/lib/vdsm/hooks.py @@ -453,8 +453,8 @@ def _getHookInfo(dir): - return dict([(os.path.basename(script), _getScriptInfo(script)) - for script in _scriptsPerDir(dir)]) + return dict((os.path.basename(script), _getScriptInfo(script)) + for script in _scriptsPerDir(dir)) def installed(): -- To view, visit https://gerrit.ovirt.org/65273 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id6b0505ebabef0ec33a738c3698dc446e8849a2a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

3 5

Change in vdsm[master]: utils: Fix terminating context manager
by automation＠ovirt.org 10 Oct '16

10 Oct '16

gerrit-hooks has posted comments on this change. Change subject: utils: Fix terminating context manager ...................................................................... Patch Set 2: * Update tracker: IGNORE, no Bug-Url found * Check Bug-Url::WARN, no bug url found, make sure header matches 'Bug-Url: ' and is a valid url. * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/65294 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8674c9c3c2118041c74213cd8ce0d383086d6cbf Gerrit-PatchSet: 2 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: storage: Add inplace virt-sparsify support
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has posted comments on this change. Change subject: storage: Add inplace virt-sparsify support ...................................................................... Patch Set 13: From the discussion with Richard and Kevin, we learned that we can use this only on the top volume in an image. We need to validate that the volume requested by the caller is the leaf volume, and fail the request if not. We also must prevent usage of this verb on a template image used by any other image. Doing this check safely may be hard since it can be done only on the spm. Finally we must protect the image during the operation using volume generation and mark the image as illegal, so another host cannot consume this image while we are sparsifying it. See the latest version of copy_data for the details how it is done. -- To view, visit https://gerrit.ovirt.org/57347 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I6ac2bb1fbd2acbe0fc47694d17313c6ccd01a227 Gerrit-PatchSet: 13 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shmuel Leib Melamud <smelamud(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Maor Lipchuk <mlipchuk(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: Shahar Havivi <shavivi(a)redhat.com> Gerrit-Reviewer: Shmuel Leib Melamud <smelamud(a)redhat.com> Gerrit-Reviewer: Shmuel Melamud <smelamud(a)redhat.com> Gerrit-Reviewer: Yaniv Kaul <ykaul(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: Revert "build tests: Require NOSE 1.3.7 and up for running t...
by edwardh＠redhat.com 09 Oct '16

09 Oct '16

Edward Haas has uploaded a new change for review. Change subject: Revert "build tests: Require NOSE 1.3.7 and up for running tests" ...................................................................... Revert "build tests: Require NOSE 1.3.7 and up for running tests" NOSE is used during the VDSM build ('make rpm') where builders do not use pip, just rpms provided by the standard repos. Therefore, enforcing the nose version of 1.3.7 is not possible until it is updated in the repos of centos7/rhel7. This reverts commit 4e729ddd2b243d0953e2de5d31c42fc59859bf23. Change-Id: I175c0752a5929d1b644cd41cf8d111bcce10f32d Signed-off-by: Edward Haas <edwardh(a)redhat.com> --- M Makefile.am 1 file changed, 1 insertion(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/35/64035/1 diff --git a/Makefile.am b/Makefile.am index 51fe2a5..aefd2b3 100644 --- a/Makefile.am +++ b/Makefile.am @@ -132,12 +132,7 @@ .PHONY: tests tests: - @version=`nosetests --version | cut -d' ' -f3`; \ - if build-aux/vercmp $$version "1.3.7"; then \ - echo "Error: NOSE is too old, please install NOSE 1.3.7 or later"; \ - exit 1; \ - fi - @$(MAKE) -C tests/ check + $(MAKE) -C tests/ check all-local: \ vdsm.spec -- To view, visit https://gerrit.ovirt.org/64035 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I175c0752a5929d1b644cd41cf8d111bcce10f32d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Edward Haas <edwardh(a)redhat.com>

4 9

Change in vdsm[master]: travis: Separate Fedora and Centos builds
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: travis: Separate Fedora and Centos builds ...................................................................... travis: Separate Fedora and Centos builds Add build matrix, creating two sub-build for each build, one using the Fedora image and the other the CentOS image. Change-Id: I0fe524df5f4baf4f3c24f8da56c30d52ee22f8b1 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M .travis.yml 1 file changed, 7 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/84/65284/1 diff --git a/.travis.yml b/.travis.yml index f167e8d..c5d2ec3 100644 --- a/.travis.yml +++ b/.travis.yml @@ -7,11 +7,14 @@ git: depth: 9999999 +env: + matrix: + - DOCKER_IMAGE=nirs/vdsm-centos + - DOCKER_IMAGE=nirs/vdsm-fedora + before_install: - docker --version - - docker pull nirs/vdsm-centos - - docker pull nirs/vdsm-fedora + - docker pull $DOCKER_IMAGE script: - - docker run --privileged --rm -it -v `pwd`:/vdsm:Z nirs/vdsm-centos bash -c "cd /vdsm && ./autogen.sh --system && make && make check" - - docker run --privileged --rm -it -v `pwd`:/vdsm:Z nirs/vdsm-fedora bash -c "cd /vdsm && ./autogen.sh --system && make && make check" + - docker run --privileged --rm -it -v `pwd`:/vdsm:Z $DOCKER_IMAGE bash -c "cd /vdsm && ./autogen.sh --system && make && make check" -- To view, visit https://gerrit.ovirt.org/65284 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0fe524df5f4baf4f3c24f8da56c30d52ee22f8b1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>

2 3

Change in vdsm[master]: dokerfile: Rename Dockerfile to Dockerfile.centos
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: dokerfile: Rename Dockerfile to Dockerfile.centos ...................................................................... dokerfile: Rename Dockerfile to Dockerfile.centos We need another dockerfile for Fedora. Change-Id: Ibd8245d5a5f7eabe01b8f33ab1e4210d3edf3777 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- R contrib/Dockerfile.centos 1 file changed, 0 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/69/65269/1 diff --git a/contrib/Dockerfile b/contrib/Dockerfile.centos similarity index 100% rename from contrib/Dockerfile rename to contrib/Dockerfile.centos -- To view, visit https://gerrit.ovirt.org/65269 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibd8245d5a5f7eabe01b8f33ab1e4210d3edf3777 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 5

Change in vdsm[master]: readme: Covert to markdown format
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: readme: Covert to markdown format ...................................................................... readme: Covert to markdown format Markdown is nicer to work with and render nicer in github. Change-Id: I74ce4bcb8bb276e98fc6466ace7709733dd809c1 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M Makefile.am R README.md M vdsm.spec.in 3 files changed, 26 insertions(+), 25 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/60/65260/1 diff --git a/Makefile.am b/Makefile.am index 64cd9a0..789221c 100644 --- a/Makefile.am +++ b/Makefile.am @@ -19,6 +19,9 @@ # TOX_MIN_VERSION = 2.1.1 +# We don't want the standard gnu file setup (README, INSTALL, etc.) +AUTOMAKE_OPTIONS = foreign + SUBDIRS = \ client \ contrib \ @@ -42,6 +45,7 @@ EXTRA_DIST = \ .gitignore \ README.logging \ + README.md \ autobuild.sh \ autogen.sh \ build-aux/pkg-version \ diff --git a/README b/README.md similarity index 68% rename from README rename to README.md index 955dfa9..9398827 100644 --- a/README +++ b/README.md @@ -1,5 +1,4 @@ - Vdsm: Virtual Desktop Server Manager - ==================================== +# Vdsm: Virtual Desktop Server Manager The Vdsm service exposes an API for managing virtualization hosts running the KVM hypervisor technology. Vdsm manages and monitors @@ -7,53 +6,53 @@ creation, other host administration tasks, statistics gathering, and log collection. -Installation -============ + +## Installation The Vdsm service can be used by following the standard autotools installation process, documented in the INSTALL file. As a quick start you can do - ./configure --prefix=/usr --sysconfdir=/etc \ + ./configure --prefix=/usr --sysconfdir=/etc \ --localstatedir=/var --libdir=/usr/lib - make - sudo make install + make + sudo make install In order to start vdsm at first try, please perform: - vdsm-tool configure [--force] + vdsm-tool configure [--force] - * --force flag will override old conf files with vdsm defaults and restart - services that were configured (if were already running) +--force flag will override old conf files with vdsm defaults and restart +services that were configured (if were already running) -Packaging -========= + +## Packaging The 'vdsm.spec' file demonstrates how to distribute Vdsm as an RPM package. -Getting Help -============ + +## Getting Help There are two mailing lists for discussions: - - For technical discussions about the project and its code base. +- For technical discussions about the project and its code base. - http://lists.ovirt.org/mailman/listinfo/devel + http://lists.ovirt.org/mailman/listinfo/devel - - For questions by users, who do not want to be swamped by - technicalities. +- For questions by users, who do not want to be swamped by + technicalities. - http://lists.ovirt.org/mailman/listinfo/users + http://lists.ovirt.org/mailman/listinfo/users The developers also hang out on IRC at #vdsm hosted on freenode.net The latest upstream code can be obtained from GIT: - git clone https://gerrit.ovirt.org/vdsm + git clone https://gerrit.ovirt.org/vdsm -Licensing -========= + +## Licensing Vdsm is provided under the terms of the GNU General Public License, version 2 or later. Please see the COPYING file for complete GPLv2+ @@ -71,5 +70,3 @@ If you modify this program, you may extend this exception to your version, but you are not obligated to do so. If you do not wish to do so, delete this exception statement from your version. - --- End of readme diff --git a/vdsm.spec.in b/vdsm.spec.in index 2842afa..b91ad3b 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -895,7 +895,7 @@ exit 0 %files -%doc README +%doc README.md %doc lib/vdsm/vdsm.conf.sample %doc README.logging %license COPYING -- To view, visit https://gerrit.ovirt.org/65260 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I74ce4bcb8bb276e98fc6466ace7709733dd809c1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 13

Change in vdsm[master]: qemuimg: Add wrapper to execute command
by ahino＠redhat.com 09 Oct '16

09 Oct '16

Ala Hino has uploaded a new change for review. Change subject: qemuimg: Add wrapper to execute command ...................................................................... qemuimg: Add wrapper to execute command Add wrapper to execute qemuimg command and raise QImgError if error encountered. Change-Id: I3483a202f9543098ad75e8be8bf4b398d26c8e4a Signed-off-by: Ala Hino <ahino(a)redhat.com> --- M lib/vdsm/qemuimg.py 1 file changed, 11 insertions(+), 16 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/78/65178/1 diff --git a/lib/vdsm/qemuimg.py b/lib/vdsm/qemuimg.py index ef98149..c9e72ec 100644 --- a/lib/vdsm/qemuimg.py +++ b/lib/vdsm/qemuimg.py @@ -71,9 +71,7 @@ cmd.extend(("-f", format)) cmd.append(image) - rc, out, err = commands.execCmd(cmd, raw=True) - if rc != 0: - raise QImgError(rc, out, err) + out = _run_cmd(cmd) try: qemu_info = _parse_qemuimg_json(out) @@ -125,10 +123,7 @@ if size is not None: cmd.append(str(size)) - rc, out, err = commands.execCmd(cmd, cwd=cwdPath) - - if rc != 0: - raise QImgError(rc, out, err) + _run_cmd(cmd, cwd=cwdPath) def check(image, format=None): @@ -138,11 +133,7 @@ cmd.extend(("-f", format)) cmd.append(image) - rc, out, err = commands.execCmd(cmd, raw=True) - - # FIXME: handle different error codes and raise errors accordingly - if rc != 0: - raise QImgError(rc, out, err) + out = _run_cmd(cmd) try: qemu_check = _parse_qemuimg_json(out) @@ -300,10 +291,7 @@ cmd.extend(("-f", format)) cmd.extend((image, str(newSize))) - rc, out, err = commands.execCmd(cmd) - - if rc != 0: - raise QImgError(rc, out, err) + _run_cmd(cmd) def rebase(image, backing, format=None, backingFormat=None, unsafe=False, @@ -351,3 +339,10 @@ if value not in _QCOW2_COMPAT_SUPPORTED: raise ValueError("Invalid compat version %r" % value) return value + + +def _run_cmd(cmd, cwd=None): + rc, out, err = commands.execCmd(cmd, raw=True, cwd=cwd) + if rc != 0: + raise QImgError(rc, out, err) + return out -- To view, visit https://gerrit.ovirt.org/65178 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I3483a202f9543098ad75e8be8bf4b398d26c8e4a Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ala Hino <ahino(a)redhat.com>

3 15

Change in vdsm[master]: qemuimg: Introduce InvalidOutput exception
by ahino＠redhat.com 09 Oct '16

09 Oct '16

Ala Hino has uploaded a new change for review. Change subject: qemuimg: Introduce InvalidOutput exception ...................................................................... qemuimg: Introduce InvalidOutput exception Raised when the command output is not valid. Change-Id: If3f801f05f130c2417e4946877e31030260269a1 Signed-off-by: Ala Hino <ahino(a)redhat.com> --- M lib/vdsm/qemuimg.py 1 file changed, 20 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/08/65208/1 diff --git a/lib/vdsm/qemuimg.py b/lib/vdsm/qemuimg.py index 28cddee..8f6dc35 100644 --- a/lib/vdsm/qemuimg.py +++ b/lib/vdsm/qemuimg.py @@ -65,6 +65,21 @@ self.cmd, self.ecode, self.stdout, self.stderr, self.message) +class InvalidOutput(QImgError): + """ + Raised when the command output is not valid. + """ + + def __init__(self, cmd, stdout, message): + self.cmd = cmd + self.stdout = stdout + self.message = message + + def __str__(self): + return "cmd=%s, stdout=%s, message=%s" % ( + self.cmd, self.stdout, self.message) + + def info(image, format=None): cmd = [_qemuimg.cmd, "info", "--output", "json"] @@ -79,7 +94,7 @@ try: qemu_info = _parse_qemuimg_json(out) except ValueError: - raise QImgError(cmd, rc, out, err, "Failed to process qemu-img output") + raise InvalidOutput(cmd, out, "Failed to process qemu-img output") try: info = { @@ -87,7 +102,7 @@ 'virtualsize': qemu_info['virtual-size'], } except KeyError as key: - raise QImgError(cmd, rc, out, err, "Missing field: %r" % key) + raise InvalidOutput(cmd, out, "Missing field: %r" % key) if 'cluster-size' in qemu_info: info['clustersize'] = qemu_info['cluster-size'] @@ -97,7 +112,7 @@ try: info['compat'] = qemu_info['format-specific']['data']['compat'] except KeyError: - raise QImgError(cmd, rc, out, err, "'compat' expected but not found") + raise InvalidOutput(cmd, out, "'compat' expected but not found") return info @@ -148,11 +163,11 @@ try: qemu_check = _parse_qemuimg_json(out) except ValueError: - raise QImgError(cmd, rc, out, err, "Failed to process qemu-img output") + raise InvalidOutput(cmd, out, "Failed to process qemu-img output") try: return {"offset": qemu_check["image-end-offset"]} except KeyError: - raise QImgError(cmd, rc, out, err, "unable to parse qemu-img check output") + raise InvalidOutput(cmd, out, "unable to parse qemu-img check output") def convert(srcImage, dstImage, srcFormat=None, dstFormat=None, -- To view, visit https://gerrit.ovirt.org/65208 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If3f801f05f130c2417e4946877e31030260269a1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ala Hino <ahino(a)redhat.com>

3 21

Change in vdsm[master]: qemuimg: Add the command to QImgError
by ahino＠redhat.com 09 Oct '16

09 Oct '16

Ala Hino has uploaded a new change for review. Change subject: qemuimg: Add the command to QImgError ...................................................................... qemuimg: Add the command to QImgError Change-Id: I226a54b29ae4afc3854056efea78767383b89619 Signed-off-by: Ala Hino <ahino(a)redhat.com> --- M lib/vdsm/qemuimg.py 1 file changed, 5 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/79/65179/1 diff --git a/lib/vdsm/qemuimg.py b/lib/vdsm/qemuimg.py index c9e72ec..26d0ec1 100644 --- a/lib/vdsm/qemuimg.py +++ b/lib/vdsm/qemuimg.py @@ -53,15 +53,16 @@ class QImgError(Exception): - def __init__(self, ecode, stdout, stderr, message=None): + def __init__(self, cmd, ecode, stdout, stderr, message=None): + self.cmd = cmd self.ecode = ecode self.stdout = stdout self.stderr = stderr self.message = message def __str__(self): - return "ecode=%s, stdout=%s, stderr=%s, message=%s" % ( - self.ecode, self.stdout, self.stderr, self.message) + return "cmd=%s, ecode=%s, stdout=%s, stderr=%s, message=%s" % ( + self.cmd, self.ecode, self.stdout, self.stderr, self.message) def info(image, format=None): @@ -344,5 +345,5 @@ def _run_cmd(cmd, cwd=None): rc, out, err = commands.execCmd(cmd, raw=True, cwd=cwd) if rc != 0: - raise QImgError(rc, out, err) + raise QImgError(cmd, rc, out, err) return out -- To view, visit https://gerrit.ovirt.org/65179 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I226a54b29ae4afc3854056efea78767383b89619 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ala Hino <ahino(a)redhat.com>

3 21

Change in vdsm[master]: procwatch: Move CommandStream to new module
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: procwatch: Move CommandStream to new module ...................................................................... procwatch: Move CommandStream to new module Move CommandStream from the utils junkyard to the new procwatch module and the tests to a new test module, with minimal changes to keep it working. Change-Id: I045569d350aad19049a33338a72b59e68ba91e5b Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- A lib/vdsm/procwatch.py M lib/vdsm/qemuimg.py M lib/vdsm/utils.py M tests/Makefile.am A tests/procwatch_test.py M tests/utilsTests.py M vdsm.spec.in 7 files changed, 221 insertions(+), 158 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/65072/1 diff --git a/lib/vdsm/procwatch.py b/lib/vdsm/procwatch.py new file mode 100644 index 0000000..1ae168a --- /dev/null +++ b/lib/vdsm/procwatch.py @@ -0,0 +1,85 @@ +# +# Copyright 2014-2016 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +from __future__ import absolute_import + +import io +import os +import select + +from vdsm import utils + + +class CommandStream(object): + + def __init__(self, command, stdoutcb, stderrcb): + self._command = command + self._poll = select.epoll() + self._iocb = {} + + # In case both stderr and stdout are using the same fd the + # output is squashed to the stdout (given the order of the + # entries in the dictionary) + self._iocb[self._command.stderr.fileno()] = stderrcb + self._iocb[self._command.stdout.fileno()] = stdoutcb + + for fd in self._iocb: + self._poll.register(fd, select.EPOLLIN) + + def _poll_input(self, fileno): + self._iocb[fileno](os.read(fileno, io.DEFAULT_BUFFER_SIZE)) + + def _poll_event(self, fileno): + self._poll.unregister(fileno) + del self._iocb[fileno] + + def _poll_timeout(self, timeout): + # TODO: Kill NoIntrPoll, stopping polling on EINTR and procesing events + # is good enough, we don't need to check timeout here, caller is + # checking if timeout has expired. + fdevents = utils.NoIntrPoll(self._poll.poll, timeout) + + for fileno, event in fdevents: + if event & select.EPOLLIN: + self._poll_input(fileno) + elif event & (select.EPOLLHUP | select.EPOLLERR): + self._poll_event(fileno) + + @property + def closed(self): + return len(self._iocb) == 0 + + def receive(self, timeout=None): + """ + Receiving data from the command can raise OSError + exceptions as described in read(2). + """ + if timeout is None: + poll_remaining = -1 + else: + endtime = utils.monotonic_time() + timeout + + while not self.closed: + if timeout is not None: + poll_remaining = endtime - utils.monotonic_time() + if poll_remaining <= 0: + break + + self._poll_timeout(poll_remaining) diff --git a/lib/vdsm/qemuimg.py b/lib/vdsm/qemuimg.py index 8eef8fa..8d60e4c 100644 --- a/lib/vdsm/qemuimg.py +++ b/lib/vdsm/qemuimg.py @@ -29,6 +29,7 @@ from . import utils from . import cmdutils from . import commands +from . import procwatch from .compat import CPopen from . config import config @@ -202,7 +203,7 @@ with_ioclass=utils.IOCLASS.IDLE) _log.debug(cmdutils.command_log_line(cmd, cwd=cwd)) self._command = CPopen(cmd, cwd=cwd, deathSignal=signal.SIGKILL) - self._stream = utils.CommandStream( + self._stream = procwatch.CommandStream( self._command, self._recvstdout, self._recvstderr) def _recvstderr(self, buffer): diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py index a428c8e..5d1d822 100644 --- a/lib/vdsm/utils.py +++ b/lib/vdsm/utils.py @@ -280,60 +280,6 @@ timeout = max(0, endtime - monotonic_time()) -class CommandStream(object): - def __init__(self, command, stdoutcb, stderrcb): - self._command = command - self._poll = select.epoll() - self._iocb = {} - - # In case both stderr and stdout are using the same fd the - # output is squashed to the stdout (given the order of the - # entries in the dictionary) - self._iocb[self._command.stderr.fileno()] = stderrcb - self._iocb[self._command.stdout.fileno()] = stdoutcb - - for fd in self._iocb: - self._poll.register(fd, select.EPOLLIN) - - def _poll_input(self, fileno): - self._iocb[fileno](os.read(fileno, io.DEFAULT_BUFFER_SIZE)) - - def _poll_event(self, fileno): - self._poll.unregister(fileno) - del self._iocb[fileno] - - def _poll_timeout(self, timeout): - fdevents = NoIntrPoll(self._poll.poll, timeout) - - for fileno, event in fdevents: - if event & select.EPOLLIN: - self._poll_input(fileno) - elif event & (select.EPOLLHUP | select.EPOLLERR): - self._poll_event(fileno) - - @property - def closed(self): - return len(self._iocb) == 0 - - def receive(self, timeout=None): - """ - Receiving data from the command can raise OSError - exceptions as described in read(2). - """ - if timeout is None: - poll_remaining = -1 - else: - endtime = monotonic_time() + timeout - - while not self.closed: - if timeout is not None: - poll_remaining = endtime - monotonic_time() - if poll_remaining <= 0: - break - - self._poll_timeout(poll_remaining) - - def stripNewLines(lines): return [l[:-1] if l.endswith('\n') else l for l in lines] diff --git a/tests/Makefile.am b/tests/Makefile.am index dcb36f8..faa1df4 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -95,6 +95,7 @@ periodicTests.py \ permutationTests.py \ persistentDictTests.py \ + procwatch_test.py \ properties_test.py \ protocoldetectorTests.py \ pthreadTests.py \ diff --git a/tests/procwatch_test.py b/tests/procwatch_test.py new file mode 100644 index 0000000..e6b9d7f --- /dev/null +++ b/tests/procwatch_test.py @@ -0,0 +1,132 @@ +# +# Copyright 2014-2016 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +from __future__ import absolute_import +import signal +import operator + +from testlib import VdsmTestCase +from testlib import expandPermutations, permutations + +from vdsm import compat +from vdsm import procwatch + + +@expandPermutations +class CommandStreamTests(VdsmTestCase): + + def assertUnexpectedCall(self, data): + raise AssertionError("Unexpected data: %r" % data) + + def _startCommand(self, command): + return compat.CPopen(command) + + @permutations([ + (['echo', '-n', '%s'], True, False), + (['sh', '-c', 'echo -n "%s" >&2'], False, True), + ]) + def test_receive(self, cmd, recv_out, recv_err): + text = bytes('Hello World') + received = bytearray() + + def recv_data(buffer): + # cannot use received += buffer with a variable + # defined in the parent function. + operator.iadd(received, buffer) + + cmd[-1] = cmd[-1] % text + + c = self._startCommand(cmd) + p = procwatch.CommandStream( + c, + recv_data if recv_out else self.assertUnexpectedCall, + recv_data if recv_err else self.assertUnexpectedCall) + + while not p.closed: + p.receive() + + retcode = c.wait() + + self.assertEqual(retcode, 0) + self.assertEqual(text, received) + + @permutations([ + (['cat'], True, False), + (['sh', '-c', 'cat >&2'], False, True), + ]) + def test_write(self, cmd, recv_out, recv_err): + text = bytes('Hello World') + received = bytearray() + + def recv_data(buffer): + # cannot use received += buffer with a variable + # defined in the parent function. + operator.iadd(received, buffer) + + c = self._startCommand(cmd) + p = procwatch.CommandStream( + c, + recv_data if recv_out else self.assertUnexpectedCall, + recv_data if recv_err else self.assertUnexpectedCall) + + c.stdin.write(text) + c.stdin.flush() + c.stdin.close() + + while not p.closed: + p.receive() + + retcode = c.wait() + + self.assertEqual(retcode, 0) + self.assertEqual(text, str(received)) + + def test_timeout(self): + c = self._startCommand(["sleep", "5"]) + p = procwatch.CommandStream(c, self.assertUnexpectedCall, + self.assertUnexpectedCall) + + with self.assertElapsed(2): + p.receive(2) + + self.assertEqual(p.closed, False) + + c.terminate() + + self.assertEqual(c.wait(), -signal.SIGTERM) + + @permutations(( + ('kill', -signal.SIGKILL), + ('terminate', -signal.SIGTERM), + )) + def test_signals(self, method, expected_retcode): + c = self._startCommand(["sleep", "2"]) + p = procwatch.CommandStream(c, self.assertUnexpectedCall, + self.assertUnexpectedCall) + + getattr(c, method)() + + try: + with self.assertElapsed(0): + p.receive(2) + finally: + retcode = c.wait() + + self.assertEqual(retcode, expected_retcode) diff --git a/tests/utilsTests.py b/tests/utilsTests.py index cdbfd7b..5aa2c41 100644 --- a/tests/utilsTests.py +++ b/tests/utilsTests.py @@ -774,109 +774,6 @@ self.assertEqual(utils.round(n, size), result) -@expandPermutations -class CommandStreamTests(TestCaseBase): - - def assertUnexpectedCall(self, data): - raise AssertionError("Unexpected data: %r" % data) - - def _startCommand(self, command): - return cpopen.CPopen(command) - - @permutations([ - (['echo', '-n', '%s'], True, False), - (['sh', '-c', 'echo -n "%s" >&2'], False, True), - ]) - def test_receive(self, cmd, recv_out, recv_err): - text = bytes('Hello World') - received = bytearray() - - def recv_data(buffer): - # cannot use received += buffer with a variable - # defined in the parent function. - operator.iadd(received, buffer) - - cmd[-1] = cmd[-1] % text - - c = self._startCommand(cmd) - p = utils.CommandStream( - c, - recv_data if recv_out else self.assertUnexpectedCall, - recv_data if recv_err else self.assertUnexpectedCall) - - while not p.closed: - p.receive() - - retcode = c.wait() - - self.assertEqual(retcode, 0) - self.assertEqual(text, received) - - @permutations([ - (['cat'], True, False), - (['sh', '-c', 'cat >&2'], False, True), - ]) - def test_write(self, cmd, recv_out, recv_err): - text = bytes('Hello World') - received = bytearray() - - def recv_data(buffer): - # cannot use received += buffer with a variable - # defined in the parent function. - operator.iadd(received, buffer) - - c = self._startCommand(cmd) - p = utils.CommandStream( - c, - recv_data if recv_out else self.assertUnexpectedCall, - recv_data if recv_err else self.assertUnexpectedCall) - - c.stdin.write(text) - c.stdin.flush() - c.stdin.close() - - while not p.closed: - p.receive() - - retcode = c.wait() - - self.assertEqual(retcode, 0) - self.assertEqual(text, str(received)) - - def test_timeout(self): - c = self._startCommand(["sleep", "5"]) - p = utils.CommandStream(c, self.assertUnexpectedCall, - self.assertUnexpectedCall) - - with self.assertElapsed(2): - p.receive(2) - - self.assertEqual(p.closed, False) - - c.terminate() - - self.assertEqual(c.wait(), -signal.SIGTERM) - - @permutations(( - ('kill', -signal.SIGKILL), - ('terminate', -signal.SIGTERM), - )) - def test_signals(self, method, expected_retcode): - c = self._startCommand(["sleep", "2"]) - p = utils.CommandStream(c, self.assertUnexpectedCall, - self.assertUnexpectedCall) - - getattr(c, method)() - - try: - with self.assertElapsed(0): - p.receive(2) - finally: - retcode = c.wait() - - self.assertEqual(retcode, expected_retcode) - - class FakeLogger(object): def __init__(self, level): diff --git a/vdsm.spec.in b/vdsm.spec.in index 2842afa..5a54307 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -1262,6 +1262,7 @@ %{python_sitelib}/%{vdsm_name}/storage/threadlocal.py* %{python_sitelib}/%{vdsm_name}/storage/volumemetadata.py* %{python_sitelib}/%{vdsm_name}/storage/workarounds.py* +%{python_sitelib}/%{vdsm_name}/procwatch.py* %{python_sitelib}/%{vdsm_name}/properties.py* %{python_sitelib}/%{vdsm_name}/protocoldetector.py* %{python_sitelib}/%{vdsm_name}/pthread.py* -- To view, visit https://gerrit.ovirt.org/65072 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I045569d350aad19049a33338a72b59e68ba91e5b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 7

Change in vdsm[master]: procwatch: Rename CommandStream to ProcessWatcher
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: procwatch: Rename CommandStream to ProcessWatcher ...................................................................... procwatch: Rename CommandStream to ProcessWatcher This class is about watching child process standard streams, and invoking callbacks when data is read from one of the streams. Change-Id: I13981d80786bda2f185851febfd40754a38282a8 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M lib/vdsm/procwatch.py M lib/vdsm/qemuimg.py M tests/procwatch_test.py 3 files changed, 18 insertions(+), 18 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/73/65073/1 diff --git a/lib/vdsm/procwatch.py b/lib/vdsm/procwatch.py index 1ae168a..fd4b543 100644 --- a/lib/vdsm/procwatch.py +++ b/lib/vdsm/procwatch.py @@ -27,7 +27,7 @@ from vdsm import utils -class CommandStream(object): +class ProcessWatcher(object): def __init__(self, command, stdoutcb, stderrcb): self._command = command diff --git a/lib/vdsm/qemuimg.py b/lib/vdsm/qemuimg.py index 8d60e4c..0527a15 100644 --- a/lib/vdsm/qemuimg.py +++ b/lib/vdsm/qemuimg.py @@ -203,7 +203,7 @@ with_ioclass=utils.IOCLASS.IDLE) _log.debug(cmdutils.command_log_line(cmd, cwd=cwd)) self._command = CPopen(cmd, cwd=cwd, deathSignal=signal.SIGKILL) - self._stream = procwatch.CommandStream( + self._watcher = procwatch.ProcessWatcher( self._command, self._recvstdout, self._recvstderr) def _recvstderr(self, buffer): @@ -247,9 +247,9 @@ return self._command.poll() is not None def poll(self, timeout=None): - self._stream.receive(timeout=timeout) + self._watcher.receive(timeout=timeout) - if not self._stream.closed: + if not self._watcher.closed: return self._command.wait() diff --git a/tests/procwatch_test.py b/tests/procwatch_test.py index e6b9d7f..8ed9d9f 100644 --- a/tests/procwatch_test.py +++ b/tests/procwatch_test.py @@ -30,7 +30,7 @@ @expandPermutations -class CommandStreamTests(VdsmTestCase): +class ProcessWatcherTests(VdsmTestCase): def assertUnexpectedCall(self, data): raise AssertionError("Unexpected data: %r" % data) @@ -54,13 +54,13 @@ cmd[-1] = cmd[-1] % text c = self._startCommand(cmd) - p = procwatch.CommandStream( + watcher = procwatch.ProcessWatcher( c, recv_data if recv_out else self.assertUnexpectedCall, recv_data if recv_err else self.assertUnexpectedCall) - while not p.closed: - p.receive() + while not watcher.closed: + watcher.receive() retcode = c.wait() @@ -81,7 +81,7 @@ operator.iadd(received, buffer) c = self._startCommand(cmd) - p = procwatch.CommandStream( + watcher = procwatch.ProcessWatcher( c, recv_data if recv_out else self.assertUnexpectedCall, recv_data if recv_err else self.assertUnexpectedCall) @@ -90,8 +90,8 @@ c.stdin.flush() c.stdin.close() - while not p.closed: - p.receive() + while not watcher.closed: + watcher.receive() retcode = c.wait() @@ -100,13 +100,13 @@ def test_timeout(self): c = self._startCommand(["sleep", "5"]) - p = procwatch.CommandStream(c, self.assertUnexpectedCall, - self.assertUnexpectedCall) + watcher = procwatch.ProcessWatcher(c, self.assertUnexpectedCall, + self.assertUnexpectedCall) with self.assertElapsed(2): - p.receive(2) + watcher.receive(2) - self.assertEqual(p.closed, False) + self.assertEqual(watcher.closed, False) c.terminate() @@ -118,14 +118,14 @@ )) def test_signals(self, method, expected_retcode): c = self._startCommand(["sleep", "2"]) - p = procwatch.CommandStream(c, self.assertUnexpectedCall, - self.assertUnexpectedCall) + watcher = procwatch.ProcessWatcher(c, self.assertUnexpectedCall, + self.assertUnexpectedCall) getattr(c, method)() try: with self.assertElapsed(0): - p.receive(2) + watcher.receive(2) finally: retcode = c.wait() -- To view, visit https://gerrit.ovirt.org/65073 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I13981d80786bda2f185851febfd40754a38282a8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 7

Change in vdsm[master]: procwatch: There is no need for _private helpers
by Nir Soffer 09 Oct '16

09 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: procwatch: There is no need for _private helpers ...................................................................... procwatch: There is no need for _private helpers In the context of tests classes, there is not need for private helpers. The only caller is the test framework, and it calls only the test functions. Change-Id: I54c7acf9c599716527ae3b4919fc19bdbe7de5c1 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/procwatch_test.py 1 file changed, 5 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/74/65074/1 diff --git a/tests/procwatch_test.py b/tests/procwatch_test.py index 8ed9d9f..1c6b365 100644 --- a/tests/procwatch_test.py +++ b/tests/procwatch_test.py @@ -35,7 +35,7 @@ def assertUnexpectedCall(self, data): raise AssertionError("Unexpected data: %r" % data) - def _startCommand(self, command): + def startCommand(self, command): return compat.CPopen(command) @permutations([ @@ -53,7 +53,7 @@ cmd[-1] = cmd[-1] % text - c = self._startCommand(cmd) + c = self.startCommand(cmd) watcher = procwatch.ProcessWatcher( c, recv_data if recv_out else self.assertUnexpectedCall, @@ -80,7 +80,7 @@ # defined in the parent function. operator.iadd(received, buffer) - c = self._startCommand(cmd) + c = self.startCommand(cmd) watcher = procwatch.ProcessWatcher( c, recv_data if recv_out else self.assertUnexpectedCall, @@ -99,7 +99,7 @@ self.assertEqual(text, str(received)) def test_timeout(self): - c = self._startCommand(["sleep", "5"]) + c = self.startCommand(["sleep", "5"]) watcher = procwatch.ProcessWatcher(c, self.assertUnexpectedCall, self.assertUnexpectedCall) @@ -117,7 +117,7 @@ ('terminate', -signal.SIGTERM), )) def test_signals(self, method, expected_retcode): - c = self._startCommand(["sleep", "2"]) + c = self.startCommand(["sleep", "2"]) watcher = procwatch.ProcessWatcher(c, self.assertUnexpectedCall, self.assertUnexpectedCall) -- To view, visit https://gerrit.ovirt.org/65074 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I54c7acf9c599716527ae3b4919fc19bdbe7de5c1 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 6

Change in vdsm[master]: py3: make network/conf_test pass
by Dan Kenigsberg 09 Oct '16

09 Oct '16

Dan Kenigsberg has uploaded a new change for review. Change subject: py3: make network/conf_test pass ...................................................................... py3: make network/conf_test pass re.sub should not mix strings and byte arrays. Change-Id: Idcab2d6f1f88b04d849156915a111189c4cc8589 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M tests/Makefile.am M tests/network/conf_test.py 2 files changed, 2 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/28/64928/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 132be62..fb3d7c3 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -191,7 +191,6 @@ momTests.py \ monkeypatchTests.py \ mountTests.py \ - network/conf_test.py \ network/hook_ovs_test.py \ network/iproute2_test.py \ network/ipwrapper_test.py \ diff --git a/tests/network/conf_test.py b/tests/network/conf_test.py index d94b5e6..c230a3c 100644 --- a/tests/network/conf_test.py +++ b/tests/network/conf_test.py @@ -73,8 +73,8 @@ aXml = ET.tostring(ET.fromstring(a)) bXml = ET.tostring(ET.fromstring(b)) - aXmlNrml = re.sub('>\s*\n\s*<', '><', aXml).strip() - bXmlNrml = re.sub('>\s*\n\s*<', '><', bXml).strip() + aXmlNrml = re.sub(b'>\s*\n\s*<', b'><', aXml).strip() + bXmlNrml = re.sub(b'>\s*\n\s*<', b'><', bXml).strip() self.assertEqual(aXmlNrml, bXmlNrml, msg) -- To view, visit https://gerrit.ovirt.org/64928 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idcab2d6f1f88b04d849156915a111189c4cc8589 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

2 5

Change in vdsm[master]: net libvirt: make network/libvirt.py suitable for OVS
by phoracek＠redhat.com 09 Oct '16

09 Oct '16

Petr Horáček has uploaded a new change for review. Change subject: net libvirt: make network/libvirt.py suitable for OVS ...................................................................... net libvirt: make network/libvirt.py suitable for OVS Make network/libvirt.py suitable for OVS (add support for vlan and virtualport). Change-Id: I6de49e456d8c49f0309690a7fc2bee995f9846bb Signed-off-by: Petr Horáček <phoracek(a)redhat.com> --- M lib/vdsm/network/configurators/ifcfg.py M lib/vdsm/network/configurators/iproute2.py M lib/vdsm/network/configurators/pyroute_two.py M lib/vdsm/network/legacy_switch.py M lib/vdsm/network/libvirt.py M lib/vdsm/network/netinfo/cache.py M lib/vdsm/network/netswitch.py M tests/network/libvirt_test.py M vdsm_hooks/ovs/ovs_before_network_setup_libvirt.py 9 files changed, 141 insertions(+), 31 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/65/65065/4 diff --git a/lib/vdsm/network/configurators/ifcfg.py b/lib/vdsm/network/configurators/ifcfg.py index aac39d1..b9d093f 100644 --- a/lib/vdsm/network/configurators/ifcfg.py +++ b/lib/vdsm/network/configurators/ifcfg.py @@ -370,7 +370,9 @@ def createLibvirtNetwork(self, network, bridged=True, iface=None, skipBackup=False): - netXml = libvirt.createNetworkDef(network, bridged, iface) + netXml = libvirt.createNetworkDef(network, + bridge=network if bridged else None, + iface=iface if not bridged else None) if not skipBackup: self._networkBackup(network) libvirt.createNetwork(netXml) diff --git a/lib/vdsm/network/configurators/iproute2.py b/lib/vdsm/network/configurators/iproute2.py index 4a96966..317dc5a 100644 --- a/lib/vdsm/network/configurators/iproute2.py +++ b/lib/vdsm/network/configurators/iproute2.py @@ -333,7 +333,9 @@ f.write(value) def createLibvirtNetwork(self, network, bridged=True, iface=None): - netXml = libvirt.createNetworkDef(network, bridged, iface) + netXml = libvirt.createNetworkDef(network, + bridge=network if bridged else None, + iface=iface if not bridged else None) libvirt.createNetwork(netXml) def removeLibvirtNetwork(self, network): diff --git a/lib/vdsm/network/configurators/pyroute_two.py b/lib/vdsm/network/configurators/pyroute_two.py index 8aece53..ff0b95f 100644 --- a/lib/vdsm/network/configurators/pyroute_two.py +++ b/lib/vdsm/network/configurators/pyroute_two.py @@ -164,7 +164,9 @@ f.write(value) def createLibvirtNetwork(self, network, bridged=True, iface=None): - netXml = libvirt.createNetworkDef(network, bridged, iface) + netXml = libvirt.createNetworkDef(network, + bridge=network if bridged else None, + iface=iface if not bridged else None) libvirt.createNetwork(netXml) def removeLibvirtNetwork(self, network): diff --git a/lib/vdsm/network/legacy_switch.py b/lib/vdsm/network/legacy_switch.py index 139ddfc..24d13cb 100644 --- a/lib/vdsm/network/legacy_switch.py +++ b/lib/vdsm/network/legacy_switch.py @@ -398,6 +398,7 @@ """ _netinfo = CachingNetInfo() _netinfo.networks[network] = netAttr + _netinfo.networks[network]['bridged'] = netAttr['bridge'] is not None _netinfo.networks[network]['dhcpv4'] = False if _netinfo.networks[network]['bridged']: diff --git a/lib/vdsm/network/libvirt.py b/lib/vdsm/network/libvirt.py index 12271d7..e80ba54 100644 --- a/lib/vdsm/network/libvirt.py +++ b/lib/vdsm/network/libvirt.py @@ -42,7 +42,8 @@ raise -def createNetworkDef(network, bridged=True, iface=None): +def createNetworkDef(network, bridge=None, vlan=None, iface=None, + virtualport=None): """ Creates Network Xml e.g.: <network> @@ -58,6 +59,9 @@ or interface subelement. """ + if bridge and iface: + raise AttributeError('Cannot set both bridge and iface at once') + netName = LIBVIRT_NET_PREFIX + network def EtreeElement(tagName, text=None, **attrs): @@ -70,16 +74,25 @@ return elem root = etree.Element('network') - nameElem = EtreeElement('name', netName) + + root.append(EtreeElement('name', netName)) + forwardElem = EtreeElement( - 'forward', mode='bridge' if bridged else 'passthrough' - ) - root.append(nameElem) + 'forward', mode='bridge' if bridge else 'passthrough') root.append(forwardElem) - if bridged: - root.append(EtreeElement('bridge', name=network)) - else: + if iface: forwardElem.append(EtreeElement('interface', dev=iface)) + elif bridge: + root.append(EtreeElement('bridge', name=bridge)) + + if vlan is not None: + vlanElem = EtreeElement('vlan') + root.append(vlanElem) + vlanElem.append(EtreeElement('tag', id=vlan)) + + if virtualport: + root.append(EtreeElement('virtualport', type=virtualport)) + return etree.tostring(root) @@ -107,8 +120,17 @@ :returns: dict of networkname={properties} :rtype: dict of dict - { 'ovirtmgmt': { 'bridge': 'ovirtmgmt', 'bridged': True} - 'red': { 'iface': 'red', 'bridged': False}} + for legacy: + {'ovirtmgmt': { + 'bridge': 'ovirtmgmt', 'vlan': None, 'iface': None, + 'virtualport': None} + 'red': { + 'bridge': None, 'vlan': None, 'iface': 'red', + 'virtualport': None}} + for ovs: + {'ovirtmgmt': { + 'bridge': 'vdsmbr_xxx', 'vlan': 10, 'iface': None, + 'virtualport': 'openvswitch'}} """ nets = {} conn = libvirtconnection.get() @@ -116,13 +138,27 @@ for net, netname in allNets: if netname.startswith(LIBVIRT_NET_PREFIX): netname = netname[len(LIBVIRT_NET_PREFIX):] - nets[netname] = {} xml = etree.fromstring(net.XMLDesc(0)) - interface = xml.find('.//interface') - if interface is not None: - nets[netname]['iface'] = interface.get('dev') - nets[netname]['bridged'] = False - else: - nets[netname]['bridge'] = xml.find('.//bridge').get('name') - nets[netname]['bridged'] = True + + _bridge = xml.find('.//bridge') + bridge = _bridge.get('name') if _bridge is not None else None + + _vlan = xml.find('.//vlan') + _tag = _vlan.find('.//tag') if _vlan is not None else None + vlan = _tag.get('id') if _tag is not None else None + canonicalized_vlan = int(vlan) if vlan is not None else vlan + + _iface = xml.find('.//interface') + iface = _iface.get('dev') if _iface is not None else None + + _virtualport = xml.find('.//virtualport') + virtualport = (_virtualport.get('type') if _virtualport is not None + else None) + + nets[netname] = { + 'bridge': bridge, + 'vlan': canonicalized_vlan, + 'iface': iface, + 'virtualport': virtualport + } return nets diff --git a/lib/vdsm/network/netinfo/cache.py b/lib/vdsm/network/netinfo/cache.py index 73c87c6..d01d2a8 100644 --- a/lib/vdsm/network/netinfo/cache.py +++ b/lib/vdsm/network/netinfo/cache.py @@ -126,7 +126,8 @@ for net, netAttr in six.iteritems(nets): try: # Pass the iface if the net is _not_ bridged, the bridge otherwise - d[net] = _getNetInfo(netAttr.get('iface', net), netAttr['bridged'], + d[net] = _getNetInfo(netAttr['iface'] or net, + netAttr['bridge'] is not None, routes, ipAddrs, running_config.networks.get(net, None)) except KeyError: diff --git a/lib/vdsm/network/netswitch.py b/lib/vdsm/network/netswitch.py index a5d393c..357d62c 100644 --- a/lib/vdsm/network/netswitch.py +++ b/lib/vdsm/network/netswitch.py @@ -22,9 +22,9 @@ import six +from vdsm.network import libvirt from vdsm.network.ip import address from vdsm.network.ip import dhclient -from vdsm.network.libvirt import networks as libvirt_nets from vdsm.network.link import iface from vdsm.network.link.bond import Bond from vdsm.network.link.setup import SetupBonds @@ -136,7 +136,7 @@ def _setup_legacy(networks, bondings, options, in_rollback): - _libvirt_nets = libvirt_nets() + _libvirt_nets = libvirt.networks() _netinfo = CachingNetInfo(netinfo_get(libvirtNets2vdsm(_libvirt_nets))) with legacy_switch.ConfiguratorClass(in_rollback) as configurator: diff --git a/tests/network/libvirt_test.py b/tests/network/libvirt_test.py index b62351a..a182024 100644 --- a/tests/network/libvirt_test.py +++ b/tests/network/libvirt_test.py @@ -19,6 +19,7 @@ # from __future__ import absolute_import +from contextlib import contextmanager import re import xml.etree.ElementTree as ET @@ -30,11 +31,13 @@ NETWORK = 'awesome_net' LIBVIRT_NETWORK = 'vdsm-' + NETWORK +OVS_BRIDGE = 'vdsmbr_123' +VLAN = 10 IFACE = 'dummy' @attr(type='unit') -class LibvirtTests(TestCaseBase): +class LibvirtUnitTests(TestCaseBase): def assertEqualXml(self, a, b): """Compare two xml strings for equality""" @@ -53,7 +56,7 @@ <forward mode='bridge'/> <bridge name='{}'/> </network>""".format(LIBVIRT_NETWORK, NETWORK) - actual_doc = libvirt.createNetworkDef(NETWORK, bridged=True) + actual_doc = libvirt.createNetworkDef(NETWORK, bridge=NETWORK) self.assertEqualXml(expected_doc, actual_doc) def test_create_net_xml_with_iface(self): @@ -63,6 +66,71 @@ <interface dev='{}'/> </forward> </network>""".format(LIBVIRT_NETWORK, IFACE) - actual_doc = libvirt.createNetworkDef( - NETWORK, bridged=False, iface=IFACE) + actual_doc = libvirt.createNetworkDef(NETWORK, iface=IFACE) self.assertEqualXml(expected_doc, actual_doc) + + def test_create_net_xml_with_ovs_bridge(self): + expected_doc = """<network> + <name>{}</name> + <forward mode='bridge'/> + <bridge name='{}'/> + <virtualport type='openvswitch'/> + </network>""".format(LIBVIRT_NETWORK, OVS_BRIDGE) + actual_doc = libvirt.createNetworkDef( + NETWORK, bridge=OVS_BRIDGE, virtualport='openvswitch') + self.assertEqualXml(expected_doc, actual_doc) + + def test_create_net_xml_with_ovs_bridge_and_vlan(self): + expected_doc = """<network> + <name>{}</name> + <forward mode='bridge'/> + <bridge name='{}'/> + <vlan> + <tag id='{}'/> + </vlan> + <virtualport type='openvswitch'/> + </network>""".format(LIBVIRT_NETWORK, OVS_BRIDGE, + VLAN) + actual_doc = libvirt.createNetworkDef( + NETWORK, bridge=OVS_BRIDGE, vlan=VLAN, virtualport='openvswitch') + self.assertEqualXml(expected_doc, actual_doc) + + +@attr(type='integration') +class LibvirtIntegrationTests(TestCaseBase): + + def _test_create_net(self, **net_attrs): + net_def = libvirt.createNetworkDef(NETWORK, **net_attrs) + with libvirt_net(NETWORK, net_def): + nets = libvirt.networks() + self.assertIn(NETWORK, nets) + self.assertEqual(nets[NETWORK], net_attrs) + nets = libvirt.networks() + self.assertNotIn(NETWORK, nets) + + def test_create_net_with_bridge(self): + self._test_create_net( + bridge=NETWORK, iface=None, vlan=None, virtualport=None) + + def test_create_net_with_iface(self): + self._test_create_net( + bridge=None, iface=IFACE, vlan=None, virtualport=None) + + def test_create_net_with_ovs_bridge(self): + self._test_create_net( + bridge=OVS_BRIDGE, iface=None, vlan=None, + virtualport='openvswitch') + + def test_create_net_with_ovs_bridge_and_vlan(self): + self._test_create_net( + bridge=OVS_BRIDGE, iface=None, vlan=VLAN, + virtualport='openvswitch') + + +@contextmanager +def libvirt_net(net, net_def): + try: + libvirt.createNetwork(net_def) + yield + finally: + libvirt.removeNetwork(net) diff --git a/vdsm_hooks/ovs/ovs_before_network_setup_libvirt.py b/vdsm_hooks/ovs/ovs_before_network_setup_libvirt.py index 9eb7495..55d8e75 100644 --- a/vdsm_hooks/ovs/ovs_before_network_setup_libvirt.py +++ b/vdsm_hooks/ovs/ovs_before_network_setup_libvirt.py @@ -34,9 +34,7 @@ else: if net in running_config.networks: libvirt_remove.add(net) - libvirt_network_xml = libvirt.createNetworkDef( - net, bridged=True, iface=(attrs.get('nic') or - attrs.get('bonding'))) + libvirt_network_xml = libvirt.createNetworkDef(net, bridge=net) libvirt_create[net] = libvirt_network_xml return libvirt_create, libvirt_remove -- To view, visit https://gerrit.ovirt.org/65065 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6de49e456d8c49f0309690a7fc2bee995f9846bb Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Petr Horáček <phoracek(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>

3 5

Change in vdsm[master]: testlib: Handle EINTR in waitpid
by Nir Soffer 08 Oct '16

08 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: testlib: Handle EINTR in waitpid ...................................................................... testlib: Handle EINTR in waitpid Handle EINTR when waiting for a child process in a @forked test. Fixes this error seen on travis: Traceback (most recent call last): File "/vdsm/tests/testlib.py", line 521, in wrapper _, status = os.waitpid(pid, 0) OSError: [Errno 4] Interrupted system call Change-Id: Ibe64ef55111afe84c20f11a001419312cf4cfff6 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/testlib.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/62/65262/1 diff --git a/tests/testlib.py b/tests/testlib.py index e8c4274..837349c 100644 --- a/tests/testlib.py +++ b/tests/testlib.py @@ -518,7 +518,7 @@ os.write(w, pickle.dumps(e)) os._exit(1) else: - _, status = os.waitpid(pid, 0) + _, status = osutils.uninterruptible(os.waitpid, pid, 0) if status != 0: e = pickle.loads(os.read(r, 4006)) raise e -- To view, visit https://gerrit.ovirt.org/65262 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibe64ef55111afe84c20f11a001419312cf4cfff6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 5

Change in vdsm[master]: dockerfile: Add missing networking packages
by Nir Soffer 08 Oct '16

08 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: dockerfile: Add missing networking packages ...................................................................... dockerfile: Add missing networking packages This patch adds: - openvswitch - bridge-utils Change-Id: I00f2caff47b5e488536cfd898b462ce932ec4862 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M contrib/Dockerfile 1 file changed, 2 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/59/65259/1 diff --git a/contrib/Dockerfile b/contrib/Dockerfile index 621ba22..1090cbb 100644 --- a/contrib/Dockerfile +++ b/contrib/Dockerfile @@ -5,6 +5,7 @@ PyYAML \ autoconf \ automake \ + bridge-utils \ dosfstools \ e2fsprogs \ genisoimage \ @@ -13,6 +14,7 @@ libvirt-python \ make \ mom \ + openvswitch \ psmisc \ python-cpopen \ python-decorator \ -- To view, visit https://gerrit.ovirt.org/65259 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I00f2caff47b5e488536cfd898b462ce932ec4862 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 6

Change in vdsm[master]: mount: Add mounts monitor
by Nir Soffer 08 Oct '16

08 Oct '16

Nir Soffer has restored this change. Change subject: mount: Add mounts monitor ...................................................................... Restored -- To view, visit https://gerrit.ovirt.org/56516 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: restore Gerrit-Change-Id: I25ad36ebfde12d2e3250e893de98bba66f160623 Gerrit-PatchSet: 4 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Freddy Rolland <frolland(a)redhat.com> Gerrit-Reviewer: Idan Shaby <ishaby(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Jenkins CI RO Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>

1 0

Change in vdsm[master]: hsm: Report vg name in getDeviceList
by Nir Soffer 08 Oct '16

08 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: hsm: Report vg name in getDeviceList ...................................................................... hsm: Report vg name in getDeviceList Hosted engine needs the iscsi session info used by the hosted engine storage domain. getDeviceList() seems to include the needed info, but it does not report the vg name for each device, making it hard to match the iscsi session info and the hosted engine storage domain. We return now the vg name of each device, which seems to be useful info regardless of hosted engine needs, and can be used on the engine side for reconstructing host state or validating engine view vs host view. Change-Id: I116714cb5143ea92f5cb54c3f80f895c07ada536 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/rpc/vdsmapi-schema.json M vdsm/storage/hsm.py 2 files changed, 11 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/23/45823/1 diff --git a/vdsm/rpc/vdsmapi-schema.json b/vdsm/rpc/vdsmapi-schema.json index e0e95b9..1b2a171 100644 --- a/vdsm/rpc/vdsmapi-schema.json +++ b/vdsm/rpc/vdsmapi-schema.json @@ -1473,6 +1473,10 @@ # # @status: The device status (free/used/unknown) # +# @vgname: The LVM volume group name, if this device is used as +# a physical volume. This is typically a storage domain +# UUID. +# # Since: 4.10.0 # # Notes: The value of @serial may be dependent on the current host so this @@ -1490,7 +1494,8 @@ 'pathstatus': ['BlockDevicePathInfo'], 'pathlist': ['IscsiSessionInfo'], 'logicalblocksize': 'uint', 'physicalblocksize': 'uint', 'partitioned': 'bool', - 'pvsize': 'uint', 'status': 'BlockDeviceStatus'}} + 'pvsize': 'uint', 'status': 'BlockDeviceStatus', + 'vgname': 'str'}} ## # @Host.getDeviceList: diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 1b8c064..32e16c3 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -2019,14 +2019,18 @@ pvuuid = pv.uuid pvsize = pv.size vguuid = pv.vg_uuid + vgname = pv.vg_name else: pvuuid = "" pvsize = "" vguuid = "" + vgname = "" devInfo = {'GUID': dev.get("guid", ""), 'pvUUID': pvuuid, 'pvsize': str(pvsize), - 'vgUUID': vguuid, 'vendorID': dev.get("vendor", ""), + 'vgUUID': vguuid, + 'vgname': vgname, + 'vendorID': dev.get("vendor", ""), 'productID': dev.get("product", ""), 'fwrev': dev.get("fwrev", ""), "serial": dev.get("serial", ""), -- To view, visit https://gerrit.ovirt.org/45823 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I116714cb5143ea92f5cb54c3f80f895c07ada536 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

7 22

Change in vdsm[master]: tests: Test both ssl and m2c
by Nir Soffer 08 Oct '16

08 Oct '16

Nir Soffer has restored this change. Change subject: tests: Test both ssl and m2c ...................................................................... Restored -- To view, visit https://gerrit.ovirt.org/48823 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: restore Gerrit-Change-Id: Iaa6415c68261f0458b2befa1139b3c8f155820ac Gerrit-PatchSet: 3 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Jenkins CI RO Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org>

1 0

Change in vdsm[master]: utils: Consider sleep time in deadline calculation
by Nir Soffer 08 Oct '16

08 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: utils: Consider sleep time in deadline calculation ...................................................................... utils: Consider sleep time in deadline calculation This patches fixes 2 utils.retry() broken tests: - The special case when deadline has reached when an operation was done. Previously we use to sleep and perform another retry, now we bail out. - The special case when deadline was not reached when an operation was done, but we don't have time for sleep (sleeping will reach or exceed the deadline). Previously we used to sleep and perform another retry, now we bail out. Change-Id: I7f41c6b21e3432159c13d46cfe75d1f6236cbb8c Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M lib/vdsm/utils.py M tests/utilsTests.py 2 files changed, 3 insertions(+), 8 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/46400/1 diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py index caf4cc2..4a4b153 100644 --- a/lib/vdsm/utils.py +++ b/lib/vdsm/utils.py @@ -938,10 +938,7 @@ if tries in [0, None]: tries = -1 - if timeout in [0, None]: - timeout = -1 - - startTime = monotonic_time() + deadline = monotonic_time() + timeout if timeout else None while True: tries -= 1 @@ -951,7 +948,7 @@ if tries == 0: raise - if (timeout > 0) and ((monotonic_time() - startTime) > timeout): + if deadline and monotonic_time() + sleep >= deadline: raise if stopCallback is not None and stopCallback(): diff --git a/tests/utilsTests.py b/tests/utilsTests.py index d213de5..fd4f42e 100644 --- a/tests/utilsTests.py +++ b/tests/utilsTests.py @@ -44,7 +44,7 @@ from testlib import permutations, expandPermutations from testlib import VdsmTestCase as TestCaseBase from testValidation import checkSudo -from testValidation import brokentest, stresstest +from testValidation import stresstest from multiprocessing import Process EXT_SLEEP = "sleep" @@ -86,7 +86,6 @@ # Make sure we had the proper amount of iterations before failing self.assertEquals(counter[0], limit) - @brokentest("deadline is not respected") @MonkeyPatch(utils, 'monotonic_time', FakeTime(0)) @MonkeyPatch(time, 'sleep', fake_sleep) def testTimeoutDeadlineReached(self): @@ -104,7 +103,6 @@ timeout=3, sleep=1) self.assertEqual(utils.monotonic_time.now, 3) - @brokentest("sleep is not considered in deadline calculation") @MonkeyPatch(utils, 'monotonic_time', FakeTime(0)) @MonkeyPatch(time, 'sleep', fake_sleep) def testTimeoutNoTimeForSleep(self): -- To view, visit https://gerrit.ovirt.org/46400 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7f41c6b21e3432159c13d46cfe75d1f6236cbb8c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

4 10

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches October 2016