vdsm-patches April 2015

vdsm-patches@lists.fedorahosted.org

51 participants
798 discussions

Change in vdsm[master]: storage: Make StorageDomain._getRepoPath public
by alitke＠redhat.com 26 May '15

26 May '15

Adam Litke has uploaded a new change for review. Change subject: storage: Make StorageDomain._getRepoPath public ...................................................................... storage: Make StorageDomain._getRepoPath public This method is needed by the new SDM code to perform operations on images and volumes. Make it public. Change-Id: I92b6604df74b0d984895c6e0d4f8ff7007403e37 Signed-off-by: Adam Litke <alitke(a)redhat.com> --- M vdsm/storage/fileSD.py M vdsm/storage/sd.py 2 files changed, 4 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/66/39066/1 diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 9ceda8d..1220939 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -641,7 +641,7 @@ return templateImage = tImgs[0] relinkImgs = tuple(tImgs[1:]) - repoPath = self._getRepoPath() + repoPath = self.getRepoPath() basePath = os.path.join(repoPath, self.sdUUID, sd.DOMAIN_IMAGES) volFiles = [volUUID, volUUID + fileVolume.META_FILEEXT] if self.hasVolumeLeases(): diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index a771abf..bbec420 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -426,7 +426,7 @@ Create a new volume """ return self.getVolumeClass().create( - self._getRepoPath(), self.sdUUID, imgUUID, size, volFormat, + self.getRepoPath(), self.sdUUID, imgUUID, size, volFormat, preallocate, diskType, volUUID, desc, srcImgUUID, srcVolUUID) def getMDPath(self): @@ -658,7 +658,7 @@ self.mountMaster() self.createMasterTree() - def _getRepoPath(self): + def getRepoPath(self): # This is here to make sure no one tries to get a repo # path from an ISO domain. if self.getDomainClass() == ISO_DOMAIN: @@ -668,7 +668,7 @@ return os.path.join(self.storage_repository, self.getPools()[0]) def getLinkBCImagePath(self, imgUUID): - return image.Image(self._getRepoPath()) \ + return image.Image(self.getRepoPath()) \ .getImageDir(self.sdUUID, imgUUID) def getIsoDomainImagesDir(self): -- To view, visit https://gerrit.ovirt.org/39066 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I92b6604df74b0d984895c6e0d4f8ff7007403e37 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <alitke(a)redhat.com>

6 22

Change in vdsm[master]: task: add the support for abortEvent
by Federico Simoncelli 26 May '15

26 May '15

Federico Simoncelli has uploaded a new change for review. Change subject: task: add the support for abortEvent ...................................................................... task: add the support for abortEvent Change-Id: Ib82289e28e5ad9ea142850c31ccff3366b8397dc Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/task.py 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/89/33689/1 diff --git a/vdsm/storage/task.py b/vdsm/storage/task.py index cc1d85e..97c30bf 100644 --- a/vdsm/storage/task.py +++ b/vdsm/storage/task.py @@ -58,6 +58,7 @@ from threadLocal import vars from weakref import proxy from vdsm.config import config +from vdsm.eventfd import EventFile import outOfProcess as oop from logUtils import SimpleLogAdapter @@ -488,6 +489,7 @@ self.mng = None self._aborting = False + self.abortEvent = EventFile() self._forceAbort = False self.ref = 0 @@ -544,6 +546,7 @@ def __state_aborting(self, fromState): if self.ref > 1: return + self.abortEvent.set() self.log.debug("_aborting: recover policy %s", self.recoveryPolicy) if self.recoveryPolicy == TaskRecoveryType.auto: self._updateState(State.racquiring) @@ -564,6 +567,7 @@ def __state_raborting(self, fromState): if self.ref == 1: + self.abortEvent.set() self._updateState(State.failed) else: self.log.warn("State was change to 'raborting' " @@ -1225,6 +1229,7 @@ "ignoring", self.state) return + self.abortEvent.set() self._aborting = True self._forceAbort = force finally: -- To view, visit http://gerrit.ovirt.org/33689 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib82289e28e5ad9ea142850c31ccff3366b8397dc Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

6 25

Change in vdsm[master]: v2v: Job related actions
by shavivi＠redhat.com 26 May '15

26 May '15

Shahar Havivi has uploaded a new change for review. Change subject: v2v: Job related actions ...................................................................... v2v: Job related actions Added few job related actions: Delete job Abort Job Get ovf (jobs output) Change-Id: I9a58583a88e2f17ce0847626cd601efbe1a00ac9 Signed-off-by: Shahar Havivi <shaharh(a)redhat.com> --- M client/vdsClient.py M vdsm/API.py M vdsm/rpc/BindingXMLRPC.py M vdsm/rpc/Bridge.py M vdsm/rpc/vdsmapi-schema.json M vdsm/v2v.py 6 files changed, 181 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/61/39361/1 diff --git a/client/vdsClient.py b/client/vdsClient.py index 9704fdb..b5be5a1 100755 --- a/client/vdsClient.py +++ b/client/vdsClient.py @@ -1928,6 +1928,26 @@ print "v2v job started" return status['status']['code'], status['status']['message'] + def getConvertedVm(self, args): + if len(args) != 1: + raise ValueError('Wrong number of arguments') + status = self.s.getConvertedVm(args[0]) + if status['status']['code'] == 0: + print status['ovf'] + return status['status']['code'], status['status']['message'] + + def abortV2VJob(self, args): + if len(args) != 1: + raise ValueError('Wrong number of arguments') + status = self.s.abortV2VJob(args[0]) + return status['status']['code'], status['status']['message'] + + def deleteV2VJob(self, args): + if len(args) != 1: + raise ValueError('Wrong number of arguments') + status = self.s.deleteV2VJob(args[0]) + return status['status']['code'], status['status']['message'] + if __name__ == '__main__': if _glusterEnabled: serv = ge.GlusterService() @@ -2811,6 +2831,21 @@ 'If auth argument is provided, password will be ' 'ignored (yet has to be specified, ie -)' )), + 'getConvertedVm': ( + serv.getConvertedVm, ( + '<jobId>', + 'Return converted VMs OVF' + )), + 'abortV2VJob': ( + serv.abortV2VJob, ( + '<jobId>', + 'Abort running V2V convert process' + )), + 'deleteV2VJob': ( + serv.deleteV2VJob, ( + '<jobId>', + 'Delete V2v job when job is done' + )), } if _glusterEnabled: commands.update(ge.getGlusterCmdDict(serv)) diff --git a/vdsm/API.py b/vdsm/API.py index e9454b4..f2bbfe9 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -1410,6 +1410,15 @@ return v2v.convert(uri, username, password, vmProperties, jobId, self._cif) + def getConvertedVm(self, jobId): + return v2v.get_converted_vm(jobId) + + def deleteV2VJob(self, jobId): + return v2v.delete_job(jobId) + + def abortV2VJob(self, jobId): + return v2v.abort_job(jobId) + # Networking-related functions def setupNetworks(self, networks, bondings, options): """Add a new network to this vds, replacing an old one.""" diff --git a/vdsm/rpc/BindingXMLRPC.py b/vdsm/rpc/BindingXMLRPC.py index 9aefe9f..68d8b44 100644 --- a/vdsm/rpc/BindingXMLRPC.py +++ b/vdsm/rpc/BindingXMLRPC.py @@ -370,6 +370,18 @@ return api.convertVmFromExternalSystem(uri, username, password, vmProperties, jobId) + def getConvertedVm(self, jobId): + api = API.Global() + return api.getConvertedVm(jobId) + + def abortV2VJob(self, jobId): + api = API.Global() + return api.abortV2VJob(jobId) + + def deleteV2VJob(self, jobId): + api = API.Global() + return api.deleteV2VJob(jobId) + def vmPause(self, vmId): vm = API.VM(vmId) return vm.pause() @@ -1044,7 +1056,10 @@ (self.vmGetIoTunePolicy, 'getIoTunePolicy'), (self.getExternalVMs, 'getExternalVMs'), (self.convertVmFromExternalSystem, - 'convertVmFromExternalSystem')) + 'convertVmFromExternalSystem'), + (self.getConvertedVm, 'getConvertedVm'), + (self.abortV2VJob, 'abortV2VJob'), + (self.deleteV2VJob, 'deleteV2VJob')) def getIrsMethods(self): return ((self.domainActivate, 'activateStorageDomain'), diff --git a/vdsm/rpc/Bridge.py b/vdsm/rpc/Bridge.py index 3adb71b..30c9579 100644 --- a/vdsm/rpc/Bridge.py +++ b/vdsm/rpc/Bridge.py @@ -403,6 +403,7 @@ 'Host_getDeviceList': {'ret': 'devList'}, 'Host_getDevicesVisibility': {'ret': 'visible'}, 'Host_getExternalVMs': {'ret': 'vmList'}, + 'Host_getConvertedVm': {'ret': 'ovf'}, 'Host_getHardwareInfo': {'ret': 'info'}, 'Host_getLVMVolumeGroups': {'ret': 'vglist'}, 'Host_getRoute': {'ret': 'info'}, diff --git a/vdsm/rpc/vdsmapi-schema.json b/vdsm/rpc/vdsmapi-schema.json index 7a8b2f9..c22ab94 100644 --- a/vdsm/rpc/vdsmapi-schema.json +++ b/vdsm/rpc/vdsmapi-schema.json @@ -3816,6 +3816,43 @@ 'vmProperties': 'ExternalVmInfo', 'jobId': 'UUID'}} ## +# @Host.getConvertedVm: +# +# Return OVF string that created via convertVmFromExternalSystem call +# +# @jobId: UUID to identify the job that created in convertVmFromExternalSystem +# +# Since: 4.17.0 +## +{'command': {'class': 'Host', 'name': 'getConvertedVm'}, + 'data': {'jobId': 'UUID'} + 'returns': 'str'} + +## +# @Host.abortV2VJob: +# +# Abort V2V importing process +# +# @jobId: UUID to identify the job that created in convertVmFromExternalSystem +# +# Since: 4.17.0 +## +{'command': {'class': 'Host', 'name': 'abortV2VJob'}, + 'data': {'jobId': 'UUID'}} + +## +# @Host.deleteV2VJob: +# +# Delete V2V job when job is done or stop due to errors +# +# @jobId: UUID to identify the job that created in convertVmFromExternalSystem +# +# Since: 4.17.0 +## +{'command': {'class': 'Host', 'name': 'deleteV2VJob'}, + 'data': {'jobId': 'UUID'}} + +## # @VMFullInfo: # # Full information about VM. diff --git a/vdsm/v2v.py b/vdsm/v2v.py index 9b63f70..a0370f7 100644 --- a/vdsm/v2v.py +++ b/vdsm/v2v.py @@ -89,6 +89,21 @@ err_name = 'V2VJobExists' +class NoSuchJob(V2VError): + ''' Job not exists in _jobs collection ''' + err_name = 'V2VNoSuchJob' + + +class JobNotDone(V2VError): + ''' Import process still in progress ''' + err_name = 'V2VJobNotDone' + + +class NoSuchOvf(V2VError): + ''' Ovf path is not exists in /var/run/vdsm/v2v/ ''' + err_name = 'V2VNoSuchOvf' + + class VolumeError(RuntimeError): ''' Error preparing volume ''' @@ -150,6 +165,39 @@ return {'status': doneCode} +def get_converted_vm(job_id): + try: + job = _get_job(job_id) + _validate_job_done(job) + ovf = _read_ovf(job_id) + except V2VError as e: + logging.warning('Converted VM error %s', e) + return errCode[e.err_name] + return {'status': doneCode, 'ovf': ovf} + + +def delete_job(job_id): + try: + job = _get_job(job_id) + _validate_job_finished(job) + _remove_job(job_id) + except V2VError as e: + logging.warning('Error deleting job: %s, error: %s', job_id, e) + return errCode[e.err_name] + return {'status': doneCode} + + +def abort_job(job_id): + try: + job = _get_job(job_id) + job.abort() + _remove_job(job_id) + except V2VError as e: + logging.warning('Error aboring job: %s, error: %s', job_id, e) + return errCode[e.err_name] + return {'status': doneCode} + + def jobs(): ret = {} with _lock: @@ -162,6 +210,17 @@ 'progress': job.progress } return ret + + +def _read_ovf(job_id): + file_name = os.path.join(_V2V_DIR, "%s.ovf" % job_id) + try: + with open(file_name, 'r') as f: + return f.read() + except IOError as e: + if e.errno != errno.ENOENT: + raise + raise NoSuchOvf("No such ovf %r" % file_name) def _add_job(job_id, job): @@ -179,6 +238,30 @@ return path.rsplit(os.sep, 3)[0] +def _get_job(job_id): + with _lock: + if job_id not in _jobs: + raise NoSuchJob() + return _jobs[job_id] + + +def _remove_job(job_id): + with _lock: + if job_id not in _jobs: + raise NoSuchJob("No such job %r" % job_id) + del _jobs[job_id] + + +def _validate_job_done(job): + if job.status != STATUS.DONE: + raise JobNotDone("Job %r is %s" % (job.id, job.status)) + + +def _validate_job_finished(job): + if job.status not in (STATUS.DONE, STATUS.FAILED, STATUS.ABORTED): + raise JobNotDone("Job %r is %s" % (job.id, job.status)) + + class ImportVm(object): ABORT_RETRIES = 3 ABORT_DELAY = 10 -- To view, visit https://gerrit.ovirt.org/39361 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9a58583a88e2f17ce0847626cd601efbe1a00ac9 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shahar Havivi <shavivi(a)redhat.com>

7 98

Change in vdsm[master]: tests: add more tests for custom error responses
by fromani＠redhat.com 26 May '15

26 May '15

Francesco Romani has uploaded a new change for review. Change subject: tests: add more tests for custom error responses ...................................................................... tests: add more tests for custom error responses An upcoming patchset wants to introduce and leverage the response module. This patch prepares the soil by adding unit tests. We start from the error paths, since they are (hopefully?) less often tested. Moreover, to test error path is often clumsy due to the need to trigger faults or inject them in the code, so this is a useful safety net to have. Change-Id: Icf3d03f125f9c53339d9ad7fef8c31482bab0bfb Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M tests/vmTests.py M tests/vmfakelib.py 2 files changed, 133 insertions(+), 13 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/65/38665/1 diff --git a/tests/vmTests.py b/tests/vmTests.py index 26c39ef..4adea76 100644 --- a/tests/vmTests.py +++ b/tests/vmTests.py @@ -51,6 +51,7 @@ from vmTestsData import CONF_TO_DOMXML_PPC64 from vmTestsData import CONF_TO_DOMXML_NO_VDSM import vmfakelib as fake +from vmStorageTests import drive_config from testValidation import slowtest @@ -873,6 +874,10 @@ GRAPHIC_DEVICES = [{'type': 'graphics', 'device': 'spice', 'port': '-1'}, {'type': 'graphics', 'device': 'vnc', 'port': '-1'}] + TICKET_PARAMS = { + 'userName': 'admin', + 'userId': 'fdfc627c-d875-11e0-90f0-83df133b58cc'} + @MonkeyPatch(libvirtconnection, 'get', lambda x: fake.Connection()) @permutations([[define.NORMAL], [define.ERROR]]) def testTimeOffsetNotPresentByDefault(self, exitCode): @@ -951,25 +956,27 @@ self._verifyDeviceUpdate( device, self.GRAPHIC_DEVICES, domXml, devXml) - def _verifyDeviceUpdate(self, device, allDevices, domXml, devXml): - TICKET_PARAMS = { - 'userName': 'admin', - 'userId': 'fdfc627c-d875-11e0-90f0-83df133b58cc'} + def _getGraphicsDeviceParams(self, graphicsType): + return { + 'deviceType': hwclass.GRAPHICS, + 'graphicsType': graphicsType, + 'password': '***', + 'ttl': 0, + 'existingConnAction': 'disconnect', + 'params': self.TICKET_PARAMS} + def _verifyDeviceUpdate(self, device, allDevices, domXml, devXml): def _check_ticket_params(domXML, conf, params): - self.assertEqual(params, TICKET_PARAMS) + self.assertEqual(params, self.TICKET_PARAMS) with MonkeyPatchScope([(hooks, 'before_vm_set_ticket', _check_ticket_params)]): with fake.VM(devices=allDevices) as testvm: testvm._dom = fake.Domain(domXml) - testvm.updateDevice({ - 'deviceType': 'graphics', - 'graphicsType': device['device'], - 'password': '***', - 'ttl': 0, - 'existingConnAction': 'disconnect', - 'params': TICKET_PARAMS}) + + testvm.updateDevice( + self._getGraphicsDeviceParams(device['device'])) + self.assertEquals(testvm._dom.devXml, devXml) def testDomainNotRunningWithoutDomain(self): @@ -1018,6 +1025,45 @@ 'hdc': libvirt.VIR_DOMAIN_DISK_ERROR_UNSPEC}) testvm._dom = dom self.assertEqual(testvm._readPauseCode(), 'EOTHER') + + @permutations([[libvirt.VIR_ERR_OPERATION_DENIED, 'setNumberOfCpusErr'], + [libvirt.VIR_ERR_NO_DOMAIN, 'noVM']]) + def testSetNumberOfVcpusFailed(self, virtError, vdsmError): + # caveat emptor: in this test we DON't + # overwrite the message, thus will be the default one; but for + # errors != 'noVM', it can change. + with MonkeyPatchScope([(hooks, 'before_set_num_of_cpus', + lambda: None)]): + with fake.VM() as testvm: + testvm._dom = fake.Domain(virtError=virtError) + + res = testvm.setNumberOfCpus(4) # random value + + self.assertEqual(res, define.errCode[vdsmError]) + + def testUpdateDeviceGraphicsFailed(self): + def _check_ticket_params(domXML, conf, params): + pass + + with MonkeyPatchScope([(hooks, 'before_vm_set_ticket', + _check_ticket_params)]): + with fake.VM(devices=self.GRAPHIC_DEVICES) as testvm: + # caveat: we intentionally recycle the very same + # error message to simplify assert() down under + expected_status = define.errCode['ticketErr'] + message = expected_status['status']['message'] + device = 'spice' + domXml = ''' + <devices> + <graphics type="%s" port="5900" /> + </devices>''' % device + + testvm._dom = fake.Domain(domXml, timeoutError=message) + + res = testvm.updateDevice( + self._getGraphicsDeviceParams(device)) + + self.assertEqual(res, expected_status) VM_EXITS = tuple(product((define.NORMAL, define.ERROR), @@ -1298,6 +1344,7 @@ self.assertEqual(int(testvm.conf['smp']), cpus) +@expandPermutations class ChangeBlockDevTests(TestCaseBase): def test_change_cd_failure(self): @@ -1315,3 +1362,65 @@ expected_status = define.errCode['changeDisk']['status'] self.assertEqual(res['status'], expected_status) + + @permutations([[libvirt.VIR_ERR_OPERATION_DENIED, 'hotplugDisk'], + [libvirt.VIR_ERR_NO_DOMAIN, 'noVM']]) + def test_hotplug_disk_failure(self, virtError, vdsmError): + # caveat emptor: in this test we DON't + # overwrite the message, thus will be the default one; but for + # errors != 'noVM', it can change. + PATH = '/this/path/does/not/exists' + + def _fake_hook(driveXml, conf, params): + return driveXml + + cif = fake.ClientIF() + with MonkeyPatchScope([(cif, 'prepareVolumePath', + lambda _: PATH), + (cif, 'teardownVolumePath', + lambda _: None), + (hooks, 'before_disk_hotplug', + _fake_hook), + (utils, 'isBlockDevice', + lambda path: True)]): + with fake.VM(devices=[drive_config()], + cif=cif) as fakevm: + # no specific meaning, actually any error != None is good + fakevm._dom = fake.Domain( + virtError=libvirt.VIR_ERR_ACCESS_DENIED) + + res = fakevm.hotplugDisk({ + 'drive': drive_config( + path=PATH, + serial='65-a561-23e5b495b9bc')}) + + expected_status = define.errCode['hotplugDisk']['status'] + self.assertEqual(res['status'], expected_status) + + def test_hotunplug_inexistent_disk(self): + PATH = '/this/path/does/not/exists' + + def _fake_hook(driveXml, conf, params): + return driveXml + + cif = fake.ClientIF() + with MonkeyPatchScope([(cif, 'prepareVolumePath', + lambda _: PATH), + (cif, 'teardownVolumePath', + lambda _: None), + (hooks, 'before_disk_hotunplug', + _fake_hook), + (utils, 'isBlockDevice', + lambda path: True)]): + with fake.VM(cif=cif) as fakevm: + # no specific meaning, actually any error != None is good + fakevm._dom = fake.Domain( + virtError=libvirt.VIR_ERR_ACCESS_DENIED) + + res = fakevm.hotunplugDisk({}) + + expected_status = define.errCode['hotunplugDisk']['status'] + self.assertEqual(res['status']['code'], + expected_status['code']) + self.assertEqual(res['status']['message'], + "Disk not found") diff --git a/tests/vmfakelib.py b/tests/vmfakelib.py index 6919351..99c1b42 100644 --- a/tests/vmfakelib.py +++ b/tests/vmfakelib.py @@ -61,7 +61,8 @@ virtError=libvirt.VIR_ERR_OK, domState=libvirt.VIR_DOMAIN_RUNNING, domReason=0, - vmId=''): + vmId='', + timeoutError=None): self._xml = xml self.devXml = '' self._virtError = virtError @@ -72,8 +73,12 @@ self._vmId = vmId self.calls = {} self._diskErrors = {} + self._timeoutError = timeoutError def _failIfRequested(self): + if self._timeoutError: + raise vm.TimeoutError(str(self._timeoutError)) + if self._virtError != libvirt.VIR_ERR_OK: err = libvirt.libvirtError(defmsg='') err.err = [self._virtError] @@ -99,6 +104,9 @@ def vcpusFlags(self, flags): return -1 + + def setVcpusFlags(self, numberOfCpus, flags): + self._failIfRequested() def metadata(self, type, uri, flags): self._failIfRequested() @@ -130,6 +138,9 @@ def diskErrors(self): return self._diskErrors + def attachDevice(self, devXml): + self._failIfRequested() + class GuestAgent(object): def __init__(self): -- To view, visit https://gerrit.ovirt.org/38665 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Icf3d03f125f9c53339d9ad7fef8c31482bab0bfb Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

6 47

Change in vdsm[master]: RFE: Report downtime for each live migration
by shavivi＠redhat.com 26 May '15

26 May '15

Shahar Havivi has uploaded a new change for review. Change subject: RFE: Report downtime for each live migration ...................................................................... RFE: Report downtime for each live migration vm.py getStat() is called when migration is done and from Engine initiative. There is no need to call libvirts domain getStats twice, only when Engine initiate the call. Change-Id: I2ff421c489ef24869502bd80461018f0aca2144d Bug-Url: https://bugzilla.redhat.com/970711 Signed-off-by: Shahar Havivi <shaharh(a)redhat.com> --- M vdsm/rpc/Bridge.py M vdsm/virt/migration.py M vdsm/virt/vm.py 3 files changed, 16 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/03/40103/1 diff --git a/vdsm/rpc/Bridge.py b/vdsm/rpc/Bridge.py index bad8f1b..0484669 100644 --- a/vdsm/rpc/Bridge.py +++ b/vdsm/rpc/Bridge.py @@ -384,6 +384,13 @@ return {'params': ret['params'], 'migrationPort': ret['migrationPort']} +def VM_getMigrationStatus_Ret(ret): + """ + Return status of the Migration, progress and downtime if available + """ + return ret + + def Volume_getsize_Ret(ret): """ Merge the two sizes into a single dictionary result. @@ -471,6 +478,7 @@ 'VM_hotunplugNic': {'ret': 'vmList'}, 'VM_mergeStatus': {'ret': 'mergeStatus'}, 'VM_migrationCreate': {'ret': VM_migrationCreate_Ret}, + 'VM_getMigrationStatus': {'ret': VM_getMigrationStatus_Ret}, 'VM_pause': {'ret': VM_running_state_change_Ret}, 'VM_setCpuTunePeriod': {'ret': 'taskStatus'}, 'VM_setCpuTuneQuota': {'ret': 'taskStatus'}, diff --git a/vdsm/virt/migration.py b/vdsm/virt/migration.py index f76fac2..8a31552 100644 --- a/vdsm/virt/migration.py +++ b/vdsm/virt/migration.py @@ -90,7 +90,7 @@ def hibernating(self): return self._mode == MODE_FILE - def getStat(self): + def getStat(self, useDowntime=False): """ Get the status of the migration. """ @@ -98,9 +98,11 @@ # fetch migration status from the monitor thread self.status['progress'] = self._monitorThread.progress - stat = self._vm._dom.jobStats(libvirt.VIR_DOMAIN_JOB_STATS_COMPLETED) - if 'downtime' in stat: - self.status['downtime'] = stat['downtime'] + if useDowntime: + stat = self._vm._dom.jobStats(libvirt. + VIR_DOMAIN_JOB_STATS_COMPLETED) + if 'downtime' in stat: + self.status['downtime'] = stat['downtime'] return self.status diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index 2690c6e..bdb488b 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -1718,7 +1718,7 @@ 'disks': {}} if 'pauseCode' in self.conf: stats['pauseCode'] = self.conf['pauseCode'] - if self.isMigrating(): + if self.isMigrating() and self.migrateStatus().has_key('progress'): stats['migrationProgress'] = self.migrateStatus()['progress'] decStats = {} @@ -1868,7 +1868,7 @@ self._guestCpuLock.release() def migrateStatus(self): - return self._migrationSourceThread.getStat() + return self._migrationSourceThread.getStat(useDowntime=True) def migrateCancel(self): self._acquireCpuLockWithTimeout() -- To view, visit https://gerrit.ovirt.org/40103 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2ff421c489ef24869502bd80461018f0aca2144d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shahar Havivi <shavivi(a)redhat.com>

6 35

Change in vdsm[master]: sampling: hoststats: rationalize getInterfaceStats
by fromani＠redhat.com 25 May '15

25 May '15

Francesco Romani has uploaded a new change for review. Change subject: sampling: hoststats: rationalize getInterfaceStats ...................................................................... sampling: hoststats: rationalize getInterfaceStats HostStatsThread._getInterfaceStats() is supposed to gather network stats. But this method also setups the empty stats with default value. This initialization doesn't belong here, so this patch moves this code outside this method, in the calling one. Change-Id: I556149271fb2786a6ff7341642989ea767eb2392 Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/sampling.py 1 file changed, 16 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/26/40426/1 diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index 7349b9f..710bba9 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -604,13 +604,25 @@ return None def get(self): - stats = self._getInterfacesStats() - stats['cpuSysVdsmd'] = stats['cpuUserVdsmd'] = 0.0 - stats['elapsedTime'] = int(time.time() - self.startTime) + stats = { + 'cpuUser': 0.0, + 'cpuSys': 0.0, + 'cpuIdle': 100.0, + 'rxRate': 0.0, + 'txRate': 0.0, + 'cpuSysVdsmd': 0.0, + 'cpuUserVdsmd': 0.0, + 'elapsedTime': int(time.time() - self.startTime) + } + if len(self._samples) < 2: return stats + + stats.update(self._getInterfacesStats()) + hs0, hs1 = self._samples[0], self._samples[-1] interval = hs1.timestamp - hs0.timestamp + jiffies = (hs1.pidcpu.user - hs0.pidcpu.user) % (2 ** 32) stats['cpuUserVdsmd'] = jiffies / interval jiffies = (hs1.pidcpu.sys - hs0.pidcpu.sys) % (2 ** 32) @@ -680,8 +692,7 @@ * rxRate * txRate """ - stats = {'cpuUser': 0.0, 'cpuSys': 0.0, 'cpuIdle': 100.0, - 'rxRate': 0.0, 'txRate': 0.0} + stats = {} if len(self._samples) < 2: return stats hs0, hs1 = self._samples[0], self._samples[-1] -- To view, visit https://gerrit.ovirt.org/40426 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I556149271fb2786a6ff7341642989ea767eb2392 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

5 24

Change in vdsm[master]: vm: Detect RNG device by type
by smelamud＠redhat.com 25 May '15

25 May '15

Shmuel Leib Melamud has uploaded a new change for review. Change subject: vm: Detect RNG device by type ...................................................................... vm: Detect RNG device by type In Vm._getUnderlyingRngDeviceInfo() method the RNG device must be detected in self.conf['devices'] list by 'type' attribute, not 'device'. See VmInfoBuilder.buildVmRngDevice() in the Engine for reference. Change-Id: I31a1c38eec5be96e298e0f1dd5a7598c2212cb21 Signed-off-by: Shmuel Melamud <smelamud(a)redhat.com> --- M vdsm/virt/vm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/95/40095/1 diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index 331cfcd..464d4aa 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -4198,7 +4198,7 @@ break for dev in self.conf['devices']: - if dev['device'] == hwclass.RNG and \ + if dev['type'] == hwclass.RNG and \ caps.RNG_SOURCES[dev['specParams']['source']] == \ source and 'alias' not in dev: dev['address'] = address -- To view, visit https://gerrit.ovirt.org/40095 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I31a1c38eec5be96e298e0f1dd5a7598c2212cb21 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shmuel Leib Melamud <smelamud(a)redhat.com>

8 16

Change in vdsm[master]: sampling: less verbose debug in _getInterfaceStats
by fromani＠redhat.com 25 May '15

25 May '15

Francesco Romani has uploaded a new change for review. Change subject: sampling: less verbose debug in _getInterfaceStats ...................................................................... sampling: less verbose debug in _getInterfaceStats HostStatsThread._getInterfaceStats needs, among other things, to compute network tx/rx rate. To help troubleshoot rate miscalculation, there is a debug log which shows all the samples used for computation. The problem here is that the log cycles over the internal samples. We want to use SampleWindow in HostStatsThread to reduce code duplication, but SampleWindow doesn't give access to samples yet. To move further, we drop this detailed log for the time being. Future patches may want to reintroduce it on top of (enhanced) SampleWindow. Change-Id: I0a6063c9ac6f9f477a03881e32d0c3821038092f Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/sampling.py 1 file changed, 1 insertion(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/40424/1 diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index f5640d0..8878120 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -710,13 +710,7 @@ if txRate > 100 or rxRate > 100: txRate = min(txRate, 100.0) rxRate = min(rxRate, 100.0) - self._log.debug('Rate above 100%%. DEBUG: ifid %s interval: ' - '%s thisRx %s thisTx %s samples %s', ifid, - interval, thisRx, thisTx, - [(hs.timestamp, hs.interfaces[ifid].rx, - hs.interfaces[ifid].tx) - for hs in self._samples if - ifid in hs.interfaces]) + self._log.debug('Rate above 100%%.') iface = hs1.interfaces[ifid] stats['network'][ifid] = {'name': ifid, 'speed': str(ifrate), 'rxDropped': str(iface.rxDropped), -- To view, visit https://gerrit.ovirt.org/40424 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0a6063c9ac6f9f477a03881e32d0c3821038092f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

5 20

Change in vdsm[master]: vm: Support replication to different storage type
by Nir Soffer 22 May '15

22 May '15

Nir Soffer has uploaded a new change for review. Change subject: vm: Support replication to different storage type ...................................................................... vm: Support replication to different storage type When monitoring replicated drive, we must use hybrid approach; For block-based drive, we get the required extend info from libvirt. For file-based drives, libvirt cannot report the physical size of the drive, so we get it from the replica using irs.getVolumeSize(). When extending replicated drive, we extend the drive or the replica only if they are block-based. Change-Id: I70eb93082af81fe54268ee0133d1252c0d537ca8 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/virt/vm.py 1 file changed, 35 insertions(+), 8 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/85/40185/1 diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index 163ab86..3c59c8b 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -1303,7 +1303,7 @@ continue try: - capacity, alloc, physical = self._dom.blockInfo(drive.path, 0) + capacity, alloc, physical = self._getExtendInfo(drive) except libvirt.libvirtError as e: self.log.error("Unable to get watermarks for drive %s: %s", drive.name, e) @@ -1312,6 +1312,27 @@ ret.append((drive, drive.volumeID, capacity, alloc, physical)) return ret + + def _getExtendInfo(self, drive): + """ + Return extension info for a chunked drive + """ + capacity, alloc, physical = self._dom.blockInfo(drive.path, 0) + + # Libvirt reports watermarks only for the source drive, but for + # file-based drives it reports the same alloc and physical, which + # breaks our extend logic. Since drive is chunked, we must have a + # disk-based replica, so we get the physical size from the replica. + + if not drive.blockDev: + replica = drive.diskReplicate + volsize = self._getVolumeSize(replica["domainID"], + replica["poolID"], + replica["imageID"], + replica["volumeID"]) + physical = volsize.apparentsize + + return capacity, alloc, physical def _shouldExtendVolume(self, drive, volumeID, capacity, alloc, physical): nextPhysSize = drive.getNextVolumeSize(physical, capacity) @@ -1374,7 +1395,11 @@ """ newSize = vmDrive.getNextVolumeSize(curSize, capacity) - if hasattr(vmDrive, 'diskReplicate'): + # If drive is replicated to a block device, we extend first the + # replica, and handle drive later in __afterReplicaExtension. + + replica = getattr(vmDrive, 'diskReplicate', {}) + if replica.get('diskType') == DISK_TYPE.BLOCK: self.__extendDriveReplica(vmDrive, newSize) else: self.__extendDriveVolume(vmDrive, volumeID, newSize) @@ -1406,10 +1431,12 @@ def __afterReplicaExtension(self, volInfo): self.__verifyVolumeExtension(volInfo) vmDrive = self._findDriveByName(volInfo['name']) - self.log.debug("Requesting extension for the original drive: %s " - "(domainID: %s, volumeID: %s)", - vmDrive.name, vmDrive.domainID, vmDrive.volumeID) - self.__extendDriveVolume(vmDrive, vmDrive.volumeID, volInfo['newSize']) + if vmDrive.blockDev: + self.log.debug("Requesting extension for the original drive: %s " + "(domainID: %s, volumeID: %s)", + vmDrive.name, vmDrive.domainID, vmDrive.volumeID) + self.__extendDriveVolume(vmDrive, vmDrive.volumeID, + volInfo['newSize']) def __extendDriveVolume(self, vmDrive, volumeID, newSize): volInfo = { @@ -3464,7 +3491,7 @@ if drive.chunked: try: - capacity, alloc, physical = self._dom.blockInfo(drive.path, 0) + capacity, alloc, physical = self._getExtendInfo(drive) self.extendDriveVolume(drive, drive.volumeID, physical, capacity) except Exception: @@ -4880,7 +4907,7 @@ # plus a bit more to accomodate additional writes to 'top' during the # live merge operation. if drive.chunked: - capacity, alloc, physical = self._dom.blockInfo(drive.path, 0) + capacity, alloc, physical = self._getExtendInfo(drive) self.extendDriveVolume(drive, baseVolUUID, topSize, capacity) # Trigger the collection of stats before returning so that callers -- To view, visit https://gerrit.ovirt.org/40185 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I70eb93082af81fe54268ee0133d1252c0d537ca8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

7 46

Change in vdsm[master]: virt: Consider drive replica in Drive.chunked
by Nir Soffer 22 May '15

22 May '15

Nir Soffer has uploaded a new change for review. Change subject: virt: Consider drive replica in Drive.chunked ...................................................................... virt: Consider drive replica in Drive.chunked When monitoring drive high watermark, it is not enough to check if the drive is chunked, since we may replicate a non-chunked drive to chunked replica, and the replica needs to be extended. Drive high watermark code will check now the watermark also on non-chunked drives, if they are being replicated to chunked storage. Change-Id: Iadf19fa3262e356ee87c92357b8d90aff7dc0356 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/vmStorageTests.py M vdsm/virt/vmdevices/storage.py 2 files changed, 36 insertions(+), 16 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/84/40184/1 diff --git a/tests/vmStorageTests.py b/tests/vmStorageTests.py index 5cd9d54..d454e83 100644 --- a/tests/vmStorageTests.py +++ b/tests/vmStorageTests.py @@ -193,17 +193,6 @@ self.assertXMLEqual(drive.getReplicaXML().toxml(), xml) -def replica(diskType): - return { - "cache": "none", - "device": "disk", - "diskType": diskType, - "format": "cow", - "path": "/path/to/replica", - "propagateErrors": "off", - } - - @expandPermutations class DriveValidation(VdsmTestCase): @@ -328,10 +317,23 @@ ('lun', True, 'raw', False), ('disk', True, 'cow', True), ]) - def test_chunked(self, device, blockDev, format, chunked): + def test_drive(self, device, blockDev, format, chunked): conf = drive_config(device=device, format=format) drive = Drive({}, self.log, **conf) drive._blockDev = blockDev + self.assertEqual(drive.chunked, chunked) + + @permutations([ + # replica diskType, replica format, chunked + (DISK_TYPE.FILE, 'raw', False), + (DISK_TYPE.FILE, 'cow', False), + (DISK_TYPE.BLOCK, 'raw', False), + (DISK_TYPE.BLOCK, 'cow', True), + ]) + def test_replica(self, diskType, format, chunked): + conf = drive_config(diskReplicate=replica(diskType, format=format)) + drive = Drive({}, self.log, **conf) + drive._blockDev = False self.assertEqual(drive.chunked, chunked) @@ -378,3 +380,14 @@ } conf.update(kw) return conf + + +def replica(diskType, format="cow"): + return { + "cache": "none", + "device": "disk", + "diskType": diskType, + "format": format, + "path": "/path/to/replica", + "propagateErrors": "off", + } diff --git a/vdsm/virt/vmdevices/storage.py b/vdsm/virt/vmdevices/storage.py index 9f3f547..641d2bc 100644 --- a/vdsm/virt/vmdevices/storage.py +++ b/vdsm/virt/vmdevices/storage.py @@ -172,13 +172,20 @@ @property def chunked(self): """ - Return True if drive is using chunks and may require extending. + Return True if drive or its replica are using chunks and may require + extending. If a drive is chunked, current drive write watermark and - Drive.volExtensionChunk is used to detect if a drive should be - extended, and getNextVolumeSize to find the new size. + Drive.volExtensionChunk is used to detect if a drive or its replica + should be extended, and getNextVolumeSize to find the new size. """ - return self.blockDev and self.format == "cow" + if self.blockDev and self.format == "cow": + return True + + replica = getattr(self, "diskReplicate", {}) + + return (replica.get("diskType") == DISK_TYPE.BLOCK and + replica.get("format") == "cow") @property def networkDev(self): -- To view, visit https://gerrit.ovirt.org/40184 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Iadf19fa3262e356ee87c92357b8d90aff7dc0356 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

6 31

← Newer
1
...
22
23
24
25
26
27
28
...
80
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches April 2015