vdsm-patches November 2012

vdsm-patches@lists.fedorahosted.org

50 participants
416 discussions

Change in vdsm[master]: schema: New type VmParameters
by agl＠us.ibm.com 08 Nov '12

08 Nov '12

Adam Litke has uploaded a new change for review. Change subject: schema: New type VmParameters ...................................................................... schema: New type VmParameters Although the format of the data passed via VM.create is very similar to a VmDefinition, some of the parameters don't make sense to pass in the context of creating a new VM. Solve this by creating a new type for VM parameters. Change-Id: I00d1b9aed55cbfc2210c1a4091bce17d45b90e67 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 48 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/39/7839/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index f6d3c77..b1dc60c 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -2346,6 +2346,53 @@ 'vmId': 'UUID', 'vmName': 'str', 'vmType': 'VmType'}} ## +# @VmDefinition: +# +# Full virtual machine status and properties. +# +# @acpiEnable: Indicates if ACPI is enabled inside the VM +# +# @custom: #opional A dictionary of custom, free-form properties +# +# @devices: #optional An array of VM devices requested +# +# @display: The type of display +# +# @kvmEnable: Indicates if KVM hardware acceleration is enabled +# +# @memSize: The amount of memory assigned to the VM in MB +# +# @nice: The host scheduling priority +# +# @smp: The number of CPUs presented to the VM +# +# @smpCoresPerSocket: #optional Indicates the number of CPU cores per socket +# +# @smpThreadsPerCore: #optional Indicates the number of CPU threads per core +# +# @timeOffset: The time difference from host to the VM in seconds +# +# @transparentHugePages: Indicates if the Transparent Huge Pages feature is +# enabled for this virtual machine +# +# @vmId: The VM UUID +# +# @vmName: The VM name +# +# @vmType: The type of VM +# +# Since: 4.10.0 +## +{'type': 'VmParameters', + 'data': {'acpiEnable': 'bool', + '*custom': 'StringMap', '*devices': ['VmDevice'], + 'display': 'VmDisplayType', 'kvmEnable': 'bool', 'memSize': 'uint', + 'nice': 'int', 'smp': 'uint', 'smpCoresPerSocket': 'uint', + 'smpThreadsPerCore': 'uint', 'timeOffset': 'uint', + 'transparentHugePages': 'bool', 'vmId': 'UUID', 'vmName': 'str', + 'vmType': 'VmType'}} + +## # @VmFullStatus: # # A VmDefinition uses the same format as a VmFullStatus. @@ -4278,7 +4325,7 @@ # Since: 4.10.0 ## {'command': {'class': 'VM', 'name': 'create'}, - 'data': {'vmParams': 'VmDefinition'}, + 'data': {'vmParams': 'VmParameters'}, 'returns': 'VmDefinition'} ## -- To view, visit http://gerrit.ovirt.org/7839 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I00d1b9aed55cbfc2210c1a4091bce17d45b90e67 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

4 12

Change in vdsm[master]: BZ#855924 -- fix 3.1 -> 3.0 migration again
by peet＠redhat.com 07 Nov '12

07 Nov '12

Peter V. Saveliev has uploaded a new change for review. Change subject: BZ#855924 -- fix 3.1 -> 3.0 migration again ...................................................................... BZ#855924 -- fix 3.1 -> 3.0 migration again The issue was that fixing self._vm.conf is not enough -- we should fix self._machineParams, that will work in all the cases. Signed-off-by: Peter V. Saveliev <peet(a)redhat.com> Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=855924 Change-Id: I334cd39cad3eed30c2c3f1efa7085526454a44fb --- M vdsm/vm.py 1 file changed, 4 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/9036/1 diff --git a/vdsm/vm.py b/vdsm/vm.py index 2229882..591ac1a 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -236,15 +236,15 @@ """ # care only about "drives" list, since # "devices" doesn't cause errors - if 'drives' in self._vm.conf: + if 'drives' in self._machineParams: for item in ("cdrom", "floppy"): new_drives = [] - for drive in self._vm.conf['drives']: + for drive in self._machineParams['drives']: if drive['device'] == item: - self._vm.conf[item] = drive['path'] + self._machineParams[item] = drive['path'] else: new_drives.append(drive) - self._vm.conf['drives'] = new_drives + self._machineParams['drives'] = new_drives def run(self): try: -- To view, visit http://gerrit.ovirt.org/9036 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I334cd39cad3eed30c2c3f1efa7085526454a44fb Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Peter V. Saveliev <peet(a)redhat.com>

4 9

Change in vdsm[master]: schema: Return objects rather than UUIDs
by agl＠us.ibm.com 07 Nov '12

07 Nov '12

Adam Litke has uploaded a new change for review. Change subject: schema: Return objects rather than UUIDs ...................................................................... schema: Return objects rather than UUIDs Rather than returning UUIDs and forcing the API user to instantiate their own objects using those UUIDs, just return initialized objects. This patch just changes the API schema. The specific implementation of this idea appears in the patch that implements the code generator. This is not exhaustive, but an example of the idea. The rest can be converted over time. Change-Id: I7fdfe69436d65ef8190d8a5f1d941e7a628c77c0 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 87 insertions(+), 71 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/40/7840/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index b1dc60c..7e8b9d3 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -939,12 +939,12 @@ # Get a list of all Storage Pools that are connected to this host. # # Returns: -# A list of Storage Pool UUIDs +# A list of Storage Pools # # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getConnectedStoragePools'}, - 'returns': ['UUID']} + 'returns': ['StoragePool']} ## # @BlockDeviceType: @@ -1561,15 +1561,15 @@ ## # @Host.getStorageDomains: # -# Get a list of known Storage Domain UUIDs. +# Get a list of known Storage Domains. # -# @spUUID: #optional Limit to Domains belonging to this Storage Pool +# @storagepoolID: #optional Limit to Domains belonging to this Storage Pool # -# @domainClass: #optional Limit to Domains of this @StorageDomainImageClass +# @domainClass: #optional Limit to Domains of this @StorageDomainImageClass # -# @storageType: #optional Limit to Domains of this @StorageDomainType +# @storageType: #optional Limit to Domains of this @StorageDomainType # -# @remotePath: #optional Limit to Domains having this remotePath +# @remotePath: #optional Limit to Domains having this remotePath # # Returns: # A list of known Storage Domains @@ -1577,9 +1577,9 @@ # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getStorageDomains'}, - 'data': {'*spUUID': 'UUID', '*domainClass': 'StorageDomainImageClass', + 'data': {'*storagepoolID': 'UUID', '*domainClass': 'StorageDomainImageClass', '*storageType': 'StorageDomainType', '*remotePath': 'str'}, - 'returns': ['UUID']} + 'returns': ['StorageDomain']} ## # @Host.getStorageRepoStats: @@ -2419,18 +2419,16 @@ # # Get information about the current virtual machines. # -# @fullStatus: #optional Set to request complete VM information -# # @vmList: #optional Filter the results by a list of UUIDs # # Returns: -# A list of VM definitions +# A list of VMs # # Since: 4.10.0 ## {'command': {'class': 'Host', 'name': 'getVMList'}, - 'data': {'*fullStatus': 'bool', '*vmList': ['UUID']}, - 'returns': ['VmInfo']} + 'data': {'*vmList': ['UUID']}, + 'returns': ['VM']} ## # @Host.ping: @@ -2810,18 +2808,19 @@ # # Image API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: The UUID of the Image +# @imageID: The UUID of the Image # -# @spUUID: The UUID of the Storage Pool associated with the Image +# @storagepoolID: The UUID of the Storage Pool associated with the Image # -# @sdUUID: The UUID of the Storage Domain associated with the Image +# @storagedomainID: The UUID of the Storage Domain associated with the Image # # Since: 4.10.0 ## {'class': 'Image', - 'data': {'conn': 'Host', 'UUID': 'UUID', 'spUUID': 'UUID', 'sdUUID': 'UUID'}} + 'data': {'conn': 'Host', 'imageID': 'UUID', 'storagepoolID': 'UUID', + 'storagedomainID': 'UUID'}} ## # @Image.delete: @@ -2867,12 +2866,12 @@ # Get a list of Volumes associated with this Image. # # Returns: -# A list of Image UUIDs +# A list of Volumes # # Since: 4.10.0 ## {'command': {'class': 'Image', 'name': 'getVolumes'}, - 'returns': ['UUID']} + 'returns': ['Volume']} ## # @Image.mergeSnapshots: @@ -2938,13 +2937,15 @@ # # LVMVolumeGroup API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: #optional Associate this object with an existing LVM Volume Group +# @lvmvolumegroupID: #optional Associate this object with an existing LVM +# Volume Group # # Since: 4.10.0 ## -{'class': 'LVMVolumeGroup', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'LVMVolumeGroup', + 'data': {'conn': 'Host', 'lvmvolumegroupID': 'UUID'}} ## # @LVMVolumeGroup.create: @@ -2997,16 +2998,17 @@ # # StorageDomain API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with a new or existing Storage Domain +# @storagedomainID: Associate this object with a new or existing Storage Domain # -# @spUUID: #optional The Storage Pool UUID if this Storage Domain is attached +# @storagepoolID: #optional The Storage Pool UUID if this Storage Domain is +# attached # # Since: 4.10.0 ## {'class': 'StorageDomain', - 'data': {'conn': 'Host', 'UUID': 'UUID', '*spUUID': 'UUID'}} + 'data': {'conn': 'Host', 'storagedomainID': 'UUID', 'storagepoolID': 'UUID'}} ## # @StorageDomain.activate: @@ -3022,12 +3024,12 @@ # # Attach a Storage Domain to a Storage Pool. # -# @spUUID: The Storage Pool to which the Storage Domain should be attached +# @storagepoolID: The UUID of the pool to which this domain should be attached # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'attach'}, - 'data': {'spUUID': 'UUID'}} + 'data': {'storagepoolID': 'UUID'}} ## # @StorageDomainCreateArgumentsBlock: @@ -3209,12 +3211,12 @@ # Get a list of Images associated with this Storage Domain. # # Returns: -# An array of Image UUIDs +# An array of Imags # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'getImages'}, - 'returns': ['UUID']} + 'returns': ['Image']} ## # @StorageDomainRole: @@ -3316,16 +3318,16 @@ # # Get a list of Volumes contained within a Storage Domain. # -# @imgUUID: Limit results to Volumes associated with a single Image +# @imageID: Limit results to Volumes associated with a single Image # # Returns: -# A list of Volume UUIDs +# A list of Volumes # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'getVolumes'}, - 'data': {'imgUUID': 'UUID'}, - 'returns': ['UUID']} + 'data': {'imageID': 'UUID'}, + 'returns': ['Volume']} ## # @StorageDomain.setDescription: @@ -3357,21 +3359,21 @@ # # Upload a Volume into a Storage Domain from a remote location. # -# @imgUUID: The UUID of the image that is associated with the Volume +# @imageID: The UUID of the image that is associated with the Volume # -# @volUUID: The UUID of an existing Volume where the data will be uploaded +# @volumeID: The UUID of an existing Volume where the data will be uploaded # -# @srcPath: The remote location of the Volume data. Must be in a format that -# can be understood by the command indicated in @method. +# @srcPath: The remote location of the Volume data. Must be in a format that +# can be understood by the command indicated in @method. # -# @size: The size of the volume data in blocks +# @size: The size of the volume data in blocks # -# @method: The method (command) to use to upload the data +# @method: The method (command) to use to upload the data # # Since: 4.10.0 ## {'command': {'class': 'StorageDomain', 'name': 'uploadVolume'}, - 'data': {'imgUUID': 'UUID', 'volUUID': 'UUID', 'srcPath': 'str', + 'data': {'imageID': 'UUID', 'volumeID': 'UUID', 'srcPath': 'str', 'size': 'int', 'method': 'UploadVolumeMethod'}} ## @@ -3389,13 +3391,13 @@ # # StoragePool API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with a new or existing Storage Pool +# @storagepoolID: Associate this object with a new or existing Storage Pool # # Since: 4.10.0 ## -{'class': 'StoragePool', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'StoragePool', 'data': {'conn': 'Host', 'storagepoolID': 'UUID'}} ## # @StoragePool.connect: @@ -3600,7 +3602,7 @@ # # Get information about backed-up virtual machines from a Backup Storage Domain. # -# @sdUUID: The UUID of the Backup Storage Domain to check +# @storagedomainID: The UUID of the Backup Storage Domain to check # # @vmList: Limit results to a list of VM UUIDs # @@ -3613,7 +3615,7 @@ # previously injected it into the Backup Storage Domain. ## {'command': {'class': 'StoragePool', 'name': 'getBackedUpVmsInfo'}, - 'data': {'sdUUID': 'UUID', 'vmList': ['UUID']}, + 'data': {'storagedomainID': 'UUID', 'vmList': ['UUID']}, 'returns': 'OVFMap'} ## @@ -3621,7 +3623,7 @@ # # Get a list of backed up virtual machines from a Backup Storage Domain. # -# @sdUUID: The UUID of the Backup Storage Domain to check +# @storagedomainID: The UUID of the Backup Storage Domain to check # # Returns: # A list of VM UUIDs @@ -3629,7 +3631,7 @@ # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'getBackedUpVmsList'}, - 'data': {'sdUUID': 'UUID'}, + 'data': {'storagedomainID': 'UUID'}, 'returns': ['UUID']} ## @@ -3650,18 +3652,18 @@ # # Get a list of Storage Domains that contain an Image. # -# @imgUUID: The UUID of the Image to search for +# @imageID: The UUID of the Image to search for # # @onlyDataDomains: #optional Only include Data Storage Domains # # Returns: -# A list of Storage Domain UUIDs that contain the Image +# A list of Storage Domains that contain the Image # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'getDomainsContainingImage'}, - 'data': {'imgUUID': 'UUID', '*onlyDataDomains': 'bool'}, - 'returns': ['UUID']} + 'data': {'imageID': 'UUID', '*onlyDataDomains': 'bool'}, + 'returns': ['StorageDomain']} ## # @StoragePool.getIsoList: @@ -4065,26 +4067,26 @@ # # @vmList: A list of virtual machine definitions to store # -# @sdUUID: The Storage Domain to use for storing the VM definitions +# @storagedomainID: The Storage Domain to use for storing the VM definitions # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'updateVMs'}, - 'data': {'vmList': ['UpdateVmDefinition'], 'sdUUID': 'UUID'}} + 'data': {'vmList': ['UpdateVmDefinition'], 'storagedomainID': 'UUID'}} ## # @StoragePool.removeVM: # # Remove a previously saved virtual machine definition. # -# @vmUUID: Remove the saved definition of the VM with this UUID +# @vmUUID: Remove the saved definition of the VM with this UUID # -# @sdUUID: The Storage Domain where the VM is stored +# @storagedomainID: The Storage Domain where the VM is stored # # Since: 4.10.0 ## {'command': {'class': 'StoragePool', 'name': 'removeVM'}, - 'data': {'vmUUID': 'UUID', 'sdUUID': 'UUID'}} + 'data': {'vmUUID': 'UUID', 'storagedomainID': 'UUID'}} ## Category: @Task ############################################################# ## @@ -4092,13 +4094,13 @@ # # Task API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: Associate this object with an existing Task +# @taskID: Associate this object with an existing Task # # Since: 4.10.0 ## -{'class': 'Task', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'Task', 'data': {'conn': 'Host', 'taskID': 'UUID'}} ## # @Task.clear: @@ -4161,11 +4163,11 @@ # # @conn: A connected base API object # -# @UUID: Associate this object with an existing VM +# @vmID: Associate this object with an existing VM # # Since: 4.10.0 ## -{'class': 'VM', 'data': {'conn': 'Host', 'UUID': 'UUID'}} +{'class': 'VM', 'data': {'conn': 'Host', 'vmID': 'UUID'}} ## # @DriveSpecVolume: @@ -4387,6 +4389,20 @@ # Since: 4.10.0 ## {'command': {'class': 'VM', 'name': 'destroy'}} + +## +# @VM.getInfo: +# +# Get detailed information about a VM. +# +# Returns: +# VM information +# +# Since: 4.10.0 +## +{'command': {'class': 'VM', 'name': 'getInfo'}, + 'returns': 'VmFullStatus'} + ## # @VM.getMigrationStatus: @@ -5179,21 +5195,21 @@ # # Volume API object. # -# @conn: A connected base API object +# @conn: A connected base API object # -# @UUID: The UUID of the Volume +# @volumeID: The UUID of the Volume # -# @spUUID: The Storage Pool associated with @UUID +# @storagepoolID: The Storage Pool associated with @UUID # -# @sdUUID: The Storage Domain associated with @UUID +# @storagedomainID: The Storage Domain associated with @UUID # -# @imgUUID: The Image associated with @UUID +# @imageID: The Image associated with @UUID # # Since: 4.10.0 ## {'class': 'Volume', - 'data': {'conn': 'Host', 'UUID': 'UUID', 'spUUID': 'UUID', 'sdUUID': 'UUID', - 'imgUUID': 'UUID'}} + 'data': {'conn': 'Host', 'volumeID': 'UUID', 'storagepoolID': 'UUID', + 'storagedomainID': 'UUID', 'imageID': 'UUID'}} ## # @VolumeRole: -- To view, visit http://gerrit.ovirt.org/7840 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7fdfe69436d65ef8190d8a5f1d941e7a628c77c0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

3 4

Change in vdsm[master]: schema: Remove VmFullStatus
by agl＠us.ibm.com 07 Nov '12

07 Nov '12

Adam Litke has uploaded a new change for review. Change subject: schema: Remove VmFullStatus ...................................................................... schema: Remove VmFullStatus The VmFullStatus alias is a meaningless indirection that simply causes confusion to API users. In all cases, its use just means VmDefinition so use that type instead. Change-Id: Id16979a105521e3c059ba27029493b71158f8815 Signed-off-by: Adam Litke <agl(a)us.ibm.com> --- M vdsm_api/vdsmapi-schema.json 1 file changed, 2 insertions(+), 11 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/8613/1 diff --git a/vdsm_api/vdsmapi-schema.json b/vdsm_api/vdsmapi-schema.json index 5d4e10f..3f8f0a0 100644 --- a/vdsm_api/vdsmapi-schema.json +++ b/vdsm_api/vdsmapi-schema.json @@ -2394,15 +2394,6 @@ 'vmType': 'VmType'}} ## -# @VmFullStatus: -# -# A VmFullStatus uses the same format as a VmDefinition. -# -# Since: 4.10.0 -## -{'alias': 'VmFullStatus', 'data': 'VmDefinition'} - -## # @VmInfo: # # A discriminated record of VM status information. @@ -2413,7 +2404,7 @@ ## {'type': 'VmInfo', 'data': {}, - 'union': ['VmFullStatus', 'VmShortStatus']} + 'union': ['VmDefinition', 'VmShortStatus']} ## # @Host.getVMList: @@ -4910,7 +4901,7 @@ # Since: 4.10.0 ## {'type': 'MigrationCreateState', - 'data': {'migrationPort': 'int', 'params': 'VmFullStatus'}} + 'data': {'migrationPort': 'int', 'params': 'VmDefinition'}} ## # @VM.migrationCreate: -- To view, visit http://gerrit.ovirt.org/8613 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id16979a105521e3c059ba27029493b71158f8815 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Adam Litke <agl(a)us.ibm.com>

4 4

Change in vdsm[master]: fix REMOVED_IMAGE_PREFIX not defined error
by zhshzhou＠linux.vnet.ibm.com 07 Nov '12

07 Nov '12

Zhou Zheng Sheng has uploaded a new change for review. Change subject: fix REMOVED_IMAGE_PREFIX not defined error ...................................................................... fix REMOVED_IMAGE_PREFIX not defined error Recently http://gerrit.ovirt.org/#/c/8506 is merged, it moves REMOVED_IMAGE_PREFIX = "_remove_me_" from vdsm/storage/image.py to vdsm/storage/sd.py. So all occurrence of REMOVED_IMAGE_PREFIX in image.py must be changed to sd.REMOVED_IMAGE_PREFIX, otherwise pyflakes will say REMOVED_IMAGE_PREFIX not defined. Change-Id: Ifc6ec49ff20c7f8a18ae14fe205570dfc6ce7634 Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/storage/image.py 1 file changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/98/9098/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 6ae5908..4cd9785 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -234,7 +234,7 @@ # Otherwise move it out of the way newImgUUID = self.deletedVolumeName(imgUUID) self.log.info("Rename image %s -> %s", imgUUID, newImgUUID) - if not imgUUID.startswith(REMOVED_IMAGE_PREFIX): + if not imgUUID.startswith(sd.REMOVED_IMAGE_PREFIX): removedImage = os.path.join(os.path.dirname(imageDir), newImgUUID) os.rename(imageDir, removedImage) else: @@ -242,7 +242,7 @@ volumes = [volclass(self.repoPath, sdUUID, newImgUUID, volUUID) for volUUID in uuidlist] for vol in volumes: - if not vol.volUUID.startswith(REMOVED_IMAGE_PREFIX): + if not vol.volUUID.startswith(sd.REMOVED_IMAGE_PREFIX): vol.rename(self.deletedVolumeName(vol.volUUID), recovery=False) else: self.log.warning("Volume %s of image %s already renamed", vol.volUUID, imgUUID) -- To view, visit http://gerrit.ovirt.org/9098 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ifc6ec49ff20c7f8a18ae14fe205570dfc6ce7634 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

3 3

Re: [oVirt Jenkins] vdsm_unit_tests - Build # 1502 - Still Failing!
by Dan Kenigsberg 07 Nov '12

07 Nov '12

On Tue, Nov 06, 2012 at 12:41:42PM -0500, Jenkins oVirt Server wrote: > > BUILD FAILURE > Build URL: http://jenkins.ovirt.org/job/vdsm_unit_tests/1502/ > Project: vdsm_unit_tests > Date of build: Tue, 06 Nov 2012 12:41:22 -0500 > Build duration: 20 sec Eyal, please take a look at this machine: it seems that /home/jenkins/workspace/vdsm_unit_tests/ has some trash remainder, an old vdsm/pthread.pyc. Would you make sure that the directory is clean before checking out from git? Dan.

2 1

Change in vdsm[master]: Make utils pep8 compliant
by smizrahi＠redhat.com 07 Nov '12

07 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Make utils pep8 compliant ...................................................................... Make utils pep8 compliant Change-Id: I514660d73071f8fb417fea0cc54973a55002992f Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M Makefile.am M vdsm/utils.py 2 files changed, 117 insertions(+), 69 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/9092/1 diff --git a/Makefile.am b/Makefile.am index bd262af..1aa6e86 100644 --- a/Makefile.am +++ b/Makefile.am @@ -97,6 +97,7 @@ vdsm/supervdsm.py \ vdsm/supervdsmServer.py \ vdsm/tc.py \ + vdsm/utils.py \ vdsm/vdsm \ vdsm/vdsmDebugPlugin.py \ vdsm/vm.py \ diff --git a/vdsm/utils.py b/vdsm/utils.py index 002094a..b4614e7 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -19,7 +19,8 @@ # """ -A module containing miscellaneous functions and classes that are user plentifuly around vdsm. +A module containing miscellaneous functions and classes that are user +plentifuly around vdsm. .. attribute:: utils.symbolerror @@ -28,7 +29,8 @@ from SimpleXMLRPCServer import SimpleXMLRPCServer import SocketServer import threading -import os, time +import os +import time import logging import errno import subprocess @@ -47,6 +49,7 @@ if not os.path.exists(_THP_STATE_PATH): _THP_STATE_PATH = '/sys/kernel/mm/redhat_transparent_hugepage/enabled' + def isBlockDevice(path): path = os.path.abspath(path) return stat.S_ISBLK(os.stat(path).st_mode) @@ -63,6 +66,7 @@ os.unlink(fileToRemove) except: pass + def readMemInfo(): """ @@ -92,9 +96,12 @@ raise time.sleep(0.1) + #Threaded version of SimpleXMLRPCServer -class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, SimpleXMLRPCServer): +class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, + SimpleXMLRPCServer): allow_reuse_address = True + class InterfaceSample: """ @@ -123,7 +130,8 @@ try: s = file(f).read() except IOError, e: - if e.errno != errno.ENOENT: # silently ignore missing wifi stats + # silently ignore missing wifi stats + if e.errno != errno.ENOENT: logging.debug("Could not read %s", f, exc_info=True) return 0 try: @@ -131,7 +139,7 @@ except: if s != '': logging.warning("Could not parse statistics (%s) from %s", - f, s, exc_info=True) + f, s, exc_info=True) logging.debug('bad %s: (%s)', f, s) if not tries: raise @@ -140,7 +148,8 @@ """ Return the operational state of the interface. - :returns: ``'up'`` if interface is up. ``'down'`` or ``0`` id it's down. + :returns: ``'up'`` if interface is up. ``'down'`` or ``0`` id it's + down. """ try: flags = ethtool.get_flags(ifid) @@ -166,8 +175,9 @@ """ def __init__(self): self.user, userNice, self.sys, self.idle = \ - map(int, file('/proc/stat').readline().split()[1:5]) + map(int, file('/proc/stat').readline().split()[1:5]) self.user += userNice + class PidCpuSample: """ @@ -177,11 +187,13 @@ """ def __init__(self, pid): self.user, self.sys = \ - map(int, file('/proc/%s/stat' % pid).read().split()[13:15]) + map(int, file('/proc/%s/stat' % pid).read().split()[13:15]) + class TimedSample: def __init__(self): self.timestamp = time.time() + class BaseSample(TimedSample): """ @@ -189,10 +201,11 @@ """ def __init__(self, pid, ifids): TimedSample.__init__(self) - self.interfaces= {} + self.interfaces = {} for ifid in ifids: self.interfaces[ifid] = InterfaceSample(ifid) self.pidcpu = PidCpuSample(pid) + class HostSample(BaseSample): """ @@ -200,7 +213,8 @@ Contains the sate of the host in the time of initialization. """ - MONITORED_PATHS = ['/tmp', '/var/log', '/var/log/core', constants.P_VDSM_RUN] + MONITORED_PATHS = ['/tmp', '/var/log', '/var/log/core', + constants.P_VDSM_RUN] def _getDiskStats(self): d = {} @@ -208,7 +222,7 @@ free = 0 try: stat = os.statvfs(p) - free = stat.f_bavail * stat.f_bsize / 2**20 + free = stat.f_bavail * stat.f_bsize / (2 ** 20) except: pass d[p] = {'free': str(free)} @@ -238,9 +252,10 @@ try: with file(_THP_STATE_PATH) as f: s = f.read() - self.thpState = s[s.index('[')+1:s.index(']')] + self.thpState = s[s.index('[') + 1:s.index(']')] except: self.thpState = 'never' + class AdvancedStatsFunction(object): """ @@ -265,8 +280,8 @@ return self._interval def __repr__(self): - return "<AdvancedStatsFunction %s at 0x%x>" % \ - (self._function.__name__, id(self._function.__name__)) + return "<AdvancedStatsFunction %s at 0x%x>" % ( + self._function.__name__, id(self._function.__name__)) def __call__(self, *args, **kwargs): retValue = self._function(*args, **kwargs) @@ -284,12 +299,14 @@ the first and the last return value in the defined 'window' and the time difference. """ - if len(self._sample) < 2: return None, None, None + if len(self._sample) < 2: + return None, None, None bgn_time, bgn_sample = self._sample[0] end_time, end_sample = self._sample[-1] return bgn_sample, end_sample, (end_time - bgn_time) + class AdvancedStatsThread(threading.Thread): """ @@ -386,8 +403,8 @@ waitInterval = maxInterval for statsFunction in self._statsFunctions: - thisInterval = statsFunction.interval - \ - (intervalAccum % statsFunction.interval) + thisInterval = statsFunction.interval - ( + intervalAccum % statsFunction.interval) waitInterval = min(waitInterval, thisInterval) if intervalAccum % statsFunction.interval == 0: @@ -401,6 +418,7 @@ self._stopEvent.wait(waitInterval) intervalAccum = (intervalAccum + waitInterval) % maxInterval + class StatsThread(threading.Thread): """ A thread that periodically checks the stats of interfaces @@ -408,6 +426,7 @@ AVERAGING_WINDOW = 5 SAMPLE_INTERVAL_SEC = 2 MBITTOBYTES = 1000000 / 8 + def __init__(self, log, ifids, ifrates, ifmacs): threading.Thread.__init__(self) self._log = log @@ -418,7 +437,8 @@ self._ifrates = ifrates self._ifmacs = ifmacs self._ncpus = 1 - self._lineRate = (sum(ifrates) or 1000) * 10**6 / 8 # in bytes-per-second + # in bytes-per-second + self._lineRate = (sum(ifrates) or 1000) * (10 ** 6) / 8 self._paused = False self._lastSampleTime = time.time() @@ -431,7 +451,7 @@ def cont(self): self._paused = False - def sample(self): # override + def sample(self): # override pass def run(self): @@ -449,7 +469,7 @@ self._samples.pop(0) except libvirtvm.TimeoutError: self._log.error("Timeout while sampling stats", - exc_info=True) + exc_info=True) self._stopEvent.wait(self.SAMPLE_INTERVAL_SEC) except: if not self._stopEvent.isSet(): @@ -480,27 +500,33 @@ for ifid, ifrate, ifmac in zip(self._ifids, self._ifrates, self._ifmacs): ifrate = ifrate or 1000 - Mbps2Bps = 10**6 / 8 - thisRx = (hs1.interfaces[ifid].rx - hs0.interfaces[ifid].rx) % 2**32 - thisTx = (hs1.interfaces[ifid].tx - hs0.interfaces[ifid].tx) % 2**32 + Mbps2Bps = (10 ** 6) / 8 + thisRx = (hs1.interfaces[ifid].rx - hs0.interfaces[ifid].rx) % \ + (2 ** 32) + thisTx = (hs1.interfaces[ifid].tx - hs0.interfaces[ifid].tx) % \ + (2 ** 32) rxRate = 100.0 * thisRx / interval / ifrate / Mbps2Bps txRate = 100.0 * thisTx / interval / ifrate / Mbps2Bps if txRate > 100 or rxRate > 100: txRate = min(txRate, 100.0) rxRate = min(rxRate, 100.0) - self._log.debug('Rate above 100%%. DEBUG: ifid %s interval: %s thisRx %s thisTx %s samples %s', ifid, interval, thisRx, thisTx, [(hs.timestamp, hs.interfaces[ifid].rx, hs.interfaces[ifid].tx) for hs in self._samples]) + self._log.debug('Rate above 100%%. DEBUG: ifid %s interval: ' + '%s thisRx %s thisTx %s samples %s', ifid, + interval, thisRx, thisTx, + [(hs.timestamp, hs.interfaces[ifid].rx, + hs.interfaces[ifid].tx) + for hs in self._samples]) + iface = hs1.interfaces[ifid] stats['network'][ifid] = {'name': ifid, 'speed': str(ifrate), - 'rxDropped': str(hs1.interfaces[ifid].rxDropped), - 'txDropped': str(hs1.interfaces[ifid].txDropped), - 'rxErrors': str(hs1.interfaces[ifid].rxErrors), - 'txErrors': str(hs1.interfaces[ifid].txErrors), - 'state': hs1.interfaces[ifid].operstate, - 'rxRate': '%.1f' % rxRate, - 'txRate': '%.1f' % txRate, - 'macAddr': ifmac, -# 'interval': str(interval), 'thisRx': str(thisRx), 'thisTx': str(thisTx), -# 'samples': str([(hs.timestamp, hs.interfaces[ifid].rx, hs.interfaces[ifid].tx) for hs in self._samples]) - } + 'rxDropped': str(iface.rxDropped), + 'txDropped': str(iface.txDropped), + 'rxErrors': str(iface.rxErrors), + 'txErrors': str(iface.txErrors), + 'state': iface.operstate, + 'rxRate': '%.1f' % rxRate, + 'txRate': '%.1f' % txRate, + 'macAddr': ifmac, + } rx += thisRx tx += thisTx rxDropped += hs1.interfaces[ifid].rxDropped @@ -518,6 +544,7 @@ return stats + class HostStatsThread(StatsThread): """ A thread that periodically samples host statistics. @@ -525,7 +552,7 @@ def __init__(self, cif, log, ifids, ifrates): self.startTime = time.time() StatsThread.__init__(self, log, ifids, ifrates, - [''] * len(ifids)) # fake ifmacs + [''] * len(ifids)) # fake ifmacs self._imagesStatus = ImagePathStatus(cif) self._pid = os.getpid() self._ncpus = len(os.listdir('/sys/class/cpuid/')) @@ -540,8 +567,8 @@ i = 0 for ifid in self._ifids: - if hs0.interfaces[ifid].operstate != \ - hs1.interfaces[ifid].operstate: + if (hs0.interfaces[ifid].operstate != + hs1.interfaces[ifid].operstate): self._ifrates[i] = nicspeed(ifid) i += 1 @@ -560,25 +587,25 @@ now = time.time() for sd, d in self._imagesStatus.storageDomains.iteritems(): stats['storageDomains'][sd] = {'code': d['code'], - 'delay': d['delay'], - 'lastCheck': d['lastCheck'], - 'valid': d['valid']} + 'delay': d['delay'], + 'lastCheck': d['lastCheck'], + 'valid': d['valid']} stats['elapsedTime'] = int(now - self.startTime) if len(self._samples) < 2: return stats hs0, hs1 = self._samples[0], self._samples[-1] interval = hs1.timestamp - hs0.timestamp - jiffies = (hs1.pidcpu.user - hs0.pidcpu.user) % 2**32 - stats['cpuUserVdsmd'] = (jiffies / interval) % 2**32 + jiffies = (hs1.pidcpu.user - hs0.pidcpu.user) % (2 ** 32) + stats['cpuUserVdsmd'] = (jiffies / interval) % (2 ** 32) jiffies = hs1.pidcpu.sys - hs0.pidcpu.sys - stats['cpuSysVdsmd'] = (jiffies / interval) % 2**32 + stats['cpuSysVdsmd'] = (jiffies / interval) % (2 ** 32) - jiffies = (hs1.totcpu.user - hs0.totcpu.user) % 2**32 + jiffies = (hs1.totcpu.user - hs0.totcpu.user) % (2 ** 32) stats['cpuUser'] = jiffies / interval / self._ncpus - jiffies = (hs1.totcpu.sys - hs0.totcpu.sys) % 2**32 + jiffies = (hs1.totcpu.sys - hs0.totcpu.sys) % (2 ** 32) stats['cpuSys'] = jiffies / interval / self._ncpus stats['cpuIdle'] = max(0.0, - 100.0 - stats['cpuUser'] - stats['cpuSys']) + 100.0 - stats['cpuUser'] - stats['cpuSys']) stats['memUsed'] = hs1.memUsed stats['anonHugePages'] = hs1.anonHugePages stats['cpuLoad'] = hs1.cpuLoad @@ -587,7 +614,8 @@ stats['thpState'] = hs1.thpState return stats -def convertToStr (val): + +def convertToStr(val): varType = type(val) if varType is float: return '%.2f' % (val) @@ -596,10 +624,12 @@ else: return val + def execCmd(*args, **kwargs): # import only after config as been initialized from storage.misc import execCmd return execCmd(*args, **kwargs) + def checkPathStat(pathToCheck): try: @@ -610,8 +640,9 @@ except: return (False, 0) + class ImagePathStatus(threading.Thread): - def __init__ (self, cif, interval=None): + def __init__(self, cif, interval=None): if interval is None: interval = config.getint('irs', 'images_check_times') self._interval = interval @@ -622,7 +653,7 @@ if self._interval > 0: self.start() - def stop (self): + def stop(self): self._stopEvent.set() def _refreshStorageDomains(self): @@ -631,7 +662,7 @@ if "args" in self.storageDomains: del self.storageDomains["args"] - def run (self): + def run(self): try: while not self._stopEvent.isSet(): if self._cif.irs: @@ -639,7 +670,8 @@ self._stopEvent.wait(self._interval) except: logging.error("Error while refreshing storage domains", - exc_info=True) + exc_info=True) + def getPidNiceness(pid): """ @@ -651,9 +683,10 @@ stat = file('/proc/%s/stat' % (pid)).readlines()[0] return int(stat.split(') ')[-1].split()[16]) + def tobool(s): try: - if s == None: + if s is None: return False if type(s) == bool: return s @@ -662,6 +695,7 @@ return bool(int(s)) except: return False + def _getAllMacs(): @@ -689,6 +723,8 @@ return set(macs) - set(["", "00:00:00:00:00:00"]) __hostUUID = '' + + def getHostUUID(): global __hostUUID if __hostUUID: @@ -702,12 +738,14 @@ __hostUUID = f.readline().replace("\n", "") else: p = subprocess.Popen([constants.EXT_SUDO, - constants.EXT_DMIDECODE, "-s", "system-uuid"], + constants.EXT_DMIDECODE, "-s", + "system-uuid"], close_fds=True, stdin=subprocess.PIPE, - stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) out, err = p.communicate() - out = '\n'.join( line for line in out.splitlines() - if not line.startswith('#') ) + out = '\n'.join(line for line in out.splitlines() + if not line.startswith('#')) if p.returncode == 0 and 'Not' not in out: #Avoid error string - 'Not Settable' or 'Not Present' @@ -734,6 +772,7 @@ for code, symbol in errno.errorcode.iteritems(): symbolerror[os.strerror(code)] = symbol + def getUserPermissions(userName, path): """ Return a dictionary with user specific permissions with respect to the @@ -759,19 +798,20 @@ isSameOwner = userInfo.pw_uid == fileStats.st_uid # 'Other' permissions are the base permissions - permissions['read'] = isRead(otherBits) or \ - isSameGroup and isRead(groupBits) or \ - isSameOwner and isRead(ownerBits) + permissions['read'] = (isRead(otherBits) or + isSameGroup and isRead(groupBits) or + isSameOwner and isRead(ownerBits)) - permissions['write'] = isWrite(otherBits) or \ - isSameGroup and isWrite(groupBits) or \ - isSameOwner and isWrite(ownerBits) + permissions['write'] = (isWrite(otherBits) or + isSameGroup and isWrite(groupBits) or + isSameOwner and isWrite(ownerBits)) - permissions['exec'] = isExec(otherBits) or \ - isSameGroup and isExec(groupBits) or \ - isSameOwner and isExec(ownerBits) + permissions['exec'] = (isExec(otherBits) or + isSameGroup and isExec(groupBits) or + isSameOwner and isExec(ownerBits)) return permissions + def listSplit(l, elem, maxSplits=None): splits = [] @@ -783,13 +823,14 @@ except ValueError: break - splits.append( l[:splitOffset] ) - l = l[splitOffset+1:] - splitCount +=1 + splits.append(l[:splitOffset]) + l = l[splitOffset + 1:] + splitCount += 1 if maxSplits is not None and splitCount >= maxSplits: break return splits + [l] + def listJoin(elem, *lists): if lists == []: @@ -799,9 +840,11 @@ l += [elem] + i return l + def closeOnExec(fd): old = fcntl.fcntl(fd, fcntl.F_GETFD, 0) fcntl.fcntl(fd, fcntl.F_SETFD, old | fcntl.FD_CLOEXEC) + class memoized(object): """Decorator that caches a function's return value each time it is called. @@ -815,6 +858,7 @@ self.func = func self.cache = {} functools.update_wrapper(self, func) + def __call__(self, *args): try: return self.cache[args] @@ -822,14 +866,17 @@ value = self.func(*args) self.cache[args] = value return value + def __get__(self, obj, objtype): """Support instance methods.""" return functools.partial(self.__call__, obj) + def validateMinimalKeySet(dictionary, reqParams): if not all(key in dictionary for key in reqParams): raise ValueError + class CommandPath(object): def __init__(self, name, *args): self.name = name -- To view, visit http://gerrit.ovirt.org/9092 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I514660d73071f8fb417fea0cc54973a55002992f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

4 4

Change in vdsm[master]: BZ#836161 - Refactored deleteImage.
by ewarszaw＠redhat.com 07 Nov '12

07 Nov '12

Eduardo has uploaded a new change for review. Change subject: BZ#836161 - Refactored deleteImage. ...................................................................... BZ#836161 - Refactored deleteImage. Change-Id: I304ff5cd70186ffc9789cd1ac9337efa6c5ff695 Signed-off-by: Eduardo <ewarszaw(a)redhat.com> --- M vdsm/storage/blockSD.py M vdsm/storage/fileSD.py M vdsm/storage/hsm.py M vdsm/storage/sd.py 4 files changed, 142 insertions(+), 14 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/06/8506/1 diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py index e596fb6..d26e53c 100644 --- a/vdsm/storage/blockSD.py +++ b/vdsm/storage/blockSD.py @@ -163,13 +163,88 @@ res[vName]['parent'] = vPar if vImg not in res[vName]['imgs']: res[vName]['imgs'].insert(0, vImg) - if vPar != sd.BLANK_UUID and \ - not vName.startswith(blockVolume.image.REMOVED_IMAGE_PREFIX) \ + if vPar != sd.BLANK_UUID \ + and not vName.startswith(sd.REMOVED_IMAGE_PREFIX) \ + and not vName.startswith(sd.ZERO_ME_IMAGE_PREFIX) \ and vImg not in res[vPar]['imgs']: res[vPar]['imgs'].append(vImg) return dict((k, sd.ImgsPar(tuple(v['imgs']), v['parent'])) for k, v in res.iteritems()) + + +def deleteVolumes(sdUUID, vols): + lvm.removeLVs(sdUUID, vols) + + +def _zeroVolume(sdUUID, volUUID): + """ + This function requires an active LV. + """ + # Change for zero 128 M chuncks and log. + # 128 M is the vdsm extent size default + cmd = tuple(constants.EXT_DD, "if=/dev/zero", + "of=%s" % os.path.join("/dev", sdUUID, volUUID)) + p = misc.execCmd(cmd, sudo=False, sync=False) + return p + + +def _getZerosFinished(zeroPs): + for volUUID, zeroProc in zeroPs.items(): + finished = {} + if not zeroProc.wait(0): + continue + else: + zeroPs.pop(volUUID) + finished[volUUID] = zeroProc.returncode + if zeroProc.returncode != 0: + log.warning("zeroing %s failed. %s", volUUID, + zeroProc.stderr) + else: + log.debug("zeroing %s finished", volUUID) + return finished + + +def zeroImgVolumes(sdUUID, imgUUID, volUUIDs): + # Put a sensible value for dd zeroing a 128 M or 1 G chunk and lvremove + # spent time. + ZERO_SLEEP = 60 # [seconds] + # Prepare for zeroing + try: + lvm.changelv(sdUUID, volUUIDs, (("-a", "y"), ("--deltag", imgUUID), + ("--addtag", sd.ZERO_ME_IMAGE_PREFIX + imgUUID))) + except se.StorageException as e: + log.debug("SD %s, Image %s pre zeroing ops failed", sdUUID, imgUUID, + volUUIDs) + try: + lvm.changelv(sdUUID, volUUIDs, ("--permission", "rw")) + except se.StorageException as e: + # Hope this only means that some volumes were already writable. + log.debug("IGNORED: %s", e) + # blank the volumes. + zeroPs = {} + for volUUID in volUUIDs: + zeroPs[volUUID] = _zeroVolume(sdUUID, volUUID) + + # Yes, this is a potentially infinite loop. Kill the vdsm task. + last = time.time() + while len(zeroPs) > 0: + time.sleep(ZERO_SLEEP) + toDelete = [] + for volUUID, rc in _getZerosFinished(zeroPs): + if rc != 0: + log.warning("zero leftover: %s/%s rc = %s", sdUUID, volUUID, rc) + else: + toDelete.append(volUUID) + if toDelete: + deleteVolumes(sdUUID, toDelete) + + now = time.time() + if now - (last + 10 * ZERO_SLEEP) > 0: + log.warning("Still zeroing VG:%s LVs: %s", + sdUUID, zeroPs.iterkeys()) + + log.debug("All rm_zeroed VG:%s LVs: %s, img: %s", sdUUID, volUUIDs, imgUUID) class VGTagMetadataRW(object): @@ -827,6 +902,12 @@ if i.startswith(blockVolume.TAG_PREFIX_IMAGE) ] return images + def deleteImage(self, sdUUID, imgUUID, volsImgs): + return deleteVolumes(sdUUID, volsImgs) + + def zeroImage(self, sdUUID, imgUUID, volsImgs): + return zeroImgVolumes(sdUUID, imgUUID, volsImgs) + def getAllVolumes(self): """ Return all the images that depend on a volume. diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index 054fadb..d5b298f 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -62,6 +62,21 @@ return True +def getDomPath(sdUUID): + # /rhev/data-center/mnt/export-path/sdUUID/dom_md/metadata + # /rhev/data-center/mnt/blockSD/sdUUID/dom_md/metadata + pattern = os.path.join(sd.mountBasePath, '*', sdUUID) + # Warning! You need a global proc pool big as the number of NFS domains. + domPaths = getProcPool.glob.glob(pattern) + if len(domPaths) != 1: + raise se.StorageDomainLayoutError("domain", sdUUID) + return domPaths[0] + + +def getImagePath(sdUUID, imgUUID): + return os.path.join(getDomPath(sdUUID), 'images', imgUUID) + + def getDomUuidFromMetafilePath(metafile): # Metafile path has pattern: # /rhev/data-center/mnt/export-path/sdUUID/dom_md/metadata @@ -298,6 +313,31 @@ imgList.append(os.path.basename(i)) return imgList + def deleteImage(self, sdUUID, imgUUID, volsImgs): + oPath = getImagePath(sdUUID, imgUUID) + head, tail = self.oop.os.path.split(oPath) + nPath = os.tempnam(head, sd.REMOVED_IMAGE_PREFIX + tail) + try: + self.oop.os.rename(oPath, nPath) + except OSError as e: + self.log.error("image: %s can't be moved", oPath) + raise se.ImageDeleteError("%s %s" % (imgUUID, str(e))) + for volUUID in volsImgs: + volPath = os.path.join(nPath, volUUID) + try: + self.oop.os.remove(volPath) + self.oop.os.remove(volPath + '.meta') + except OSError as e: + self.log.error("vol: %s can't be removed.", volPath) + try: + self.oop.os.remove(nPath) + except OSError as e: + self.log.error("removed image dir: %s can't be removed", nPath) + raise se.ImageDeleteError("%s %s" % (imgUUID, str(e))) + + def zeroImage(self, sdUUID, imgUUID, volsImgs): + return self.deleteImage(sdUUID, imgUUID, volsImgs) + def getAllVolumes(self): """ Return dict {volUUID: ((imgUUIDs,), parentUUID)} of the domain. diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 78eef77..2a8bc59 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -1260,24 +1260,26 @@ Delete Image folder with all volumes """ #vars.task.setDefaultException(se.ChangeMeError("%s" % args)) - pool = self.getPool(spUUID) #Validates that the pool is connected. WHY? - self.validateSdUUID(sdUUID) + self.getPool(spUUID) #Validates that the pool is connected. WHY? + dom = self.validateSdUUID(sdUUID) vars.task.getExclusiveLock(STORAGE, imgUUID) vars.task.getSharedLock(STORAGE, sdUUID) + allVols = dom.getAllVolumes() + imgsByVol = sd.getVolsOfImage(allVols, imgUUID) # Do not validate if forced. - if not misc.parseBool(force): - pool.validateDelete(sdUUID, imgUUID) - # Rename image if postZero and perform delete as async operation - # else delete image in sync. stage + if not misc.parseBool(force) and not dom.isBackup() \ + and not all(len(v.imgs) == 1 for v in imgsByVol.itervalues()): + msg = "Cannot delete shared image %s. volImgs: %s" \ + % (imgUUID, imgsByVol) + raise se.CannotDeleteSharedVolume(msg) + + # zeroImage will delete zeroed volumes at the end. if misc.parseBool(postZero): - newImgUUID = pool.preDeleteRename(sdUUID, imgUUID) - self._spmSchedule(spUUID, "deleteImage", pool.deleteImage, sdUUID, newImgUUID, - misc.parseBool(postZero), misc.parseBool(force) - ) + self._spmSchedule(spUUID, "zeroImage_%s" % imgUUID, dom.zeroImage, + sdUUID, imgUUID, imgsByVol.keys()) else: - pool.deleteImage(sdUUID, imgUUID, - misc.parseBool(postZero), misc.parseBool(force)) + dom.deleteImage(sdUUID, imgUUID, imgsByVol) # This is a hack to keep the interface consistent # We currently have race conditions in delete image, to quickly fix # this we delete images in the "synchronous" state. This only works diff --git a/vdsm/storage/sd.py b/vdsm/storage/sd.py index 55c5b12..fcb796c 100644 --- a/vdsm/storage/sd.py +++ b/vdsm/storage/sd.py @@ -134,12 +134,17 @@ ImgsPar = namedtuple("ImgsPar", "imgs,parent") ISO_IMAGE_UUID = '11111111-1111-1111-1111-111111111111' BLANK_UUID = '00000000-0000-0000-0000-000000000000' +REMOVED_IMAGE_PREFIX = "_remove_me_" +ZERO_ME_IMAGE_PREFIX = "_zero_me_" # Blocks used for each lease (valid on all domain types) LEASE_BLOCKS = 2048 UNICODE_MINIMAL_VERSION = 3 +storage_repository = config.get('irs', 'repository') +mountBasePath = os.path.join(storage_repository, DOMAIN_MNT_POINT) + def getVolsOfImage(allVols, imgUUID): """ Filter allVols dict for volumes related to imgUUID. -- To view, visit http://gerrit.ovirt.org/8506 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I304ff5cd70186ffc9789cd1ac9337efa6c5ff695 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>

8 37

Change in vdsm[master]: Remove rebundant plus between two strings in parenthesis
by shaohef＠linux.vnet.ibm.com 06 Nov '12

06 Nov '12

ShaoHe Feng has uploaded a new change for review. Change subject: Remove rebundant plus between two strings in parenthesis ...................................................................... Remove rebundant plus between two strings in parenthesis Python will join them automatically without plus Change-Id: Ie24a04bb2002a84179b9b281d0a5fa25c333bf10 Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com> --- M tests/miscTests.py M tests/persistentDictTests.py M tests/resourceManagerTests.py M vdsm/API.py M vdsm/storage/safelease.py M vdsm_reg/register-to-engine.py 6 files changed, 20 insertions(+), 20 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/8812/1 diff --git a/tests/miscTests.py b/tests/miscTests.py index 6339986..19ae5f8 100644 --- a/tests/miscTests.py +++ b/tests/miscTests.py @@ -182,7 +182,7 @@ self.assertEquals(list(misc.tmap(dummy, data)), data) def testErrMethod(self): - exceptionStr = ("It's time to kick ass and chew bubble gum... " + + exceptionStr = ("It's time to kick ass and chew bubble gum... " "and I'm all outta gum.") def dummy(arg): @@ -221,8 +221,8 @@ """ #Prepare prefix = "prefix" - stubContent = ('"Multiple exclamation marks", ' + - 'he went on, shaking his head, ' + + stubContent = ('"Multiple exclamation marks", ' + 'he went on, shaking his head, ' '"are a sure sign of a diseased mind."') # (C) Terry Pratchet - Small Gods dir = tempfile.mkdtemp() @@ -801,8 +801,8 @@ Test if method works when given a valid file. """ #create - writeData = ("Trust me, I know what self-loathing is," + - "but to kill myself? That would put a damper on my " + + writeData = ("Trust me, I know what self-loathing is," + "but to kill myself? That would put a damper on my " "search for answers. Not at all productive.") # (C) Jhonen Vasquez - Johnny the Homicidal Maniac fd, path = tempfile.mkstemp() diff --git a/tests/persistentDictTests.py b/tests/persistentDictTests.py index fe7c391..c223d67 100644 --- a/tests/persistentDictTests.py +++ b/tests/persistentDictTests.py @@ -24,7 +24,7 @@ class DummyFailWriter(object): def writelines(self, lines): - raise RuntimeError("You might have a very minor case of " + + raise RuntimeError("You might have a very minor case of " "serious brain damage") # (C) Valve - Portal 2 diff --git a/tests/resourceManagerTests.py b/tests/resourceManagerTests.py index 84708cf..7575bf4 100644 --- a/tests/resourceManagerTests.py +++ b/tests/resourceManagerTests.py @@ -197,7 +197,7 @@ except AttributeError: return except Exception as ex: - self.fail("Wrong exception was raised. " + + self.fail("Wrong exception was raised. " "Expected AttributeError got %s", ex.__class__.__name__) @@ -217,7 +217,7 @@ except AttributeError: return except Exception as ex: - self.fail("Wrong exception was raised. " + + self.fail("Wrong exception was raised. " "Expected AttributeError got %s", ex.__class__.__name__) finally: req.wait() diff --git a/vdsm/API.py b/vdsm/API.py index aa256a1..08dfafa 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -176,7 +176,7 @@ self.log.error('Missing required parameter %s' % (param)) return {'status': {'code': errCode['MissParam'] ['status']['code'], - 'message': 'Missing required ' + + 'message': 'Missing required ' 'parameter %s' % (param)}} try: storage.misc.validateUUID(vmParams['vmId']) @@ -359,7 +359,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, nic') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, nic'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -375,7 +375,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, nic') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, nic'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -391,7 +391,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, drive') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, drive'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -407,7 +407,7 @@ except ValueError: self.log.error('Missing one of required parameters: vmId, drive') return {'status': {'code': errCode['MissParam']['status']['code'], - 'message': 'Missing one of required ' + + 'message': 'Missing one of required ' 'parameters: vmId, drive'}} try: curVm = self._cif.vmContainer[self._UUID] @@ -1019,7 +1019,7 @@ cleantext += line return cleantext - self.log.debug('fenceNode(addr=%s,port=%s,agent=%s,user=%s,' + + self.log.debug('fenceNode(addr=%s,port=%s,agent=%s,user=%s,' 'passwd=%s,action=%s,secure=%s,options=%s)', addr, port, agent, username, 'XXXX', action, secure, options) @@ -1037,7 +1037,7 @@ return errCode['fenceAgent'] raise - inp = ('agent=fence_%s\nipaddr=%s\nlogin=%s\noption=%s\n' + + inp = ('agent=fence_%s\nipaddr=%s\nlogin=%s\noption=%s\n' 'passwd=%s\n') % (agent, addr, username, action, password) if port != '': inp += 'port=%s\n' % (port,) @@ -1120,7 +1120,7 @@ Doesn't survive a restart """ - logging.getLogger('clientIF.setLogLevel').info('Setting loglevel ' + + logging.getLogger('clientIF.setLogLevel').info('Setting loglevel ' 'to %s' % level) handlers = logging.getLogger().handlers [fileHandler] = [h for h in handlers if @@ -1205,7 +1205,7 @@ if vlan or bond or nics: # Backwards compatibility - self.log.warn('Specifying vlan, ' + + self.log.warn('Specifying vlan, ' 'bond or nics to delNetwork is deprecated') _netinfo = netinfo.NetInfo() try: @@ -1215,7 +1215,7 @@ configNetwork.validateVlanId(vlan) if nics and bond and set(nics) != \ set(_netinfo.bondings[bond]["slaves"]): - self.log.error('delNetwork: not all nics specified ' + + self.log.error('delNetwork: not all nics specified ' 'are enslaved (%s != %s)' % (nics, _netinfo.bondings[bond]["slaves"])) diff --git a/vdsm/storage/safelease.py b/vdsm/storage/safelease.py index 551dda5..88a4eae 100644 --- a/vdsm/storage/safelease.py +++ b/vdsm/storage/safelease.py @@ -120,7 +120,7 @@ (rc, out, err) = misc.execCmd(releaseLockCommand, sudo=False, cwd=self.lockUtilPath) if rc != 0: - self.log.error("Could not release cluster lock " + + self.log.error("Could not release cluster lock " "rc=%s out=%s, err=%s" % (str(rc), out, err)) self.log.debug("Cluster lock released successfully") diff --git a/vdsm_reg/register-to-engine.py b/vdsm_reg/register-to-engine.py index d397c84..6c6cc45 100755 --- a/vdsm_reg/register-to-engine.py +++ b/vdsm_reg/register-to-engine.py @@ -112,7 +112,7 @@ try: port = config.get('vars', 'vdc_host_port') except ConfigParser.NoOptionError: - sys.stderr.write("Failed to retrieve port number " + + sys.stderr.write("Failed to retrieve port number " "from config file: %s\n" % VDSM_REG_CONF_FILE) sys.exit(CONF_FILE_READ_ERROR) -- To view, visit http://gerrit.ovirt.org/8812 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie24a04bb2002a84179b9b281d0a5fa25c333bf10 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>

6 11

Change in vdsm[master]: Don't forget to close thread pool when stopping the SPM mail...
by smizrahi＠redhat.com 06 Nov '12

06 Nov '12

Saggi Mizrahi has uploaded a new change for review. Change subject: Don't forget to close thread pool when stopping the SPM mailer ...................................................................... Don't forget to close thread pool when stopping the SPM mailer Change-Id: I0fd696a20254380e0a0067dd33c7709445e38afb Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/storage/storage_mailbox.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/80/8980/1 diff --git a/vdsm/storage/storage_mailbox.py b/vdsm/storage/storage_mailbox.py index c537883..041e872 100644 --- a/vdsm/storage/storage_mailbox.py +++ b/vdsm/storage/storage_mailbox.py @@ -687,5 +687,6 @@ time.sleep(self._monitorInterval) finally: self._stopped = True + self._mailman.tp.joinAll(waitForTasks=False) self.log.info("SPM_MailMonitor - Incoming mail monitoring thread stopped") -- To view, visit http://gerrit.ovirt.org/8980 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0fd696a20254380e0a0067dd33c7709445e38afb Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

5 10

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches November 2012