vdsm-patches May 2014

vdsm-patches@lists.fedorahosted.org

44 participants
645 discussions

Change in vdsm[master]: volumeTests: add BlockDomainMetadataSlotTest
by Federico Simoncelli 17 Jul '14

17 Jul '14

Federico Simoncelli has uploaded a new change for review. Change subject: volumeTests: add BlockDomainMetadataSlotTest ...................................................................... volumeTests: add BlockDomainMetadataSlotTest A new test has been added to check the metadata slot selection. Change-Id: I0ff018625443ce7cf75d3edf11644544e7f23dde Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M tests/volumeTests.py 1 file changed, 37 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/66/26266/1 diff --git a/tests/volumeTests.py b/tests/volumeTests.py index 6e4a3b2..5b6476a 100644 --- a/tests/volumeTests.py +++ b/tests/volumeTests.py @@ -24,7 +24,9 @@ from testrunner import VdsmTestCase as TestCaseBase -from storage import outOfProcess, fileSD +from storage import blockSD, blockVolume, fileSD, outOfProcess + +SDBLKSZ = 512 class FileDomainMockObject(fileSD.FileStorageDomain): @@ -40,7 +42,6 @@ class FileVolumeGetVSizeTest(TestCaseBase): VOLSIZE = 1024 - SDBLKSZ = 512 def setUp(self): self.mountpoint = tempfile.mkdtemp() @@ -54,13 +55,44 @@ volPath = os.path.join(imgPath, self.volUUID) os.makedirs(imgPath) - open(volPath, "w").truncate(self.VOLSIZE * self.SDBLKSZ) + open(volPath, "w").truncate(self.VOLSIZE * SDBLKSZ) self.sdobj = FileDomainMockObject(self.mountpoint, self.sdUUID) def tearDown(self): shutil.rmtree(self.mountpoint) def test(self): - volSize = int(self.sdobj.getVSize(self.imgUUID, self.volUUID) / - self.SDBLKSZ) + volSize = int( + self.sdobj.getVSize(self.imgUUID, self.volUUID) / SDBLKSZ) assert volSize == self.VOLSIZE + + +class BlockDomainMockObject(blockSD.BlockStorageDomain): + DOMAIN_VERSION = 3 + + def __init__(self, sdUUID, occupiedMetadataSlots=None): + self.sdUUID = sdUUID + self.stat = None + self.logBlkSize = SDBLKSZ + self.occupiedMetadataSlots = occupiedMetadataSlots + + def getVersion(self): + return self.DOMAIN_VERSION + + def _getOccupiedMetadataSlots(self): + return self.occupiedMetadataSlots + + +class BlockDomainMetadataSlotTest(TestCaseBase): + OCCUPIED_METADATA_SLOTS = [(4, 1), (7, 1)] + EXPECTED_METADATA_SLOT = 5 + + def setUp(self): + self.sdUUID = str(uuid.uuid4()) + self.blksd = BlockDomainMockObject(self.sdUUID, + self.OCCUPIED_METADATA_SLOTS) + + def testMetaSlotSelection(self): + with blockVolume.BlockVolume._tagCreateLock: + mdSlot = self.blksd.getVolumeMetadataSlot(None, 1) + self.assertEqual(mdSlot, self.EXPECTED_METADATA_SLOT) -- To view, visit http://gerrit.ovirt.org/26266 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0ff018625443ce7cf75d3edf11644544e7f23dde Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

5 13

Change in vdsm[master]: sysprep hook: unconfigure a vm clone
by Dan Kenigsberg 17 Jul '14

17 Jul '14

Dan Kenigsberg has uploaded a new change for review. Change subject: sysprep hook: unconfigure a vm clone ...................................................................... sysprep hook: unconfigure a vm clone This hook lets you run virt-sysprep on the disk images of a VM before it is first run. It is useful to do that after a VM is cloned, in order to remove outdated MAC addresses, ssh keys, or user accounts. Change-Id: I447a4b01b86b17289030b71264d5d4218c2aa1e3 Signed-off-by: Dan Kenigsberg <danken(a)redhat.com> --- M configure.ac A debian/vdsm-hook-sysprep.docs A debian/vdsm-hook-sysprep.install M vdsm.spec.in M vdsm_hooks/Makefile.am A vdsm_hooks/sysprep/Makefile.am A vdsm_hooks/sysprep/README A vdsm_hooks/sysprep/before_vm_start.py 8 files changed, 178 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/86/26886/1 diff --git a/configure.ac b/configure.ac index 12828be..ee1efb1 100644 --- a/configure.ac +++ b/configure.ac @@ -302,6 +302,7 @@ vdsm_hooks/smbios/Makefile vdsm_hooks/spiceoptions/Makefile vdsm_hooks/sriov/Makefile + vdsm_hooks/sysprep/Makefile vdsm_hooks/vhostmd/Makefile vdsm_hooks/vmdisk/Makefile vdsm_hooks/vmfex/Makefile diff --git a/debian/vdsm-hook-sysprep.docs b/debian/vdsm-hook-sysprep.docs new file mode 100644 index 0000000..5ecd9c6 --- /dev/null +++ b/debian/vdsm-hook-sysprep.docs @@ -0,0 +1 @@ +COPYING diff --git a/debian/vdsm-hook-sysprep.install b/debian/vdsm-hook-sysprep.install new file mode 100644 index 0000000..bfa51cc --- /dev/null +++ b/debian/vdsm-hook-sysprep.install @@ -0,0 +1 @@ +usr/libexec/vdsm/hooks/before_vm_start/60_sysprep diff --git a/vdsm.spec.in b/vdsm.spec.in index 08fc15e..e7e02ef 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -550,6 +550,14 @@ This vdsm hook can be used to configure some of the spice optimization attributes and values.. +%package hook-sysprep +Summary: Unconfigure guest OS using virt-sysprep +BuildArch: noarch +Requires: %{name} = %{version}-%{release} + +%description hook-sysprep +VDSM hook which unconfigures guest OS image. + %package hook-vmfex Summary: vmfex support for VDSM BuildArch: noarch @@ -1323,6 +1331,10 @@ %defattr(-, root, root, -) %{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/50_spiceoptions +%files hook-sysprep +%defattr(-, root, root, -) +%{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/60_sysprep + %files hook-vmdisk %defattr(-, root, root, -) %{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/50_vmdisk diff --git a/vdsm_hooks/Makefile.am b/vdsm_hooks/Makefile.am index b57181b..c919b23 100644 --- a/vdsm_hooks/Makefile.am +++ b/vdsm_hooks/Makefile.am @@ -43,6 +43,7 @@ smbios \ spiceoptions \ sriov \ + sysprep \ vmdisk \ vmfex \ vmfex_dev \ diff --git a/vdsm_hooks/sysprep/Makefile.am b/vdsm_hooks/sysprep/Makefile.am new file mode 100644 index 0000000..c36ee32 --- /dev/null +++ b/vdsm_hooks/sysprep/Makefile.am @@ -0,0 +1,30 @@ +# +# Copyright 2014 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +EXTRA_DIST = \ + before_vm_start.py + +install-data-local: + $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_start + $(INSTALL_SCRIPT) $(srcdir)/before_vm_start.py \ + $(DESTDIR)$(vdsmhooksdir)/before_vm_start/60_sysprep + +uninstall-local: + $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_start/60_sysprep diff --git a/vdsm_hooks/sysprep/README b/vdsm_hooks/sysprep/README new file mode 100644 index 0000000..d846b18 --- /dev/null +++ b/vdsm_hooks/sysprep/README @@ -0,0 +1,23 @@ +sysprep vdsm hook +================================= +This hook lets you run virt-sysprep on the disk images of a VM +before it is first run. + +Installation: +* Drop before_vm_start.py as /usr/libexec/vdsm/before_vm_start/60_sysprep + or (better) install vdsm-hook-sysprep.rpm on each of your hosts. + +* Use the engine-config to append the appropriate custom property: + + sudo engine-config -s "UserDefinedVMProperties=sysprep=^(true|false)$" + +Usage: +After cloning a VM, and before running it for the first time, set its "sysprep" +custom property to "true" and fire it up. virt-sysprep would be called before +the VM is started and would unconfigure the guest. See the virt-sysprep(1) +manual page for more details on which information is being removed from the +guest. + +It is highly important to remove the "sysprep" property after it is used. +Otherwise, it would be re-applied when the VM is started again, removing +valuable information. diff --git a/vdsm_hooks/sysprep/before_vm_start.py b/vdsm_hooks/sysprep/before_vm_start.py new file mode 100755 index 0000000..cea3a7e --- /dev/null +++ b/vdsm_hooks/sysprep/before_vm_start.py @@ -0,0 +1,109 @@ +#!/usr/bin/python +""" +Run virt-sysprep on the VM images before starting it. + +This hook should be triggered only via the Run Once option +""" + + +import os +import sys +import traceback + +import hooking + + +def iterate_sources(domxml): + for disk in domxml.getElementsByTagName('disk'): + if not disk.hasAttribute('device'): + continue + if disk.attributes['device'].value != 'disk': + continue + drivers = disk.getElementsByTagName('driver') + if not drivers: + continue + driver, = drivers + + source, = disk.getElementsByTagName('source') + if source.hasAttribute('file'): + path = source.attributes['file'].value + elif source.hasAttribute('dev'): + path = source.attributes['dev'].value + + yield driver.attributes['type'].value, path + + +def build_cmd_line(domxml): + cmd = ['virt-sysprep'] + for format, path in iterate_sources(domxml): + cmd.extend(['-a', path, '--format', format]) + return cmd + + +def main(): + sysprep = os.environ.get('sysprep') + if sysprep is not None: + doc = hooking.read_domxml() + out, err, rc = hooking.execCmd(build_cmd_line(doc)) + if rc: + raise RuntimeError(err, rc) + + +def test(): + from xml.dom.minidom import parseString + + TEST1 = """ + <domain type='qemu'> + <uuid>00000000-0000-0000-0000-000000000000</uuid> + <memory>219136</memory> + <currentMemory>219136</currentMemory> + <vcpu>1</vcpu> + <os> + <type arch='i686' machine='pc'>hvm</type> + <boot dev='hd'/> + </os> + <clock offset='utc'/> + <on_poweroff>destroy</on_poweroff> + <on_reboot>restart</on_reboot> + <on_crash>destroy</on_crash> + <devices> + <emulator>/usr/bin/qemu</emulator> + <disk type='block' device='disk'> + <source dev='/dev/HostVG/QEMUGuest1'/> + <target dev='hda' bus='ide'/> + </disk> + <disk type='block' device='cdrom'> + <driver name='qemu' type='raw'/> + <target dev='hdc' bus='ide' tray='open'/> + <readonly/> + </disk> + <disk type='file' device='disk'> + <driver name='qemu' type='raw'/> + <source file='/path/to/image'/> + <target dev='hdc' bus='ide'/> + <readonly/> + <alias name='ide0-1-0'/> + <address type='drive' controller='0' bus='1' target='0' unit='0'/> + </disk> + <disk type='block' device='disk'> + <driver name='qemu' type='qcow2'/> + <source dev='/dev/sdb'/> + <geometry cyls='16383' heads='16' secs='63' trans='lba'/> + <blockio logical_block_size='512' physical_block_size='4096'/> + <target dev='hda' bus='ide'/> + </disk> + </devices> + </domain>""" + + print(build_cmd_line(parseString(TEST1))) + + +if __name__ == '__main__': + try: + if '--test' in sys.argv: + test() + else: + main() + except: + hooking.exit_hook('[unexpected error]: %s\n' % + traceback.format_exc()) -- To view, visit http://gerrit.ovirt.org/26886 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I447a4b01b86b17289030b71264d5d4218c2aa1e3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>

4 3

Change in vdsm[master]: mount: Improve loop devices resolving
by Nir Soffer 15 Jul '14

15 Jul '14

Nir Soffer has uploaded a new change for review. Change subject: mount: Improve loop devices resolving ...................................................................... mount: Improve loop devices resolving _resolveLoopDevice() used O(n^2) search for resolving loop device backing file. For every block device in /proc/mounts, it read and parsed all lines of /etc/mtab, even all platforms where /etc/mtab contains not additional info. This patch changes the search so /etc/mtab is read and parsed once only on platforms where it is different from /proc/mounts. On these platforms, we can get the loop device backing file from /etc/mtab. On platforms where /etc/mtab is a symlink to /proc/self/mounts, we get the backing file from /sys/block/loop*/loop/backing_file. The new code assumes that loop devices are named /dev/loop*, which simplifes the code for resolving the backing file. Change-Id: I40c0bd772327248e6fe08189dada043ea1c6c080 Relates-to: https://bugzilla.redhat.com/1099856 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M vdsm/storage/mount.py 1 file changed, 35 insertions(+), 53 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/90/28090/1 diff --git a/vdsm/storage/mount.py b/vdsm/storage/mount.py index 1671bf8..cc633bf 100644 --- a/vdsm/storage/mount.py +++ b/vdsm/storage/mount.py @@ -22,7 +22,6 @@ from os.path import normpath import re import os -import stat from vdsm import constants import misc @@ -32,6 +31,11 @@ VFS_NFS = "nfs" VFS_NFS4 = "nfs4" VFS_EXT3 = "ext3" + +# Mounts sources + +PROC_MOUNTS = '/proc/mounts' +ETC_MTAB = '/etc/mtab' MountRecord = namedtuple("MountRecord", "fs_spec fs_file fs_vfstype " "fs_mntops fs_freq fs_passno") @@ -61,12 +65,6 @@ fs_freq, fs_passno) -def _iterateMtab(): - with open("/etc/mtab", "r") as f: - for line in f: - yield _parseFstabLine(line) - - def _parseFstabPath(path): return _RE_ESCAPE.sub(lambda s: chr(int(s.group()[1:], 8)), path) @@ -75,58 +73,42 @@ pass -def _resolveLoopDevice(path): - """ - Loop devices appear as the loop device under /proc/mount instead of the - backing file. As the mount command does the resolution so must we. - """ - if not path.startswith("/"): - return path - - try: - st = os.stat(path) - except: - return path - - if not stat.S_ISBLK(st.st_mode): - return path - - minor = os.minor(st.st_rdev) - major = os.major(st.st_rdev) - loopdir = "/sys/dev/block/%d:%d/loop" % (major, minor) - if os.path.exists(loopdir): - with open(loopdir + "/backing_file", "r") as f: - # Remove trailing newline - return f.read()[:-1] - - # Old kernels might not have the sysfs entry, this is a bit slower and does - # not work on hosts that do support the above method. - for rec in _iterateMtab(): - loopOpt = "loop=%s" % path - for opt in rec.fs_mntops: - if opt != loopOpt: - continue - - return rec.fs_spec - - return path +def _readLoopDeviceBackingFile(device): + device_name = device[len('/dev/'):] + path = '/sys/block/%s/loop/backing_file' % device_name + with open(path) as f: + return f.readline().rstrip() -def _iterKnownMounts(): - with open("/proc/mounts", "r") as f: +def _readMountRecords(path): + records = {} + with open(path) as f: for line in f: - yield _parseFstabLine(line) + rec = _parseFstabLine(line) + records[rec.fs_file] = rec + return records def _iterMountRecords(): - for rec in _iterKnownMounts(): - realSpec = _resolveLoopDevice(rec.fs_spec) - if rec.fs_spec == realSpec: - yield rec - continue - - yield MountRecord(realSpec, rec.fs_file, rec.fs_vfstype, - rec.fs_mntops, rec.fs_freq, rec.fs_passno) + mounts = _readMountRecords(PROC_MOUNTS) + if not os.path.samefile(ETC_MTAB, PROC_MOUNTS): + mtab = _readMountRecords(ETC_MTAB) + else: + mtab = None + for mountpoint, rec in mounts.iteritems(): + if rec.fs_spec.startswith('/dev/loop'): + if mtab: + fs_spec = mtab[mountpoint].fs_spec + else: + try: + fs_spec = _readLoopDeviceBackingFile(rec.fs_spec) + except IOError as e: + if e.errno != errno.ENOENT: + raise + fs_spec = rec.fs_spec + yield MountRecord(fs_spec, rec.fs_file, rec.fs_vfstype, + rec.fs_mntops, rec.fs_freq, rec.fs_passno) + yield rec def iterMounts(): -- To view, visit http://gerrit.ovirt.org/28090 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I40c0bd772327248e6fe08189dada043ea1c6c080 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

3 3

Change in vdsm[master]: Makefile.am: create vdsm logs on make install
by Douglas Schilling Landgraf 15 Jul '14

15 Jul '14

Douglas Schilling Landgraf has uploaded a new change for review. Change subject: Makefile.am: create vdsm logs on make install ...................................................................... Makefile.am: create vdsm logs on make install Currently during the RPM install vdsm creates the below logs files: /var/log/vdsm/{metadata.log,mom.log,supervdsm.log,vdsm.log} and we should do the same for non rpm distro. Change-Id: I7f5dc4ca01fecddc5226255e37b0bab68b8c479f Signed-off-by: Douglas Schilling Landgraf <dougsland(a)redhat.com> --- M vdsm/Makefile.am 1 file changed, 4 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/82/23882/1 diff --git a/vdsm/Makefile.am b/vdsm/Makefile.am index 66b9af5..5e7f2cf 100644 --- a/vdsm/Makefile.am +++ b/vdsm/Makefile.am @@ -167,6 +167,10 @@ $(MKDIR_P) $(DESTDIR)$(vdsmlibdir)/upgrade $(MKDIR_P) $(DESTDIR)$(vdsmbackupdir) $(MKDIR_P) $(DESTDIR)$(localstatedir)/lib/libvirt/qemu/channels + touch $(DESTDIR)$(vdsmlogdir)/{metadata.log,mom.log,supervdsm.log,vdsm.log} + chmod 0644 $(DESTDIR)$(vdsmlogdir)/{metadata.log,mom.log,supervdsm.log,vdsm.log} + chown $(VDSMUSER):$(VDSMGROUP) $(DESTDIR)$(vdsmlogdir)/{metadata.log,mom.log,vdsm.log} + chown root:root $(DESTDIR)$(vdsmlogdir)/supervdsm.log uninstall-local: \ uninstall-data-dhclient-hooks \ -- To view, visit http://gerrit.ovirt.org/23882 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7f5dc4ca01fecddc5226255e37b0bab68b8c479f Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Douglas Schilling Landgraf <dougsland(a)redhat.com>

4 6

Change in vdsm[master]: sp: improve pool creation error handling
by Federico Simoncelli 15 Jul '14

15 Jul '14

Federico Simoncelli has uploaded a new change for review. Change subject: sp: improve pool creation error handling ...................................................................... sp: improve pool creation error handling Change-Id: I0cce08e368dec092222c081609d0663d7990ab10 Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/storage/sp.py 1 file changed, 17 insertions(+), 14 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/18/22818/1 diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py index 4f4765e..247973c 100644 --- a/vdsm/storage/sp.py +++ b/vdsm/storage/sp.py @@ -620,27 +620,30 @@ # lock # TBD: create will receive only master domain and further attaches # should be done under SPM - - # Master domain was already attached (in createMaster), - # no need to reattach - for sdUUID in domList: - # No need to attach the master - if sdUUID != msdUUID: - self.attachSD(sdUUID) + try: + for sdUUID in domList: + # Master domain was already attached (in createMaster) + if sdUUID != msdUUID: + self.attachSD(sdUUID) + except Exception: + # FIXME: detachSD will fail for the master domain, we need a + # special handling (master must be detached from the pool). + self.__cleanupDomains(domList, msdUUID, masterVersion) except Exception: - self.log.error("Create pool %s canceled ", poolName, exc_info=True) + self.log.exception('create pool %s canceled', self.spUUID) try: fileUtils.cleanupdir(self.poolPath) - self.__cleanupDomains(domList, msdUUID, masterVersion) - except: - self.log.error("Cleanup failed due to an unexpected error", - exc_info=True) + except Exception: + self.log.exception('pool %s cleanup failed', self.spUUID) raise finally: self._setUnsafe() - self._releaseTemporaryClusterLock(msdUUID) - self.stopMonitoringDomains() + # stopMonitoringDomains needs masterDomain and the monitoring + # domains threads are started only if the master was properly + # initialized and set + if self.masterDomain: + self.stopMonitoringDomains() return True -- To view, visit http://gerrit.ovirt.org/22818 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0cce08e368dec092222c081609d0663d7990ab10 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

6 9

Change in vdsm[master]: vm: small refactor for _normalizeVdsmImg
by Federico Simoncelli 15 Jul '14

15 Jul '14

Federico Simoncelli has uploaded a new change for review. Change subject: vm: small refactor for _normalizeVdsmImg ...................................................................... vm: small refactor for _normalizeVdsmImg Change-Id: Ie68292eee4b82fbe8527e3960739979cfe117dfa Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com> --- M vdsm/vm.py 1 file changed, 18 insertions(+), 17 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/57/19157/1 diff --git a/vdsm/vm.py b/vdsm/vm.py index 92d274e..2605f24 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -1779,25 +1779,26 @@ break return str(idx) - def _normalizeVdsmImg(self, drv): - drv['reqsize'] = drv.get('reqsize', '0') # Backward compatible - if 'device' not in drv: - drv['device'] = 'disk' + def _normalizeVdsmImg(self, drive): + drive['device'] = drive.get('device', 'disk') # Disk by default + drive['reqsize'] = drive.get('reqsize', '0') # Backward compatible - if drv['device'] == 'disk': - res = self.cif.irs.getVolumeSize(drv['domainID'], drv['poolID'], - drv['imageID'], drv['volumeID']) - try: - drv['truesize'] = res['truesize'] - drv['apparentsize'] = res['apparentsize'] - except KeyError: - self.log.error("Unable to get volume size for %s", - drv['volumeID'], exc_info=True) - raise RuntimeError("Volume %s is corrupted or missing" % - drv['volumeID']) + if drive['device'] == 'disk': + volInfo = self.cif.irs.getVolumeInfo( + drive['domainID'], drive['poolID'], drive['imageID'], + drive['volumeID']) + + if volInfo.get('status', {}).get('code', -1): + self.log.error( + "Unable to get volume info for %s", drive['volumeID']) + raise RuntimeError( + "Volume %s is corrupted or missing" % drive['volumeID']) + + drive['truesize'] = volInfo['info']['truesize'] + drive['apparentsize'] = volInfo['info']['apparentsize'] else: - drv['truesize'] = 0 - drv['apparentsize'] = 0 + drive['truesize'] = 0 + drive['apparentsize'] = 0 @classmethod def _normalizeDriveSharedAttribute(self, drive): -- To view, visit http://gerrit.ovirt.org/19157 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie68292eee4b82fbe8527e3960739979cfe117dfa Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>

5 9

Change in vdsm[master]: vmDevices: introduce VmDeviceContainer
by mpoledni＠redhat.com 15 Jul '14

15 Jul '14

Martin Polednik has uploaded a new change for review. Change subject: vmDevices: introduce VmDeviceContainer ...................................................................... vmDevices: introduce VmDeviceContainer EARLY WORK IN PROGRESS: VmDeviceContainer is structure that will allow us to store devices as a class instances while keeping backwards compatibility with old self.conf['devices'] Change-Id: I65debd35115da078df0c0cb6f50c57feb984c5a3 Signed-off-by: Martin Polednik <mpoledni(a)redhat.com> --- M vdsm/vm.py 1 file changed, 33 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/38/21138/1 diff --git a/vdsm/vm.py b/vdsm/vm.py index 5ae54d7..36cdbb1 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -1727,6 +1727,39 @@ return m +class VmDeviceContainer(dict): + @property + def legacy(self): + """ + Return list of device dicts that represents backwards-compatible + self.conf['devices'] + + [..., {..., 'type': 'disk', ...}, ...] + """ + deviceList = [] + for key in self.keys(): + for device in self[key]: + # loop through devices __slots__ and return all set attributes + deviceList.append(dict((attr, getattr(device, attr)) + for attr in device.__slots__ + if hasattr(device, attr))) + + return deviceList + + def restoreLegacy(self, state): + """ + Reconstruct container using old self.conf['devices'] structure of + + [..., {..., 'type': 'disk', ...}, ...] + to + + VmDeviceContainer[DISK_DEVICES] = + [..., {..., 'type': 'disk', ...}, ...] + """ + for device in state: + self[device['type']] = device + + class Vm(object): """ Used for abstracting communication between various parts of the -- To view, visit http://gerrit.ovirt.org/21138 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I65debd35115da078df0c0cb6f50c57feb984c5a3 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Polednik <mpoledni(a)redhat.com>

4 12

Change in vdsm[master]: vmDevices: add mechanism to persist vmDevice defaults
by mpoledni＠redhat.com 15 Jul '14

15 Jul '14

Martin Polednik has uploaded a new change for review. Change subject: vmDevices: add mechanism to persist vmDevice defaults ...................................................................... vmDevices: add mechanism to persist vmDevice defaults Multiple vmDevices such as BalloonDevice or watchdogDevice currently do not persist their defaults in class attributes but rather use them in XML directly, hiding their existence. This patch aims to change this behavior by implementing vmDevice._defaults(), which adds the default attributes to instance directly to enable future persistence of these classes. Change-Id: Idc8383cbce78490c8dfab1c253883a06459f1547 Signed-off-by: Martin Polednik <mpoledni(a)redhat.com> --- M tests/vmTests.py M vdsm/vm.py 2 files changed, 45 insertions(+), 29 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/66/21066/1 diff --git a/tests/vmTests.py b/tests/vmTests.py index 1f69f0a..5c813fc 100644 --- a/tests/vmTests.py +++ b/tests/vmTests.py @@ -261,7 +261,7 @@ def testWatchdogXML(self): watchdogXML = '<watchdog action="none" model="i6300esb"/>' - dev = {'device': 'watchdog', 'type': 'watchdog', + dev = {'device': 'watchdog', 'specParams': {'model': 'i6300esb', 'action': 'none'}} watchdog = vm.WatchdogDevice(self.conf, self.log, **dev) self.assertXML(watchdog.getXML(), watchdogXML) diff --git a/vdsm/vm.py b/vdsm/vm.py index 796735f..36b9598 100644 --- a/vdsm/vm.py +++ b/vdsm/vm.py @@ -1167,6 +1167,7 @@ class VmDevice(object): def __init__(self, conf, log, **kwargs): + self.specParams = {} for attr, value in kwargs.iteritems(): try: setattr(self, attr, value) @@ -1175,6 +1176,10 @@ self.conf = conf self.log = log self._deviceXML = None + self._defaults() + + def _defaults(self): + pass def __str__(self): attrs = [':'.join((a, str(getattr(self, a)))) for a in dir(self) @@ -1192,10 +1197,13 @@ elemAttrs['type'] = deviceType for attrName in attributes: - if not hasattr(self, attrName): + if attrName in self.specParams: + attr = self.specParams[attrName] + elif hasattr(self, attrName): + attr = getattr(self, attrName) + else: continue - attr = getattr(self, attrName) if isinstance(attr, dict): element.appendChildWithArgs(attrName, **attr) else: @@ -1215,37 +1223,44 @@ class ControllerDevice(VmDevice): + def _defaults(self): + if self.device == 'virtio-serial': + if 'index' not in self.specParams: + self.index = '0' + if 'ports' not in self.specParams: + self.ports = '16' def getXML(self): """ Create domxml for controller device """ ctrl = self.createXmlElem('controller', self.device, - ['index', 'model', 'master', 'address']) - if self.device == 'virtio-serial': - ctrl.setAttrs(index='0', ports='16') + ['index', 'model', 'master', 'address', + 'ports']) return ctrl class VideoDevice(VmDevice): + def _defaults(self): + if 'vram' not in self.specParams: + self.specParams['vram'] = '32768' + if 'heads' not in self.specParams: + self.specParams['heads'] = '1' def getXML(self): """ Create domxml for video device """ video = self.createXmlElem('video', None, ['address']) - sourceAttrs = {'vram': self.specParams.get('vram', '32768'), - 'heads': self.specParams.get('heads', '1')} - if 'ram' in self.specParams: - sourceAttrs['ram'] = self.specParams['ram'] + model = self.createXmlElem('model', self.device, + ['vram', 'heads', 'ram']) - video.appendChildWithArgs('model', type=self.device, **sourceAttrs) + video.appendChild(model) return video class SoundDevice(VmDevice): - def getXML(self): """ Create domxml for sound device @@ -1256,7 +1271,6 @@ class NetworkInterfaceDevice(VmDevice): - def __init__(self, conf, log, **kwargs): # pyLint can't tell that the Device.__init__() will # set a nicModel attribute, so modify the kwarg list @@ -1646,7 +1660,6 @@ class BalloonDevice(VmDevice): - def getXML(self): """ Create domxml for a memory balloon device. @@ -1662,11 +1675,11 @@ class WatchdogDevice(VmDevice): - def __init__(self, *args, **kwargs): - super(WatchdogDevice, self).__init__(*args, **kwargs) - - if not hasattr(self, 'specParams'): - self.specParams = {} + def _defaults(self): + if 'model' not in self.specParams: + self.specParams['model'] = 'i6300esb' + if 'action' not in self.specParams: + self.specParams['action'] = 'none' def getXML(self): """ @@ -1677,9 +1690,8 @@ function='0x0'/> </watchdog> """ - m = self.createXmlElem(self.type, None, ['address']) - m.setAttrs(model=self.specParams.get('model', 'i6300esb'), - action=self.specParams.get('action', 'none')) + m = self.createXmlElem(self.device, None, ['address', 'model', + 'action']) return m @@ -1692,11 +1704,8 @@ <address ... /> </smartcard> """ - card = self.createXmlElem(self.device, None, ['address']) - sourceAttrs = {'mode': self.specParams['mode']} - if sourceAttrs['mode'] != 'host': - sourceAttrs['type'] = self.specParams['type'] - card.setAttrs(**sourceAttrs) + card = self.createXmlElem(self.device, None, ['address', 'mode', + 'type']) return card @@ -1713,6 +1722,12 @@ class ConsoleDevice(VmDevice): + def _defaults(self): + self.type = 'pty' + self.port = '0' + self.specParams['type'] = 'virtio' + self.specParams['port'] = '0' + def getXML(self): """ Create domxml for a console device. @@ -1721,8 +1736,9 @@ <target type='virtio' port='0'/> </console> """ - m = self.createXmlElem('console', 'pty') - m.appendChildWithArgs('target', type='virtio', port='0') + m = self.createXmlElem(self.device, self.type) + m.appendChildWithArgs('target', None, type=self.specParams['type'], + port=self.specParams['port']) return m -- To view, visit http://gerrit.ovirt.org/21066 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Idc8383cbce78490c8dfab1c253883a06459f1547 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Martin Polednik <mpoledni(a)redhat.com>

5 9

Change in vdsm[master]: gluster: [WIP]Get size information of gluster volume
by avishwan＠redhat.com 15 Jul '14

15 Jul '14

Aravinda VK has uploaded a new change for review. Change subject: gluster: [WIP]Get size information of gluster volume ...................................................................... gluster: [WIP]Get size information of gluster volume Change-Id: I358d4f3bf793ecc1a01e0592d68919d1405f6e19 Signed-off-by: Aravinda VK <avishwan(a)redhat.com> --- M client/vdsClientGluster.py M vdsm.spec.in M vdsm/gluster/Makefile.am M vdsm/gluster/__init__.py M vdsm/gluster/api.py A vdsm/gluster/gfapi.py 6 files changed, 100 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/22/17822/1 diff --git a/client/vdsClientGluster.py b/client/vdsClientGluster.py index 90af83e..644bfe3 100644 --- a/client/vdsClientGluster.py +++ b/client/vdsClientGluster.py @@ -424,6 +424,17 @@ pp.pprint(status) return status['status']['code'], status['status']['message'] + def do_glusterVolumeSize(self, args): + params = self._eqSplit(args) + try: + volumeName = params.get('volumeName', '') + except: + raise ValueError + + status = self.s.glusterVolumeSize(volumeName) + pp.pprint(status) + return status['status']['code'], status['status']['message'] + def getGlusterCmdDict(serv): return \ @@ -705,4 +716,9 @@ 'not set' '(swift, glusterd, smb, memcached)' )), + 'glusterVolumeSize': ( + serv.do_glusterVolumeSize, + ('volumeName=<volume name>', + 'Returns total, available and used space status of gluster volume' + )), } diff --git a/vdsm.spec.in b/vdsm.spec.in index 21b3565..e49e102 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -1271,6 +1271,7 @@ %doc COPYING %{_datadir}/%{vdsm_name}/gluster/api.py* %{_datadir}/%{vdsm_name}/gluster/vdsmapi-gluster-schema.json +%{_datadir}/%{vdsm_name}/gluster/gfapi.py* %{_datadir}/%{vdsm_name}/gluster/hooks.py* %{_datadir}/%{vdsm_name}/gluster/services.py* %endif diff --git a/vdsm/gluster/Makefile.am b/vdsm/gluster/Makefile.am index dd5434d..9c2989f 100644 --- a/vdsm/gluster/Makefile.am +++ b/vdsm/gluster/Makefile.am @@ -26,6 +26,7 @@ api.py \ cli.py \ exception.py \ + gfapi.py \ hooks.py \ hostname.py \ services.py \ diff --git a/vdsm/gluster/__init__.py b/vdsm/gluster/__init__.py index bec70ea..e5a2fd6 100644 --- a/vdsm/gluster/__init__.py +++ b/vdsm/gluster/__init__.py @@ -22,7 +22,7 @@ import tempfile from functools import wraps -MODULE_LIST = ('cli', 'hooks', 'services') +MODULE_LIST = ('cli', 'hooks', 'services', 'gfapi') def makePublic(func): diff --git a/vdsm/gluster/api.py b/vdsm/gluster/api.py index 4bd8308..89546c1 100644 --- a/vdsm/gluster/api.py +++ b/vdsm/gluster/api.py @@ -287,6 +287,22 @@ status = self.svdsmProxy.glusterServicesGet(serviceNames) return {'services': status} + @exportAsVerb + def volumeSize(self, volumeName, options=None): + data = self.svdsmProxy.glusterVolumeStatvfs(volumeName) + # f_blocks = Total number of blocks + # f_bfree = Total number of blocks free + # f_bavail = Total number of blocks available for non root user + # total blocks available = f_blocks - (f_bfree - f_bavail) + total_blocks_available = data['f_blocks'] - \ + (data['f_bfree'] - data['f_bavail']) + return { + 'total': total_blocks_available * data.f_bsize / 1024, + 'free': data['f_bavail'] * data['f_bsize'] / 1024, + 'used': (total_blocks_available - data['f_bavail']) * \ + data['f_bsize'] / 1024 + } + def getGlusterMethods(gluster): l = [] diff --git a/vdsm/gluster/gfapi.py b/vdsm/gluster/gfapi.py new file mode 100644 index 0000000..abfdabd --- /dev/null +++ b/vdsm/gluster/gfapi.py @@ -0,0 +1,65 @@ +# +# Copyright 2013 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +from ctypes import * + +# import exception as ge +from . import makePublic + +GLUSTER_VOL_PROTOCAL='tcp' +GLUSTER_VOL_HOST='localhost' +GLUSTER_VOL_PORT=24007 + +class Stat (Structure): + _fields_ = [ + ('f_bsize', c_ulong), + ('f_frsize', c_ulong), + ('f_blocks', c_ulong), + ('f_bfree', c_ulong), + ('f_bavail', c_ulong), + ('f_files', c_ulong), + ('f_ffree', c_ulong), + ('f_favail', c_ulong), + ('f_fsid', c_ulong), + ('f_flag', c_ulong), + ('f_namemax', c_ulong), + ('__f_spare', c_int * 6), + ] + + +api = CDLL("libgfapi.so",RTLD_GLOBAL) +api.glfs_statvfs.restype = c_int +api.glfs_statvfs.argtypes = [c_void_p, c_char_p, POINTER(Stat)] + +@makePublic +def volumeStatvfs(volumeId): + path = "/" + fs = api.glfs_new(volumeId) + api.glfs_set_volfile_server(fs, + GLUSTER_VOL_PROTOCAL, + GLUSTER_VOL_HOST, + GLUSTER_VOL_PORT) + api.glfs_init(fs) + + x = Stat() + rc = api.glfs_statvfs(fs, path, byref(x)) + statvfsData = {} + for k in x._fields_: + statvfsData[k[0]] = getattr(x, k[0]) + return statvfsData -- To view, visit http://gerrit.ovirt.org/17822 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I358d4f3bf793ecc1a01e0592d68919d1405f6e19 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Aravinda VK <avishwan(a)redhat.com>

8 42

Change in vdsm[master]: xmlrpc: Fix connection reset on Python 2.6
by Nir Soffer 15 Jul '14

15 Jul '14

Nir Soffer has uploaded a new change for review. Change subject: xmlrpc: Fix connection reset on Python 2.6 ...................................................................... xmlrpc: Fix connection reset on Python 2.6 When request is aborted before the reading the entire request, closing the socket cause the other side to receive connection reset, and loose the response data. Python 2.7 introduced the shutdown_request method, fixing this issue. This patch check if SimpleXMLRPCServer has the shutdown_request method, and if not, do a clean request shutdwon in close_request. Change-Id: Id77a32bb0d4fff13ee680eade52a2f60e4e808b5 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M lib/vdsm/utils.py 1 file changed, 17 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/26/24026/1 diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py index 9429308..eeea5d4 100644 --- a/lib/vdsm/utils.py +++ b/lib/vdsm/utils.py @@ -158,6 +158,23 @@ logRequests, allow_none, encoding, bind_and_activate) + # Fix for connection reset issue when running on Python 2.6. This happens + # when request is aborted before the reading the entire request. Closing + # the socket cause the other side to receive connection reset, and loose + # the response data. Python 2.7 introduced the shutdown_request method, + # fixing this issue. If we run on a version without this method, we will do + # a clean shutdown in close_request instead. + if not hasattr(SimpleXMLRPCServer, 'shutdown_request'): + def close_request(self, request): + # From Python 2.7 SocketServer.TCPServer.shutdown_request + try: + # Explicitly shutdown. socket.close() merely releases the + # socket and waits for GC to perform the actual close. + request.shutdown(socket.SHUT_WR) + except socket.error: + pass # Some platforms may raise ENOTCONN here + SimpleXMLRPCServer.close_request(self, request) + #Threaded version of SimpleXMLRPCServer class SimpleThreadedXMLRPCServer(SocketServer.ThreadingMixIn, -- To view, visit http://gerrit.ovirt.org/24026 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Id77a32bb0d4fff13ee680eade52a2f60e4e808b5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

5 8

← Newer
1
...
7
8
9
10
11
12
13
...
65
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches May 2014