Change in vdsm[master]: Avoid running scheduled jobs when not SPM.
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: Avoid running scheduled jobs when not SPM.
......................................................................
Avoid running scheduled jobs when not SPM.
Related to: BZ#960952.
Change-Id: I0402817df9c2c08a5a945082edfbb03e832facbe
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/storage/sp.py
1 file changed, 3 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/12/16412/1
diff --git a/vdsm/storage/sp.py b/vdsm/storage/sp.py
index 9b154f4..12c6e2a 100644
--- a/vdsm/storage/sp.py
+++ b/vdsm/storage/sp.py
@@ -151,6 +151,7 @@
def forceFreeSpm(self):
# DO NOT USE, STUPID, HERE ONLY FOR BC
# TODO: SCSI Fence the 'lastOwner'
+ self.taskMng.unloadTasks()
self.setMetaParams({PMDK_SPM_ID: SPM_ID_FREE, PMDK_LVER: LVER_INVALID},
__securityOverride=True)
self.spmRole = SPM_FREE
@@ -272,6 +273,7 @@
self.masterDomain.acquireHostId(self.id)
self.masterDomain.acquireClusterLock(self.id)
except:
+ self.taskMng.unloadTasks()
self.spmRole = SPM_FREE
raise
@@ -393,6 +395,7 @@
cls.log.debug("master `%s` is not mounted, skipping", master)
def stopSpm(self, force=False):
+ self.taskMng.unloadTasks()
with self.lock:
if not force and self.spmRole == SPM_FREE:
return True
--
To view, visit http://gerrit.ovirt.org/16412
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0402817df9c2c08a5a945082edfbb03e832facbe
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: [WIP] Reduce the number of pvs calls using lvm.checkpvsblksi...
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: [WIP] Reduce the number of pvs calls using lvm.checkpvsblksize()
......................................................................
[WIP] Reduce the number of pvs calls using lvm.checkpvsblksize()
Change-Id: I778a22843392f2b1345718b3c02bd2f12200056b
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/storage/blockSD.py
1 file changed, 3 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/92/11692/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index aa4059a..6a02f5a 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -503,10 +503,12 @@
except se.LogicalVolumeDoesNotExistError:
pass
- numOfPVs = len(lvm.listPVNames(vgName))
+ pvs = lvm.listPVNames(vgName)
+ numOfPVs = len(pvs)
if version in VERS_METADATA_LV and numOfPVs > MAX_PVS:
cls.log.debug("%d > %d", numOfPVs, MAX_PVS)
raise se.StorageDomainIsMadeFromTooManyPVs()
+ logBlkSize, phyBlkSize = lvm.checkpvsblksize(pvs)
# Create metadata service volume
metasize = metaSize(vgName)
@@ -548,8 +550,6 @@
md = LvBasedSDMetadata(vgName, sd.METADATA)
elif version in VERS_METADATA_TAG:
md = TagBasedSDMetadata(vgName)
-
- logBlkSize, phyBlkSize = lvm.getVGBlockSizes(vgName)
# create domain metadata
# FIXME : This is 99% like the metadata in file SD
--
To view, visit http://gerrit.ovirt.org/11692
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I778a22843392f2b1345718b3c02bd2f12200056b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: [WIP] Fixing metadata writings using cmd_type_1.
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: [WIP] Fixing metadata writings using cmd_type_1.
......................................................................
[WIP] Fixing metadata writings using cmd_type_1.
Change-Id: I88b3ede4bd15a884bcd07cf4099f53d30aa98f84
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/storage/blockSD.py
1 file changed, 13 insertions(+), 14 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/95/11695/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index c688826..359a691 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -123,7 +123,6 @@
LVM_ENC_ESCAPE = re.compile("&(\d+)&")
-# Move to lvm
def lvmTagEncode(s):
return INVALID_CHARS.sub(lambda c: "&%s&" % ord(c.group()), s)
@@ -505,7 +504,6 @@
def makeChangeCmd(toRemove, toAdd):
changeCmd = [constants.EXT_LVM, "vgchange"]
- changeCmd.extend(lvm.LVM_NOBACKUP)
changeCmd.extend(("--config", cmdConf % filter))
for tag in toRemove:
changeCmd.extend(("--deltag", tag))
@@ -623,28 +621,23 @@
DMDK_PHYBLKSIZE: phyBlkSize,
}
- initialMetadata.update(mapping)
-
if version in VERS_METADATA_LV:
md = LvBasedSDMetadata(vgName, sd.METADATA)
md.update(initialMetadata)
#Replace unready tag
toRemove = (STORAGE_UNREADY_DOMAIN_TAG,)
toAdd = (STORAGE_DOMAIN_TAG,)
+ initialMetadata.update(mapping)
elif version in VERS_METADATA_TAG:
md = TagBasedSDMetadata(vgName)
#Add all tags here and remove the unready
- toRemove = tuple(VGTagMetadataRW.METADATA_TAG_PREFIX +
- lvmTagEncode(item) for item in initialMetadata)
- toRemove.append(STORAGE_UNREADY_DOMAIN_TAG)
- toAdd = tuple(VGTagMetadataRW.METADATA_TAG_PREFIX +
- lvmTagEncode(item) for item in initialMetadata)
+ toRemove = [STORAGE_UNREADY_DOMAIN_TAG]
+ toAdd = [VGTagMetadataRW.METADATA_TAG_PREFIX +
+ lvmTagEncode("%s=%s" % item)
+ for item in initialMetadata.items()]
+ toAdd.extend([lvmTagEncode("%s,%s" % (k, v))
+ for k, v in mapping.items()])
toAdd.append(STORAGE_DOMAIN_TAG)
-
- try:
- setInitialTags(toRemove, toAdd)
- except se.VolumeGroupReplaceTagError:
- raise se.VolumeGroupUninitialized(vgName)
if version in sd.DOM_SAFELEASE_VERS:
initClusterLock()
@@ -653,6 +646,12 @@
else:
cls.log.warning("Lock method for version %s unknown", version)
+ try:
+ setInitialTags(toRemove, toAdd)
+ except se.VolumeGroupReplaceTagError:
+ raise se.VolumeGroupUninitialized(vgName)
+
+ lvm.invalidateVG(vgName)
return None
def getReadDelay(self):
--
To view, visit http://gerrit.ovirt.org/11695
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I88b3ede4bd15a884bcd07cf4099f53d30aa98f84
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
Gerrit-Reviewer: Yeela Kaplan <ykaplan(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: [WIP] createSD() and createStoragePool() explicit cmd_type_1...
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: [WIP] createSD() and createStoragePool() explicit cmd_type_1.
......................................................................
[WIP] createSD() and createStoragePool() explicit cmd_type_1.
TODO:
initSanLockLock
remove[StoragePool, SD, VG] should be type 1?
Change-Id: I1b9d533812718d29a01998934e22954cee650265
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/storage/blockSD.py
M vdsm/storage/hsm.py
2 files changed, 88 insertions(+), 26 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/94/11694/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index 6a02f5a..c688826 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -479,7 +479,65 @@
'vgUUID' - volume group UUID
'storageType' - NFS_DOMAIN, LOCALFS_DOMAIN, &etc.
'version' - DOMAIN_VERSIONS
+
+ This function intentionally bypasses the lvm cache sets since
+ uses a private configuration.
"""
+ def makeCreateCmd(lvSize, lvName):
+ createCmd = [constants.EXT_LVM, "lvcreate"]
+ createCmd.extend(("--config", cmdConf % filter))
+ createCmd.extend(("--contiguous", "y", "--size", "%sm" % lvSize))
+ createCmd.extend(("--name", lvName, vgName))
+ return createCmd
+
+ def createSpecialLvs():
+ for size, name in ((metaSize(vgName), sd.METADATA),
+ (sd.LEASES_SIZE, sd.LEASES),
+ (sd.IDS_SIZE, sd.IDS),
+ (sd.INBOX_SIZE, sd.INBOX),
+ (sd.OUTBOX_SIZE, sd.OUTBOX),
+ (MASTERLV_SIZE, MASTERLV)
+ ):
+ rc, out, err = misc.execCmd(makeCreateCmd(size, name),
+ sudo=True)
+ if rc != 0:
+ raise se.CannotCreateLogicalVolume(vgName, name)
+
+ def makeChangeCmd(toRemove, toAdd):
+ changeCmd = [constants.EXT_LVM, "vgchange"]
+ changeCmd.extend(lvm.LVM_NOBACKUP)
+ changeCmd.extend(("--config", cmdConf % filter))
+ for tag in toRemove:
+ changeCmd.extend(("--deltag", tag))
+ for tag in toAdd:
+ changeCmd.extend(("--addtag", tag))
+ changeCmd.append(vgName)
+ return changeCmd
+
+ def setInitialTags(toRemove, toAdd):
+ rc, out, err = misc.execCmd(makeChangeCmd(toRemove, toAdd),
+ sudo=True)
+ if rc != 0:
+ raise se.VolumeGroupReplaceTagError("vg:%s del:%s add:%s (%s)"
+ % (vgName,
+ ", ".join(toRemove),
+ ", ".join(toAdd),
+ err[-1]))
+
+ def initClusterLock():
+ lockUtilPath = config.get('irs', 'lock_util_path')
+ lockUtil = os.path.join(lockUtilPath, "safelease")
+ initCommand = [lockUtil, "release", "-f", lvm.lvPath(vgName,
+ sd.LEASES), "0"]
+ rc, out, err = misc.execCmd(initCommand, sudo=False,
+ cwd=lockUtilPath)
+ if rc != 0:
+ raise se.ClusterLockInitError("could not initialise spm lease"
+ "(%s): %s" % (rc, out))
+
+ def initSanLockLock():
+ pass
+
cls.log.info("sdUUID=%s domainName=%s domClass=%s vgUUID=%s "
"storageType=%s version=%s", sdUUID, domainName, domClass,
vgUUID, storageType, version)
@@ -510,9 +568,10 @@
raise se.StorageDomainIsMadeFromTooManyPVs()
logBlkSize, phyBlkSize = lvm.checkpvsblksize(pvs)
- # Create metadata service volume
- metasize = metaSize(vgName)
- lvm.createLV(vgName, sd.METADATA, "%s" % (metasize))
+ filter = 'filter = [ ' + '"a%' + " ".join(pvs) + '%", ' + '"r%.*%" ]'
+ cmdConf = lvm.LVMCONF_TEMPLATE.replace("locking_type=4",
+ "locking_type=1")
+ createSpecialLvs()
# Create the mapping right now so the index 0 is guaranteed
# to belong to the metadata volume. Since the metadata is at
# least SDMETADATA/METASIZE units, we know we can use the first
@@ -520,16 +579,8 @@
# pass metadata's dev to ensure it is the first mapping
mapping = cls.getMetaDataMapping(vgName)
- # Create the rest of the BlockSD internal volumes
- lvm.createLV(vgName, sd.LEASES, sd.LEASES_SIZE)
- lvm.createLV(vgName, sd.IDS, sd.IDS_SIZE)
- lvm.createLV(vgName, sd.INBOX, sd.INBOX_SIZE)
- lvm.createLV(vgName, sd.OUTBOX, sd.OUTBOX_SIZE)
- lvm.createLV(vgName, MASTERLV, MASTERLV_SIZE)
-
# Create VMS file system
_createVMSfs(os.path.join("/dev", vgName, MASTERLV))
-
lvm.deactivateLVs(vgName, MASTERLV)
path = lvm.lvPath(vgName, sd.METADATA)
@@ -545,11 +596,6 @@
raise
except se.StorageException:
raise se.VolumesZeroingError(path)
-
- if version in VERS_METADATA_LV:
- md = LvBasedSDMetadata(vgName, sd.METADATA)
- elif version in VERS_METADATA_TAG:
- md = TagBasedSDMetadata(vgName)
# create domain metadata
# FIXME : This is 99% like the metadata in file SD
@@ -579,20 +625,35 @@
initialMetadata.update(mapping)
- md.update(initialMetadata)
+ if version in VERS_METADATA_LV:
+ md = LvBasedSDMetadata(vgName, sd.METADATA)
+ md.update(initialMetadata)
+ #Replace unready tag
+ toRemove = (STORAGE_UNREADY_DOMAIN_TAG,)
+ toAdd = (STORAGE_DOMAIN_TAG,)
+ elif version in VERS_METADATA_TAG:
+ md = TagBasedSDMetadata(vgName)
+ #Add all tags here and remove the unready
+ toRemove = tuple(VGTagMetadataRW.METADATA_TAG_PREFIX +
+ lvmTagEncode(item) for item in initialMetadata)
+ toRemove.append(STORAGE_UNREADY_DOMAIN_TAG)
+ toAdd = tuple(VGTagMetadataRW.METADATA_TAG_PREFIX +
+ lvmTagEncode(item) for item in initialMetadata)
+ toAdd.append(STORAGE_DOMAIN_TAG)
- # Mark VG with Storage Domain Tag
try:
- lvm.replaceVGTag(vgName, STORAGE_UNREADY_DOMAIN_TAG,
- STORAGE_DOMAIN_TAG)
- except se.StorageException:
+ setInitialTags(toRemove, toAdd)
+ except se.VolumeGroupReplaceTagError:
raise se.VolumeGroupUninitialized(vgName)
- bsd = BlockStorageDomain(sdUUID)
+ if version in sd.DOM_SAFELEASE_VERS:
+ initClusterLock()
+ elif sd.DOM_SANLOCK_VERS:
+ initSanLockLock()
+ else:
+ cls.log.warning("Lock method for version %s unknown", version)
- bsd.initSPMlease()
-
- return bsd
+ return None
def getReadDelay(self):
t = time.time()
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index 7b03064..640e7bd 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -2445,7 +2445,8 @@
domVersion)
else:
raise se.StorageDomainTypeError(storageType)
- sdCache.manuallyAddDomain(newSD)
+ if newSD:
+ sdCache.manuallyAddDomain(newSD)
@public
def validateStorageDomain(self, sdUUID, options=None):
--
To view, visit http://gerrit.ovirt.org/11694
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1b9d533812718d29a01998934e22954cee650265
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: [WIP] Add cmd_type_1
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: [WIP] Add cmd_type_1
......................................................................
[WIP] Add cmd_type_1
Change-Id: I0c62d42936630132722974bcdedd619d13185cde
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/storage/lvm.py
1 file changed, 20 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/93/11693/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index 148a010..3d33ac8 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -147,6 +147,23 @@
return filt
+def _cmd_type_1(cmd, devices):
+ """
+ Use only for lvm cluster safe commands.
+
+ These are cmds that not change metadata of an existing VG.
+ _DON'T_ call this directly or indirectly.
+ """
+ filter = _buildFilter(devices)
+ conf = LVMCONF_TEMPLATE.replace("locking_type=4",
+ "locking_type=1") % filter
+ conf = conf.replace("\n", " ")
+ finalCmd = [constants.EXT_LVM, cmd[0], "--config", conf]
+ finalCmd.extend(cmd[1:])
+ rc, out, err = misc.execCmd(finalCmd, sudo=True)
+ return rc, out, err
+
+
def _buildConfig(devList):
flt = _buildFilter(chain(devList, USER_DEV_LIST))
conf = _current_lvmconf % flt
@@ -672,7 +689,7 @@
cmd.extend(("--metadatasize", metadatasize, "--metadatacopies", "2",
"--metadataignore", "y"))
cmd.extend(devices)
- rc, out, err = _lvminfo.cmd(cmd)
+ rc, out, err = _cmd_type_1(cmd)
return rc, out, err
@@ -884,7 +901,7 @@
# Activate the 1st PV metadata areas
cmd = ["pvchange", "--metadataignore", "n"]
cmd.append(pvs[0])
- rc, out, err = _lvminfo.cmd(cmd)
+ rc, out, err = _cmd_type_1(cmd, (pvs[0],))
if rc != 0:
raise se.PhysDevInitializationError(pvs[0])
@@ -892,7 +909,7 @@
if initialTag:
options.extend(("--addtag", initialTag))
cmd = ["vgcreate"] + options + [vgName] + pvs
- rc, out, err = _lvminfo.cmd(cmd)
+ rc, out, err = _cmd_type_1(cmd, pvs)
if rc == 0:
_lvminfo._invalidatepvs(pvs)
_lvminfo._invalidatevgs(vgName)
--
To view, visit http://gerrit.ovirt.org/11693
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c62d42936630132722974bcdedd619d13185cde
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: [WIP] Public checkpvsblksize
by ewarszaw＠redhat.com
Eduardo has uploaded a new change for review.
Change subject: [WIP] Public checkpvsblksize
......................................................................
[WIP] Public checkpvsblksize
Change-Id: I02d1565e39780cb0ce97caba7b44db8b191c0c35
Signed-off-by: Eduardo <ewarszaw(a)redhat.com>
---
M vdsm/storage/lvm.py
1 file changed, 6 insertions(+), 4 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/91/11691/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index 8955657..148a010 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -878,7 +878,7 @@
def createVG(vgName, devices, initialTag, metadataSize, extentsize="128m",
force=False):
pvs = [_fqpvname(pdev) for pdev in _normalizeargs(devices)]
- _checkpvsblksize(pvs)
+ checkpvsblksize(pvs)
_initpvs(pvs, metadataSize, force)
# Activate the 1st PV metadata areas
@@ -927,7 +927,7 @@
def extendVG(vgName, devices, force):
pvs = [_fqpvname(pdev) for pdev in _normalizeargs(devices)]
- _checkpvsblksize(pvs, getVGBlockSizes(vgName))
+ checkpvsblksize(pvs, getVGBlockSizes(vgName))
vg = _lvminfo.getVg(vgName)
# Format extension PVs as all the other already in the VG
_initpvs(pvs, int(vg.vg_mda_size) / 2 ** 20, force)
@@ -967,7 +967,7 @@
return multipath.getDeviceBlockSizes(dev)
-def _checkpvsblksize(pvs, vgBlkSize=None):
+def checkpvsblksize(pvs, vgBlkSize=None):
for pv in pvs:
pvBlkSize = _getpvblksize(pv)
logPvBlkSize, phyPvBlkSize = pvBlkSize
@@ -987,12 +987,14 @@
if logPvBlkSize != vgBlkSize[0]:
raise se.VolumeGroupBlockSizeError(vgBlkSize, pvBlkSize)
+ return pvBlkSize
+
def checkVGBlockSizes(vgUUID, vgBlkSize=None):
pvs = listPVNames(vgUUID)
if not pvs:
raise se.VolumeGroupDoesNotExist("vg_uuid: %s" % vgUUID)
- _checkpvsblksize(pvs, vgBlkSize)
+ checkpvsblksize(pvs, vgBlkSize)
def getVGBlockSizes(vgUUID):
--
To view, visit http://gerrit.ovirt.org/11691
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I02d1565e39780cb0ce97caba7b44db8b191c0c35
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Eduardo <ewarszaw(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: check the syntaxs of optional parameters in json schema
by shaohef＠linux.vnet.ibm.com
ShaoHe Feng has uploaded a new change for review.
Change subject: check the syntaxs of optional parameters in json schema
......................................................................
check the syntaxs of optional parameters in json schema
There are so many syntaxs error of optional parameters define in
json schema
This patch can both check the descrption errors of optional parameters
missing #optional and the define error of optional parameters missing *
Change-Id: I94f92459ee8787780a54a510b6f8cc074fb2a235
Signed-off-by: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>
---
M vdsm_api/process-schema.py
1 file changed, 17 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/46/10446/1
diff --git a/vdsm_api/process-schema.py b/vdsm_api/process-schema.py
index 8244995..5db6966 100755
--- a/vdsm_api/process-schema.py
+++ b/vdsm_api/process-schema.py
@@ -22,6 +22,11 @@
import sys
import re
import vdsmapi
+try:
+ from collections import OrderedDict
+ OrderedDict # make pyflakes happy
+except ImportError:
+ from ordereddict import OrderedDict
html_escape_table = {
"&": "&",
@@ -143,6 +148,18 @@
symbol[mode][name] = desc
# Track the name in case there is are multiple lines to append
last_arg = name
+ # check optional parameters
+ if desc.startswith('#optional '):
+ assert ('*' + name in symbol['data'].keys()), \
+ ('Define of %s optional parameter error:\n\t%s should '
+ 'start with *' % (symbol['name'], name))
+ if isinstance(symbol['data'], OrderedDict):
+ if '*' + name in symbol['data'].keys():
+ assert desc.startswith('#optional '), \
+ ('Description of %s optional parameter error:\n\t'
+ 'The description of optional parameter "%s" '
+ 'should start with "#optional"' %
+ (symbol['name'], name))
else:
# Just append it to the last one we added
symbol[mode][last_arg] += ' ' + line
--
To view, visit http://gerrit.ovirt.org/10446
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I94f92459ee8787780a54a510b6f8cc074fb2a235
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: ShaoHe Feng <shaohef(a)linux.vnet.ibm.com>
10 years, 2 months
Change in vdsm[master]: hook: spiceoptions: To provide spice option attributes to vm
by hchiramm＠redhat.com
Humble Devassy Chirammal has uploaded a new change for review.
Change subject: hook: spiceoptions: To provide spice option attributes to vm
......................................................................
hook: spiceoptions: To provide spice option attributes to vm
This hook goes through VM definitions xml file and manipulate its
graphics device if the protocol type is spice. This can be used to
configure spice options. This hook script will be really useful to
configure some of the spice optimization attributes and values
like image, streaming, clipboard..etc
For ex:
<graphics type='spice' port='-1' tlsPort='-1' autoport='yes'>
******
<image compression='auto_glz'/>
<streaming mode='filter'/>
<clipboard copypaste='no'/>
<mouse mode='client'/>
</graphics>
Change-Id: I7761aab98cdc3a0d37bed7984ae540278e6bbf70
Signed-off-by: Humble Chirammal <hchiramm(a)redhat.com>
---
M configure.ac
M vdsm.spec.in
M vdsm_hooks/Makefile.am
A vdsm_hooks/spiceoptions/Makefile.am
A vdsm_hooks/spiceoptions/README
A vdsm_hooks/spiceoptions/before_vm_start.py
6 files changed, 248 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/78/22178/1
diff --git a/configure.ac b/configure.ac
index ce05834..253fc90 100644
--- a/configure.ac
+++ b/configure.ac
@@ -295,6 +295,7 @@
vdsm_hooks/qos/Makefile
vdsm_hooks/scratchpad/Makefile
vdsm_hooks/smbios/Makefile
+ vdsm_hooks/spiceoptions/Makefile
vdsm_hooks/sriov/Makefile
vdsm_hooks/vhostmd/Makefile
vdsm_hooks/vmdisk/Makefile
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 385a29b..166ff3d 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -528,6 +528,15 @@
sr-iov hook enable to add virtual functions exposed by the device
directly to a virtual machine.
+
+%package hook-spiceoptions
+Summary: To configure spice options for vm
+BuildArch: noarch
+
+%description hook-spiceoptions
+This vdsm hook can be used to configure some of
+the spice optimization attributes and values..
+
%package hook-vmfex
Summary: vmfex support for VDSM
BuildArch: noarch
@@ -1225,6 +1234,11 @@
%{_libexecdir}/%{vdsm_name}/hooks/after_vm_destroy/50_sriov
%{_libexecdir}/%{vdsm_name}/hooks/before_vm_migrate_source/50_sriov
+
+%files hook-spiceoptions
+%defattr(-, root, root, -)
+%{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/60_spiceoptions
+
%files hook-vmdisk
%defattr(-, root, root, -)
%{_libexecdir}/%{vdsm_name}/hooks/before_vm_start/50_vmdisk
diff --git a/vdsm_hooks/Makefile.am b/vdsm_hooks/Makefile.am
index 8d446bb..d8b213c 100644
--- a/vdsm_hooks/Makefile.am
+++ b/vdsm_hooks/Makefile.am
@@ -41,6 +41,7 @@
scratchpad \
smbios \
sriov \
+ spiceoptions \
vmdisk \
vmfex
endif
diff --git a/vdsm_hooks/spiceoptions/Makefile.am b/vdsm_hooks/spiceoptions/Makefile.am
new file mode 100644
index 0000000..0aa5486
--- /dev/null
+++ b/vdsm_hooks/spiceoptions/Makefile.am
@@ -0,0 +1,30 @@
+#
+# Copyright 2013 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+#
+EXTRA_DIST = \
+ before_vm_start.py
+
+install-data-local:
+ $(MKDIR_P) $(DESTDIR)$(vdsmhooksdir)/before_vm_start
+ $(INSTALL_SCRIPT) $(srcdir)/before_vm_start.py \
+ $(DESTDIR)$(vdsmhooksdir)/before_vm_start/50_spiceoptions
+
+uninstall-local:
+ $(RM) $(DESTDIR)$(vdsmhooksdir)/before_vm_start/50_spiceoptions
diff --git a/vdsm_hooks/spiceoptions/README b/vdsm_hooks/spiceoptions/README
new file mode 100644
index 0000000..2bef8cc
--- /dev/null
+++ b/vdsm_hooks/spiceoptions/README
@@ -0,0 +1,62 @@
+spiceoptions vdsm hook:
+===============================
+
+This hook goes through VM definitions xml file and manipulate its
+graphics device if the protocol type is spice. This can be used to
+configure spice options on the fly. That said, this hook script
+will be really useful to configure some of the spice optimization
+attributes and values..
+
+For ex:
+
+ <graphics type='spice' port='-1' tlsPort='-1' autoport='yes'>
+ ******
+ <image compression='auto_glz'/>
+ <streaming mode='filter'/>
+ <clipboard copypaste='no'/>
+ <mouse mode='client'/>
+ </graphics>
+
+Spice supports variable compression settings for audio, images
+and streaming. These settings are accessible via the compression
+attribute in all following elements: image to set image compression
+(accepts auto_glz, auto_lz, quic, glz, lz, off),jpeg for JPEG
+compression for images over wan (accepts auto, never, always),
+zlib for configuring wan image compression (accepts auto, never, always)
+..etc.
+
+Streaming mode is set by the streaming element, settings its mode
+attribute to one of filter, all or off.
+
+Copy & Paste functionality (via Spice agent) is set by the clipboard
+element.It is enabled by default, and can be disabled by setting
+the copypaste property to no.
+
+Mouse mode is set by the mouse element, setting its mode attribute
+to one of server or client.If no mode is specified, the qemu default
+will be used (client mode).
+
+Syntax:
+ spiceoptions={'element': {'attribute': 'value'},..
+
+For ex:
+ spiceoptions={'image': {'compression': 'auto_glz'},
+ 'jpeg': {'compressions': 'never'},
+ 'streaming':{'mode':'filter'},
+ 'clipboard':{'copypaste':'yes'}}..etc
+
+
+Installation:
+* Use the engine-config to append the appropriate custom property as such:
+ sudo engine-config -s UserDefinedVMProperties=
+ 'previousProperties;spiceoptions=^.*$' --cver=3.2
+
+* Verify that the spiceoptions custom property was properly added:
+ sudo engine-config -g UserDefinedVMProperties
+
+Usage:
+In the VM configuration window, open the custom properites tab
+and add spiceoptions=
+
+NOTE: Live migration is **not** tested.
+
diff --git a/vdsm_hooks/spiceoptions/before_vm_start.py b/vdsm_hooks/spiceoptions/before_vm_start.py
new file mode 100755
index 0000000..4b95742
--- /dev/null
+++ b/vdsm_hooks/spiceoptions/before_vm_start.py
@@ -0,0 +1,140 @@
+#!/usr/bin/python
+
+import os
+import sys
+import hooking
+import traceback
+import ast
+
+'''
+Hook to configure spice options on a vm
+
+For ex:
+
+Syntax:
+ spiceoptions={'element': {'attribute': 'value'},..
+
+For ex:
+ spiceoptions={'image': {'compression': 'auto_glz'},
+ 'jpeg': {'compressions': 'never'},
+ 'streaming':{'mode':'filter'},
+ 'clipboard':{'copypaste':'yes'}}..etc
+
+
+ <graphics type='spice' port='-1' tlsPort='-1' autoport='yes'>
+ ******
+ <image compression='auto_glz'/>
+ <streaming mode='filter'/>
+ <clipboard copypaste='no'/>
+ <mouse mode='client'/>
+ </graphics>
+'''
+
+
+imageCompList = ['auto_glz', 'auto_lz', 'quic', 'glz', 'lz', 'off']
+
+jpegCompList = ['auto', 'never', 'always']
+
+zlibCompList = ['auto', 'never', 'always']
+
+copyPaste = ['yes', 'no']
+
+strModeList = ['filter', 'all', 'off']
+
+mouseModeList = ['server', 'client']
+
+spiceDict = {'image': {'compression': imageCompList},
+ 'jpeg': {'compression': jpegCompList},
+ 'streaming': {'mode': strModeList},
+ 'mouse': {'mode': mouseModeList},
+ 'clipboard': {'copypaste': copyPaste}}
+
+
+def createOptionElement(domxml, attrname, attroption, attrvalue):
+ if attrname == "jpeg":
+ jpegEle = domxml.createElement(attrname)
+ if attrvalue in jpegCompList:
+ jpegEle.setAttribute(attroption, attrvalue)
+ return jpegEle
+ else:
+ sys.stderr.write("\n InValid option:%s \t Available options are %s"
+ % (attrvalue, jpegCompList))
+ elif attrname == "zlib":
+ zlibEle = domxml.createElement(attrname)
+ if attrvalue in zlibCompList:
+ zlibEle.setAttribute(attroption, attrvalue)
+ return zlibEle
+ else:
+ sys.stderr.write("\n InValid option:%s \t Available options are %s"
+ % (attrvalue, zlibCompList))
+ elif attrname == "image":
+ imageEle = domxml.createElement(attrname)
+ if attrvalue in imageCompList:
+ imageEle.setAttribute(attroption, attrvalue)
+ return imageEle
+ else:
+ sys.stderr.write("\n InValid option:%s \t Available options are %s"
+ % (attrvalue, imageCompList))
+ elif attrname == "streaming":
+ streamEle = domxml.createElement(attrname)
+ if attrvalue in strModeList:
+ streamEle.setAttribute(attroption, attrvalue)
+ return streamEle
+ else:
+ sys.stderr.write("\n InValid option:%s \t Available options are %s"
+ % (attrvalue, strModeList))
+ elif attrname == "mouse":
+ mouseEle = domxml.createElement(attrname)
+ if attrvalue in mouseModeList:
+ mouseEle.setAttribute(attroption, attrvalue)
+ return mouseEle
+ else:
+ sys.stderr.write("\n InValid option:%s \t Available options are %s"
+ % (attrvalue, mouseModeList))
+
+ elif attrname == "clipboard":
+ clipEle = domxml.createElement(attrname)
+ if attrvalue in copyPaste:
+ clipEle.setAttribute(attroption, attrvalue)
+ return clipEle
+ else:
+ sys.stderr.write("\n InValid option:%s \t Available options are %s"
+ % (attrvalue, copyPaste))
+ else:
+ sys.stderr.write('\n No Valid Element created for graphic device :%s\n'
+ % (attrname))
+
+
+if 'spiceoptions' in os.environ:
+ try:
+ attrName = ''
+ attrVal = ''
+ attrOpt = ''
+ domxml = hooking.read_domxml()
+ spiceOpt = ast.literal_eval(os.environ['spiceoptions'])
+ spiceOpt = dict((k.lower(), v) for k, v in spiceOpt.iteritems())
+ for graphDev in domxml.getElementsByTagName('graphics'):
+ graphType = graphDev.getAttribute('type')
+ if graphType == 'spice':
+ for key, value in spiceOpt.items():
+ if key not in spiceDict.keys():
+ sys.stderr.write('\n Invalid KEY:%s \n' % (key,))
+ else:
+ for ak, bk in value.items():
+ if ak not in spiceDict[key].keys():
+ sys.stderr.write("\n Invalid KEY in :%s[%s] "
+ "options are:%s \n"
+ % (key, ak,
+ spiceDict[key].keys()))
+
+ else:
+ returnEle = createOptionElement(
+ domxml, key, ak, value[ak])
+ if returnEle:
+ graphDev.appendChild(returnEle)
+
+ hooking.write_domxml(domxml)
+ except:
+ sys.stderr.write('spiceoptions: [unexpected error]: %s\n' %
+ traceback.format_exc())
+ sys.exit(2)
--
To view, visit http://gerrit.ovirt.org/22178
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7761aab98cdc3a0d37bed7984ae540278e6bbf70
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Humble Devassy Chirammal <hchiramm(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: vm: improve extension check in extendDrivesIfNeeded
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: vm: improve extension check in extendDrivesIfNeeded
......................................................................
vm: improve extension check in extendDrivesIfNeeded
Change-Id: Ide5034cd5d87451c06a2ba034011fdf5696440e6
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M vdsm/vm.py
1 file changed, 23 insertions(+), 25 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/23/21423/1
diff --git a/vdsm/vm.py b/vdsm/vm.py
index e82867c..192f632 100644
--- a/vdsm/vm.py
+++ b/vdsm/vm.py
@@ -78,6 +78,9 @@
CONSOLE_DEVICES = 'console'
SMARTCARD_DEVICES = 'smartcard'
+# qcow2 format overhead is estimated to be 10% of capacity
+QCOW2_OVERHEAD = 0.1
+
def isVdsmImage(drive):
return all(k in drive.keys() for k in ('volumeID', 'domainID', 'imageID',
@@ -2288,46 +2291,41 @@
if not drive.blockDev or drive.format != 'cow':
continue
- capacity, alloc, physical = self._dom.blockInfo(drive.path, 0)
+ capacity, curAlloc, physical = self._dom.blockInfo(drive.path, 0)
- # Since the check based on nextPhysSize is extremly risky (it
- # may result in the VM being paused) we can't use the regular
- # getNextVolumeSize call as it relies on a cached value of the
- # drive apparentsize.
- nextPhysSize = physical + drive.VOLWM_CHUNK_MB * constants.MEGAB
+ if physical - curAlloc > drive.watermarkLimit:
+ continue # Extension is not needed
+
+ # When an extension is needed we compute the maxium allowed size
+ # taking in account the qcow2 format overhead and rounding it up
+ # to the next volume chunk.
+ extChunkBytes = drive.VOLWM_CHUNK_MB * constants.MEGAB
+ totAlloc = capacity * (1.0 + QCOW2_OVERHEAD)
+ maxAlloc = (
+ int((totAlloc + extChunkBytes - 1) / extChunkBytes) *
+ extChunkBytes
+ )
# NOTE: the intent of this check is to prevent faulty images to
# trick qemu in requesting extremely large extensions (BZ#998443).
- # Probably the definitive check would be comparing the allocated
- # space with capacity + format_overhead. Anyway given that:
- #
- # - format_overhead is tricky to be computed (it depends on few
- # assumptions that may change in the future e.g. cluster size)
- # - currently we allow only to extend by one chunk at time
- #
- # the current check compares alloc with the next volume size.
- # It should be noted that alloc cannot be directly compared with
- # the volume physical size as it includes also the clusters not
- # written yet (pending).
- if alloc > nextPhysSize:
+ if curAlloc > maxAlloc:
self.log.error(
"Improbable extension request for volume %s on domain "
"%s, pausing the VM to avoid corruptions (capacity: %s, "
- "allocated: %s, physical: %s, next physical size: %s)",
- drive.volumeID, drive.domainID, capacity, alloc,
- physical, nextPhysSize)
+ "allocated: %s, physical: %s, max allocation: %s)",
+ drive.volumeID, drive.domainID, capacity, curAlloc,
+ physical, maxAlloc)
self.pause(pauseCode='EOTHER')
return False
- if physical - alloc < drive.watermarkLimit:
- extend.append((drive, capacity, alloc, physical))
+ extend.append((drive, capacity, curAlloc, physical))
- for drive, capacity, alloc, physical in extend:
+ for drive, capacity, curAlloc, physical in extend:
self.log.info(
"Requesting extension for volume %s on domain %s (apparent: "
"%s, capacity: %s, allocated: %s, physical: %s)",
drive.volumeID, drive.domainID, drive.apparentsize, capacity,
- alloc, physical)
+ curAlloc, physical)
self.extendDriveVolume(drive)
return len(extend) > 0
--
To view, visit http://gerrit.ovirt.org/21423
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ide5034cd5d87451c06a2ba034011fdf5696440e6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
10 years, 2 months
Change in vdsm[master]: [WIP] Add new repository management code
by smizrahi＠redhat.com
Saggi Mizrahi has uploaded a new change for review.
Change subject: [WIP] Add new repository management code
......................................................................
[WIP] Add new repository management code
Change-Id: Ib09c89cf982b475f45d26b2428fe05e2f4565dab
Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com>
---
M tests/Makefile.am
A tests/imageMainuplatorTests.py
A tests/repositoryEngineTests.py
A vdsm/storage/imageRepository/engines/__init__.py
A vdsm/storage/imageRepository/engines/localfs/__init__.py
A vdsm/storage/imageRepository/imageManipulator.py
6 files changed, 1,339 insertions(+), 5 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/47/6247/1
--
To view, visit http://gerrit.ovirt.org/6247
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib09c89cf982b475f45d26b2428fe05e2f4565dab
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>
10 years, 2 months