Change in vdsm[master]: utils: Add changehash function for change detection
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: utils: Add changehash function for change detection
......................................................................
utils: Add changehash function for change detection
We use Python built-in hash to detect changes in vm state without sending
the state in each response. This function is not suitable for this
usage. Now we use generic utils.changehash(), implemented using md5
hexdigest.
Change-Id: I2242a594383e2d2fe64e3a581f18b8ac662648b0
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/utils.py
M vdsm/virt/vm.py
2 files changed, 13 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/45/33045/1
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 23c63e8..1b4a9d5 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -37,6 +37,7 @@
import glob
import io
import itertools
+import hashlib
import logging
import re
import sys
@@ -1133,3 +1134,13 @@
flags = fcntl.fcntl(fd, fcntl.F_GETFL)
flags |= os.O_NONBLOCK
fcntl.fcntl(fd, fcntl.F_SETFL, flags)
+
+
+def changehash(s):
+ """
+ Returns a hash of string s, suitable for change detection.
+
+ Tipically changehash(s) is sent to client frequently. When a client detect
+ that changehash(s) changed, it ask for s itself, which may be much bigger.
+ """
+ return hashlib.md5(s).hexdigest()
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 941f283..b1567f9 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -1500,7 +1500,7 @@
self.guestAgent = guestagent.GuestAgent(
self._guestSocketFile, self.cif.channelListener, self.log)
self._lastXMLDesc = '<domain><uuid>%s</uuid></domain>' % self.id
- self._devXmlHash = '0'
+ self._devXmlHash = utils.changehash('')
self._released = False
self._releaseLock = threading.Lock()
self.saveState()
@@ -4495,7 +4495,7 @@
self._lastXMLDesc = self._dom.XMLDesc(0)
devxml = _domParseStr(self._lastXMLDesc).childNodes[0]. \
getElementsByTagName('devices')[0]
- self._devXmlHash = str(hash(devxml.toxml()))
+ self._devXmlHash = utils.changehash(devxml.toxml())
return self._lastXMLDesc
--
To view, visit http://gerrit.ovirt.org/33045
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2242a594383e2d2fe64e3a581f18b8ac662648b0
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
7 years, 8 months
Change in vdsm[master]: log: Use INFO log level as default
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: log: Use INFO log level as default
......................................................................
log: Use INFO log level as default
The current logs are much too verbose which cause trouble for users, and
make us look unprofessional. Mature project should not use debug log by
default.
To debug issues that are not clear enough using INFO logs, the relevant
logger level can be modified on a user machine as needed.
Change-Id: I767dcd9bad7b9fbeebb438e9ef13cb0ec3f042ee
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/logger.conf.in
1 file changed, 4 insertions(+), 4 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/04/32504/1
diff --git a/vdsm/logger.conf.in b/vdsm/logger.conf.in
index 64b154f..8e963dd 100644
--- a/vdsm/logger.conf.in
+++ b/vdsm/logger.conf.in
@@ -8,18 +8,18 @@
keys=long,simple,none,sysform
[logger_root]
-level=DEBUG
+level=INFO
handlers=syslog,logfile
propagate=0
[logger_vds]
-level=DEBUG
+level=INFO
handlers=syslog,logfile
qualname=vds
propagate=0
[logger_Storage]
-level=DEBUG
+level=INFO
handlers=logfile
qualname=Storage
propagate=0
@@ -31,7 +31,7 @@
propagate=1
[logger_connectivity]
-level=DEBUG
+level=INFO
handlers=connlogfile
qualname=connectivity
propagate=0
--
To view, visit http://gerrit.ovirt.org/32504
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I767dcd9bad7b9fbeebb438e9ef13cb0ec3f042ee
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
7 years, 9 months
Change in vdsm[master]: tests: Add time per test if nose-timer is available
by asegurap＠redhat.com
Antoni Segura Puimedon has uploaded a new change for review.
Change subject: tests: Add time per test if nose-timer is available
......................................................................
tests: Add time per test if nose-timer is available
This patch adds an output like:
resourceManagerTests.ResourceManagerTests.testStressTest: 12.8313s
resourceManagerTests.ResourceManagerTests.testResourceAcquireTimeout: 1.0069s
resourceManagerTests.ResourceManagerTests.testResourceLockSwitch: 0.0108s
resourceManagerTests.ResourceManagerTests.testResourceAutorelease: 0.0101s
resourceManagerTests.ResourceManagerTests.testResourceInvalidation: 0.0093s
resourceManagerTests.ResourceManagerTests.testCrashOnSwitch: 0.0067s
resourceManagerTests.ResourceManagerTests.testResourceSwitchLockTypeFail:
0.0062s
resourceManagerTests.ResourceManagerTests.testCancelExclusiveBetweenShared:
0.0049s
resourceManagerTests.ResourceManagerTests.testAcquireResourceExclusive: 0.0042s
resourceManagerTests.ResourceManagerTests.testCancelRequest: 0.0031s
resourceManagerTests.ResourceManagerTests.testRequestRefCmp: 0.0030s
resourceManagerTests.ResourceManagerTests.testRequestRecancel: 0.0030s
resourceManagerTests.ResourceManagerTests.testResourceStatuses: 0.0029s
resourceManagerTests.ResourceManagerTests.testAccessAttributeNotExposedByRequestRef:
0.0029s
resourceManagerTests.ResourceManagerTests.testFailCreateAfterSwitch: 0.0025s
resourceManagerTests.ResourceManagerTests.testRequestWithBadCallbackOnCancel:
0.0025s
resourceManagerTests.ResourceManagerTests.testAcquireResourceShared: 0.0023s
resourceManagerTests.ResourceManagerTests.testRereleaseResource: 0.0022s
resourceManagerTests.ResourceManagerTests.testResourceWrapper: 0.0020s
resourceManagerTests.ResourceManagerTests.testRequestWithBadCallbackOnGrant:
0.0020s
resourceManagerTests.ResourceManagerTests.testRequestRefStr: 0.0020s
resourceManagerTests.ResourceManagerTests.testAccessAttributeNotExposedByWrapper:
0.0019s
resourceManagerTests.ResourceManagerTests.testErrorInFactory: 0.0017s
resourceManagerTests.ResourceManagerTests.testAcquireNonExistingResource:
0.0016s
resourceManagerTests.ResourceManagerTests.testRequestInvalidResource: 0.0015s
resourceManagerTests.ResourceManagerTests.testRequestRegrant: 0.0015s
resourceManagerTests.ResourceManagerTests.testResourceAcquireInvalidTimeout:
0.0014s
resourceManagerTests.ResourceManagerTests.testReleaseInvalidResource: 0.0014s
resourceManagerTests.ResourceManagerTests.testSingleton: 0.0014s
resourceManagerTests.ResourceManagerTests.testForceRegisterNamespace: 0.0013s
resourceManagerTests.ResourceManagerTests.testListNamespaces: 0.0013s
resourceManagerTests.ResourceManagerTests.testReregisterNamespace: 0.0012s
resourceManagerTests.ResourceManagerTests.testRegisterInvalidNamespace: 0.0012s
At the end of the usual tests if nose-timer is detected in the
system. It shows the tests over 500ms in yellow and those over 5s
in red.
If you want to get this output, do:
pip install nose-timer
Change-Id: I4960b30532a84abd259fb268c7e40a1751847a96
Signed-off-by: Antoni S. Puimedon <asegurap(a)redhat.com>
---
M tests/run_tests_local.sh.in
M tests/testrunner.py
2 files changed, 18 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/30171/1
diff --git a/tests/run_tests_local.sh.in b/tests/run_tests_local.sh.in
index d19f22c..f51473d 100644
--- a/tests/run_tests_local.sh.in
+++ b/tests/run_tests_local.sh.in
@@ -7,4 +7,9 @@
@top_srcdir(a)/tests/makecert.sh
fi
-PYTHONDONTWRITEBYTECODE=1 LC_ALL=C PYTHONPATH="@top_srcdir@/lib:@top_srcdir@/vdsm:@top_srcdir@/client:@top_srcdir@/vdsm_api:$PYTHONPATH" "$PYTHON_EXE" @top_srcdir(a)/tests/testrunner.py --local-modules $@
+python -c 'import nosetimer' > /dev/null 2>&1
+if [ $? -eq 0 ]; then
+ EXTRA_ARGS='--with-timer'
+fi
+
+PYTHONDONTWRITEBYTECODE=1 LC_ALL=C PYTHONPATH="@top_srcdir@/lib:@top_srcdir@/vdsm:@top_srcdir@/client:@top_srcdir@/vdsm_api:$PYTHONPATH" "$PYTHON_EXE" @top_srcdir(a)/tests/testrunner.py --local-modules $@ "$EXTRA_ARGS"
diff --git a/tests/testrunner.py b/tests/testrunner.py
index eed743f..ec24c2b 100644
--- a/tests/testrunner.py
+++ b/tests/testrunner.py
@@ -40,6 +40,16 @@
from nose import config
from nose import core
from nose import result
+try:
+ from nosetimer.plugin import TimerPlugin
+except ImportError:
+ timer_plugin = None
+else:
+ timer_plugin = TimerPlugin()
+ timer_plugin.enabled = True
+ timer_plugin.timer_ok = 500 # Okay <= 500ms
+ timer_plugin.timer_warning = 5000 # Warn > 5s
+ timer_plugin.timer_no_color = False # Enable color output
from testValidation import SlowTestsPlugin, StressTestsPlugin
@@ -378,6 +388,8 @@
plugins=core.DefaultPluginManager())
conf.plugins.addPlugin(SlowTestsPlugin())
conf.plugins.addPlugin(StressTestsPlugin())
+ if timer_plugin is not None:
+ conf.plugins.addPlugin(timer_plugin)
runner = VdsmTestRunner(stream=conf.stream,
verbosity=conf.verbosity,
--
To view, visit http://gerrit.ovirt.org/30171
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4960b30532a84abd259fb268c7e40a1751847a96
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com>
7 years, 9 months
Change in vdsm[master]: tests: Add a live merge functional test
by alitke＠redhat.com
Adam Litke has uploaded a new change for review.
Change subject: tests: Add a live merge functional test
......................................................................
tests: Add a live merge functional test
Test whether we can successfully merge the active layer. Uses lots of
the functional test infrastructure! Only runs if vdsm says it can
support live merge.
Change-Id: Idd5a2f7eedaef9e90981256de66fc3ed21658e89
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M tests/functional/utils.py
M tests/functional/virtTests.py
2 files changed, 185 insertions(+), 5 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/29824/1
diff --git a/tests/functional/utils.py b/tests/functional/utils.py
index 494be98..e3cdba6 100644
--- a/tests/functional/utils.py
+++ b/tests/functional/utils.py
@@ -228,3 +228,41 @@
def updateVmPolicy(self, vmId, vcpuLimit):
result = self.vdscli.updateVmPolicy([vmId, vcpuLimit])
return result['status']['code'], result['status']['message']
+
+ def getTaskStatus(self, taskId):
+ result = self.vdscli.getTaskStatus(taskId)
+ return result['status']['code'], result['status']['message'],\
+ result['taskStatus']
+
+ def getVolumeInfo(self, sdId, spId, imgId, volId):
+ result = self.vdscli.getVolumeInfo(sdId, spId, imgId, volId)
+ return result['status']['code'], result['status']['message'],\
+ result['info']
+
+ def createVolume(self, sdId, spId, imgId, size, volFormat, preallocate,
+ diskType, volId, desc, baseImgId, baseVolId):
+ result = self.vdscli.createVolume(sdId, spId, imgId, size, volFormat,
+ preallocate, diskType, volId, desc,
+ baseImgId, baseVolId)
+ return result['status']['code'], result['status']['message'],\
+ result['uuid']
+
+ def deleteVolume(self, sdId, spId, imgId, volIds, postZero=False,
+ force=False):
+ result = self.vdscli.deleteVolume(sdId, spId, imgId, volIds, postZero,
+ force)
+ return result['status']['code'], result['status']['message'],\
+ result['uuid']
+
+ def snapshot(self, vmId, snapDrives, snapMemVolHandle=''):
+ result = self.vdscli.snapshot(vmId, snapDrives, snapMemVolHandle)
+ return result['status']['code'], result['status']['message']
+
+ def merge(self, vmId, drive, base, top, bandwidth, jobId):
+ result = self.vdscli.merge(vmId, drive, base, top, bandwidth, jobId)
+ return result['status']['code'], result['status']['message']
+
+ def list(self, fullStatus=False, vmList=()):
+ result = self.vdscli.list(fullStatus, vmList)
+ return result['status']['code'], result['status']['message'], \
+ result['vmList']
diff --git a/tests/functional/virtTests.py b/tests/functional/virtTests.py
index 94ce240..b811b92 100644
--- a/tests/functional/virtTests.py
+++ b/tests/functional/virtTests.py
@@ -22,6 +22,7 @@
import math
import tempfile
import logging
+import uuid
from stat import S_IROTH
from functools import partial, wraps
@@ -32,7 +33,8 @@
from testrunner import temporaryPath
from vdsm.utils import CommandPath, RollbackContext
-import storageTests as storage
+import storageTests
+import storage
from storage.misc import execCmd
from utils import VdsProxy, SUCCESS
@@ -109,6 +111,18 @@
return method(self, *args, **kwargs)
else:
raise SkipTest('KVM is not enabled')
+ return wrapped
+
+
+def requireLiveMerge(method):
+ @wraps(method)
+ def wrapped(self, *args, **kwargs):
+ status, msg, result = self.vdsm.getVdsCapabilities()
+ self.assertEqual(status, SUCCESS, msg)
+ if result.get('liveMerge') == 'true':
+ return method(self, *args, **kwargs)
+ else:
+ raise SkipTest('Live Merge is not available')
return wrapped
@@ -227,9 +241,9 @@
@requireKVM
@permutations([['localfs'], ['iscsi'], ['nfs']])
def testVmWithStorage(self, backendType):
- disk = storage.StorageTest()
+ disk = storageTests.StorageTest()
disk.setUp()
- conf = storage.storageLayouts[backendType]
+ conf = storageTests.storageLayouts[backendType]
drives = disk.generateDriveConf(conf)
customization = {'vmId': '88888888-eeee-ffff-aaaa-111111111111',
'vmName': 'testVmWithStorage' + backendType,
@@ -247,8 +261,8 @@
def testVmWithDevice(self, *devices):
customization = {'vmId': '77777777-ffff-3333-bbbb-222222222222',
'vmName': 'testVm', 'devices': [], 'display': 'vnc'}
- storageLayout = storage.storageLayouts['localfs']
- diskSpecs = storage.StorageTest.generateDriveConf(storageLayout)
+ storageLayout = storageTests.storageLayouts['localfs']
+ diskSpecs = storageTests.StorageTest.generateDriveConf(storageLayout)
pciSpecs = {'bus': '0x00', 'domain': '0x0000',
'function': '0x0', 'type': 'pci'}
ccidSpecs = {'slot': '0', 'controller': '0', 'type': 'ccid'}
@@ -412,3 +426,131 @@
self.vdsm.updateVmPolicy(customization['vmId'],
'50')
self.assertEqual(status, SUCCESS, msg)
+
+
+@expandPermutations
+class LiveMergeTest(VirtTestBase):
+ def _waitTask(self, taskId):
+ def assertTaskOK():
+ status, msg, result = self.vdsm.getTaskStatus(taskId)
+ self.assertEqual(status, SUCCESS, msg)
+ self.assertEquals(result['taskState'], 'finished')
+
+ self.retryAssert(assertTaskOK, timeout=60)
+
+ def _waitBlockJobs(self, vmId, jobIds):
+ def assertJobsGone():
+ status, msg, result = self.vdsm.getVmStats(vmId)
+ self.assertEqual(status, SUCCESS, msg)
+ self.assertTrue('vmJobs' in result)
+ self.assertTrue(all([x not in result['vmJobs'].keys()
+ for x in jobIds]))
+
+ self.retryAssert(assertJobsGone, timeout=60)
+
+ def _snapshotVM(self, vmId, drives, rollback):
+ snapDrives = []
+ for drive in drives:
+ sd = drive['domainID']
+ sp = drive['poolID']
+ img = drive['imageID']
+ vol = drive['volumeID']
+ newVol = str(uuid.uuid4())
+ volFormat = storage.volume.COW_FORMAT
+ preallocate = storage.volume.SPARSE_VOL
+ desc = 'snapshot for %s' % vol
+
+ # Create volume and wait
+ status, msg, result = self.vdsm.getVolumeInfo(sd, sp, img, vol)
+ self.assertEqual(status, SUCCESS, msg)
+ size = result['capacity']
+ diskType = result['disktype']
+
+ status, msg, taskId = self.vdsm.createVolume(sd, sp, img, size,
+ volFormat,
+ preallocate, diskType,
+ newVol, desc, img,
+ vol)
+ self.assertEqual(status, SUCCESS, msg)
+ self._waitTask(taskId)
+ undo = lambda sd=sd, sp=sp, img=img, vol=newVol: \
+ self._waitTask(self.vdsm.deleteVolume(sd, sp, img, vol)[2])
+ rollback.prependDefer(undo)
+
+ snapDrives.append({'domainID': sd,
+ 'imageID': img,
+ 'volumeID': newVol,
+ 'baseVolumeID': vol})
+
+ # Create snapshot
+ status, msg = self.vdsm.snapshot(vmId, snapDrives)
+ self.assertEqual(status, SUCCESS, msg)
+ return snapDrives
+
+ def _orderChain(self, vmId, dev, chain):
+ parentMap = {}
+ for vol in chain:
+ status, msg, info = self.vdsm.getVolumeInfo(dev['domainID'],
+ dev['poolID'],
+ dev['imageID'], vol)
+ self.assertEqual(status, SUCCESS, msg)
+ parent = info['parent']
+ parentMap[vol] = parent
+
+ vol = dev['volumeID']
+ chain = list()
+ while True:
+ chain.insert(0, vol)
+ vol = parentMap.get(vol, '00000000-0000-0000-0000-000000000000')
+ if vol == '00000000-0000-0000-0000-000000000000':
+ break
+ return chain
+
+ def _getVolumeChains(self, vmId):
+ chains = {}
+ status, msg, result = self.vdsm.list(True, (vmId,))
+ self.assertEqual(status, SUCCESS, msg)
+ vmDef = result[0]
+ for dev in vmDef['devices']:
+ if dev['device'] != 'disk':
+ continue
+ chains[dev['imageID']] = self._orderChain(vmId, dev,
+ [x['volumeID'] for x in
+ dev['volumeChain']])
+ return chains
+
+ @requireKVM
+ @requireLiveMerge
+ def testCapable(self):
+ pass
+
+ @permutations([['localfs']])
+ def testMergeActiveLayer(self, backendType):
+ disk = storageTests.StorageTest()
+ disk.setUp()
+ conf = storageTests.storageLayouts[backendType]
+ drives = disk.generateDriveConf(conf)
+ vmId = '12121212-abab-baba-abab-222222222222'
+ customization = {'vmId': vmId,
+ 'vmName': 'testMergeActive' + backendType,
+ 'drives': drives,
+ 'display': 'vnc'}
+
+ with RollbackContext() as rollback:
+ disk.createVdsmStorageLayout(conf, 3, rollback)
+ with RunningVm(self.vdsm, customization) as vm:
+ self._waitForStartup(vm, VM_MINIMAL_UPTIME)
+ snapDrives = self._snapshotVM(vmId, drives, rollback)
+ chains = {}
+ jobIds = []
+ for drive in snapDrives:
+ base = drive['baseVolumeID']
+ top = drive['volumeID']
+ jobId = str(uuid.uuid4())
+ chains[drive['imageID']] = [base, top]
+ status, msg = self.vdsm.merge(vmId, drive, base, top, 0,
+ jobId)
+ jobIds.append(jobId)
+ self._waitBlockJobs(vmId, jobIds)
+ actual = self._getVolumeChains(vmId)
+ self.assertEquals(chains, actual)
--
To view, visit http://gerrit.ovirt.org/29824
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Idd5a2f7eedaef9e90981256de66fc3ed21658e89
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
7 years, 9 months
Change in vdsm[master]: block: add blkdiscard on zero image
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: block: add blkdiscard on zero image
......................................................................
block: add blkdiscard on zero image
Change-Id: I4e059c556c550440727b36b8af8e5dfc29ce2ccb
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M vdsm/storage/blockSD.py
1 file changed, 2 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/31/35631/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index daaa94f..a2370b8 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -33,6 +33,7 @@
from vdsm.config import config
from vdsm import constants
from vdsm import utils
+from vdsm import utillinux
import misc
import fileUtils
import sd
@@ -219,6 +220,7 @@
try:
misc.ddWatchCopy("/dev/zero", path, aborting, size)
+ utillinux.blkdiscard(path)
except Exception as e:
log.exception('zeroing operation failed')
raise se.VolumesZeroingError(path)
--
To view, visit http://gerrit.ovirt.org/35631
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I4e059c556c550440727b36b8af8e5dfc29ce2ccb
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
8 years, 5 months
Change in vdsm[master]: lvm: Exclude faulty devices from lvm long filter
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: lvm: Exclude faulty devices from lvm long filter
......................................................................
lvm: Exclude faulty devices from lvm long filter
lvm commands use filter to limit access to relevant devices. When the
filter includes a faulty device, lvm commands may block for several
minutes (stuck in D state). We have seen getDevicesList command stuck
for up to 10 minutes because of faulty devices in the long filter.
We used to build the filter from all multipath devices. Now we build the
filter only from devices which have at least one active paths.
# multipath -ll
360060160f4a0300038ed7058b5e9e311 dm-0 DGC ,VRAID
size=15G features='0' hwhandler='1 emc' wp=rw
|-+- policy='service-time 0' prio=0 status=enabled
| `- 4:0:3:0 sdd 8:48 failed faulty running
`-+- policy='service-time 0' prio=0 status=enabled
`- 4:0:2:0 sdb 8:16 failed faulty running
360060160f4a030003268ab211002e411 dm-1 DGC ,VRAID
size=30G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
|-+- policy='service-time 0' prio=4 status=active
| `- 4:0:3:1 sde 8:64 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
`- 4:0:2:1 sdc 8:32 active ready running
Previously, both devices were included in the filter, now only
360060160f4a030003268ab211002e411 will be included in lvm filter.
A faulty device which became active again will be included in lvm filter
after the next refresh (every 5 minutes), or after trying edit or create
a new storage domain.
lvm uses also short filter, including devices used by the certain vg or
lv. It is possible that we also have to exclude such devices from the
short filter. This will be handled later if needed.
Change-Id: I6d7a973bcefa95813fdc289847760c0955aca30c
Bug-Url: https://bugzilla.redhat.com/880738
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/lvm.py
M vdsm/storage/multipath.py
2 files changed, 13 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/75/31875/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index 86edf55..9cfad01 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -244,7 +244,7 @@
if not self._filterStale:
return self._extraCfg
- self._extraCfg = _buildConfig(multipath.getMPDevNamesIter())
+ self._extraCfg = _buildConfig(multipath.getActiveMPDevNamesIter())
_updateLvmConf(self._extraCfg)
self._filterStale = False
diff --git a/vdsm/storage/multipath.py b/vdsm/storage/multipath.py
index ba98866..2b30995 100644
--- a/vdsm/storage/multipath.py
+++ b/vdsm/storage/multipath.py
@@ -382,6 +382,18 @@
yield os.path.join(devicemapper.DMPATH_PREFIX, name)
+def getActiveMPDevNamesIter():
+ status = devicemapper.getPathsStatus()
+ for dmId, guid in getMPDevsIter():
+ active = [slave for slave in devicemapper.getSlaves(dmId)
+ if status.get(slave) == "active"]
+ if not active:
+ log.warning("Skipping device %s - no active slave", guid)
+ continue
+ log.debug("Found device %s %s", guid, active)
+ yield os.path.join(devicemapper.DMPATH_PREFIX, guid)
+
+
def getMPDevsIter():
"""
Collect the list of all the multipath block devices.
--
To view, visit http://gerrit.ovirt.org/31875
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6d7a973bcefa95813fdc289847760c0955aca30c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
8 years, 5 months
Change in vdsm[master]: utils: allow retry of multiple functions
by ykaplan＠redhat.com
Yeela Kaplan has uploaded a new change for review.
Change subject: utils: allow retry of multiple functions
......................................................................
utils: allow retry of multiple functions
Change-Id: I6c805ef33b710fbbe674ac62ae96c3fdf47423e1
Signed-off-by: Yeela Kaplan <ykaplan(a)redhat.com>
---
M lib/vdsm/libvirtconnection.py
M lib/vdsm/utils.py
M tests/functional/vmRecoveryTests.py
M tests/miscTests.py
M tests/remoteFileHandlerTests.py
M tests/utilsTests.py
M vdsm/clientIF.py
M vdsm/storage/hsm.py
M vdsm/supervdsm.py
9 files changed, 18 insertions(+), 13 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/36336/1
diff --git a/lib/vdsm/libvirtconnection.py b/lib/vdsm/libvirtconnection.py
index 492a02e..55af991 100644
--- a/lib/vdsm/libvirtconnection.py
+++ b/lib/vdsm/libvirtconnection.py
@@ -99,7 +99,7 @@
libvirtOpen = functools.partial(
libvirt.openAuth, uri, auth, 0)
- return utils.retry(libvirtOpen, timeout=10, sleep=0.2)
+ return utils.retry((libvirtOpen,), timeout=10, sleep=0.2)
def get(target=None, killOnFailure=True):
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index db38e0c..ed4dec6 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -903,13 +903,13 @@
return unicode(self.cmd)
-def retry(func, expectedException=Exception, tries=None,
+def retry(funcs, expectedException=Exception, tries=None,
timeout=None, sleep=1, stopCallback=None):
"""
- Retry a function. Wraps the retry logic so you don't have to
+ Retry functions. Wraps the retry logic so you don't have to
implement it each time you need it.
- :param func: The callable to run.
+ :param funcs: a tuple of the callables to run.
:param expectedException: The exception you expect to receive when the
function fails.
:param tries: The number of times to try. None\0,-1 means infinite.
@@ -932,7 +932,9 @@
while True:
tries -= 1
try:
- return func()
+ for func in funcs:
+ res = func()
+ return res
except expectedException:
if tries == 0:
raise
diff --git a/tests/functional/vmRecoveryTests.py b/tests/functional/vmRecoveryTests.py
index 37a96f3..0028f62 100644
--- a/tests/functional/vmRecoveryTests.py
+++ b/tests/functional/vmRecoveryTests.py
@@ -96,7 +96,8 @@
def ensure_vdsm_started(self):
vdsm.utils.retry(
- self.setUp, expectedException=(socket.error, KeyError), tries=10)
+ (self.setUp,), expectedException=(socket.error, KeyError),
+ tries=10)
@ValidateRunningAsRoot
def test_vm_recovery(self):
diff --git a/tests/miscTests.py b/tests/miscTests.py
index a955979..eb8b11d 100644
--- a/tests/miscTests.py
+++ b/tests/miscTests.py
@@ -1047,7 +1047,7 @@
nice = utils.pidStat(proc.pid).nice
self.assertEquals(nice, 10)
- utils.retry(AssertionError, test, tries=10, sleep=0.1)
+ utils.retry((test,), AssertionError, tries=10, sleep=0.1)
proc.kill()
proc.wait()
diff --git a/tests/remoteFileHandlerTests.py b/tests/remoteFileHandlerTests.py
index 0c62f60..c093c2d 100644
--- a/tests/remoteFileHandlerTests.py
+++ b/tests/remoteFileHandlerTests.py
@@ -70,7 +70,7 @@
p.stop()
test = lambda: self.assertFalse(os.path.exists(procPath))
- utils.retry(test, AssertionError, timeout=4, sleep=0.1)
+ utils.retry((test,), AssertionError, timeout=4, sleep=0.1)
class RemoteFileHandlerTruncateTests(TestCaseBase):
diff --git a/tests/utilsTests.py b/tests/utilsTests.py
index cb3da85..75119e8 100644
--- a/tests/utilsTests.py
+++ b/tests/utilsTests.py
@@ -57,7 +57,8 @@
"fool about it.")
# W. C. Fields
- self.assertRaises(RuntimeError, utils.retry, foo, tries=(limit + 10),
+ self.assertRaises(RuntimeError, utils.retry, (foo,),
+ tries=(limit + 10),
sleep=0, stopCallback=stopCallback)
# Make sure we had the proper amount of iterations before failing
self.assertEquals(counter[0], limit)
@@ -111,7 +112,7 @@
try:
test = lambda: self.assertEquals(utils.getCmdArgs(sproc.pid),
tuple())
- utils.retry(AssertionError, test, tries=10, sleep=0.1)
+ utils.retry((test,), AssertionError, tries=10, sleep=0.1)
finally:
sproc.wait()
diff --git a/vdsm/clientIF.py b/vdsm/clientIF.py
index 971d5ff..673975a 100644
--- a/vdsm/clientIF.py
+++ b/vdsm/clientIF.py
@@ -393,7 +393,7 @@
# Trying to run recover process until it works. During that time vdsm
# stays in recovery mode (_recover=True), means all api requests
# returns with "vdsm is in initializing process" message.
- utils.retry(self._recoverExistingVms, sleep=5)
+ utils.retry((self._recoverExistingVms,), sleep=5)
def _recoverExistingVms(self):
try:
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index c8aaf93..fa6eb5a 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -3155,7 +3155,7 @@
supervdsm.getProxy().appropriateDevice(guid, thiefId)
supervdsm.getProxy().udevTrigger(guid)
devPath = os.path.join(devicemapper.DMPATH_PREFIX, guid)
- utils.retry(partial(fileUtils.validateQemuReadable, devPath),
+ utils.retry((partial(fileUtils.validateQemuReadable, devPath),),
expectedException=OSError,
timeout=QEMU_READABLE_TIMEOUT)
diff --git a/vdsm/supervdsm.py b/vdsm/supervdsm.py
index 8e42af2..c29a1f2 100644
--- a/vdsm/supervdsm.py
+++ b/vdsm/supervdsm.py
@@ -75,7 +75,8 @@
self._manager.register('open')
self._log.debug("Trying to connect to Super Vdsm")
try:
- utils.retry(self._manager.connect, Exception, timeout=60, tries=3)
+ utils.retry((self._manager.connect,), Exception,
+ timeout=60, tries=3)
except Exception as ex:
msg = "Connect to supervdsm service failed: %s" % ex
utils.panic(msg)
--
To view, visit http://gerrit.ovirt.org/36336
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6c805ef33b710fbbe674ac62ae96c3fdf47423e1
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yeela Kaplan <ykaplan(a)redhat.com>
8 years, 5 months
Change in vdsm[master]: lib: add support for utillinux commands
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: lib: add support for utillinux commands
......................................................................
lib: add support for utillinux commands
Change-Id: I2ea7dd19fadc600b8fe78fb436ae430d35f52165
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M lib/vdsm/Makefile.am
A lib/vdsm/utillinux.py
M tests/Makefile.am
A tests/utillinuxTests.py
M vdsm.spec.in
5 files changed, 132 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/35629/1
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index b862e71..4e0868a 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -33,6 +33,7 @@
profile.py \
qemuimg.py \
sslutils.py \
+ utillinux.py \
utils.py \
vdscli.py \
virtsparsify.py \
diff --git a/lib/vdsm/utillinux.py b/lib/vdsm/utillinux.py
new file mode 100644
index 0000000..7fb42cf
--- /dev/null
+++ b/lib/vdsm/utillinux.py
@@ -0,0 +1,47 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+import os
+import re
+import signal
+
+from . import utils
+
+_blkdiscard = utils.CommandPath("blkdiscard",
+ "/sbin/blkdiscard",) # Fedora, EL6
+
+
+def blkdiscard(device, offset=None, length=None, secure=False):
+ cmd = [_blkdiscard.cmd]
+
+ if offset:
+ cmd.extend(("-o", str(offset)))
+
+ if length:
+ cmd.extend(("-l", str(length)))
+
+ if secure:
+ cmd.append("-s")
+
+ cmd.append(device)
+ rc, out, err = utils.execCmd(cmd, deathSignal=signal.SIGKILL)
+
+ if rc != 0:
+ raise QImgError(rc, out, err)
diff --git a/tests/Makefile.am b/tests/Makefile.am
index b365921..5aa9fc3 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -79,6 +79,7 @@
testlibTests.py \
toolTests.py \
transportWrapperTests.py \
+ utillinuxTests.py \
utilsTests.py \
vdsClientTests.py \
vmApiTests.py \
diff --git a/tests/utillinuxTests.py b/tests/utillinuxTests.py
new file mode 100644
index 0000000..dbbb50e
--- /dev/null
+++ b/tests/utillinuxTests.py
@@ -0,0 +1,82 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+from testlib import VdsmTestCase as TestCaseBase
+import monkeypatch
+from vdsm import utillinux
+from vdsm import utils
+
+BLKDISCARD = utillinux._blkdiscard.cmd
+
+
+class FakeCmd(object):
+
+ def __init__(self, module, name, *calls):
+ self.patch = monkeypatch.Patch([(module, name, self)])
+ self.calls = list(calls)
+
+ def __call__(self, cmd, **kw):
+ call = self.calls.pop(0)
+ return call(cmd, **kw)
+
+ def __enter__(self):
+ self.patch.apply()
+
+ def __exit__(self, t=None, v=None, tb=None):
+ self.patch.revert()
+
+
+class BlkDiscardTests(TestCaseBase):
+
+ def test_no_options(self):
+ def discard(cmd, **kw):
+ expected = [BLKDISCARD, 'device']
+ self.assertEqual(cmd, expected)
+ return 0, '', ''
+
+ with FakeCmd(utils, 'execCmd', discard):
+ utillinux.blkdiscard('device')
+
+ def test_offset(self):
+ def discard(cmd, **kw):
+ expected = [BLKDISCARD, '-o', '1024', 'device']
+ self.assertEqual(cmd, expected)
+ return 0, '', ''
+
+ with FakeCmd(utils, 'execCmd', discard):
+ utillinux.blkdiscard('device', offset=1024)
+
+ def test_length(self):
+ def discard(cmd, **kw):
+ expected = [BLKDISCARD, '-l', '2048', 'device']
+ self.assertEqual(cmd, expected)
+ return 0, '', ''
+
+ with FakeCmd(utils, 'execCmd', discard):
+ utillinux.blkdiscard('device', length=2048)
+
+ def test_offset_and_length(self):
+ def discard(cmd, **kw):
+ expected = [BLKDISCARD, '-o', '1024', '-l', '2048', 'device']
+ self.assertEqual(cmd, expected)
+ return 0, '', ''
+
+ with FakeCmd(utils, 'execCmd', discard):
+ utillinux.blkdiscard('device', offset=1024, length=2048)
diff --git a/vdsm.spec.in b/vdsm.spec.in
index d15f5c2..befe0e0 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -1252,6 +1252,7 @@
%{python_sitelib}/%{vdsm_name}/qemuimg.py*
%{python_sitelib}/%{vdsm_name}/netconfpersistence.py*
%{python_sitelib}/%{vdsm_name}/sslutils.py*
+%{python_sitelib}/%{vdsm_name}/utillinux.py*
%{python_sitelib}/%{vdsm_name}/utils.py*
%{python_sitelib}/%{vdsm_name}/vdscli.py*
%{python_sitelib}/%{vdsm_name}/virtsparsify.py*
--
To view, visit http://gerrit.ovirt.org/35629
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I2ea7dd19fadc600b8fe78fb436ae430d35f52165
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
8 years, 5 months
Change in vdsm[master]: lvm: Decrease number of retries before failing
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: lvm: Decrease number of retries before failing
......................................................................
lvm: Decrease number of retries before failing
Previously we configured lvm to stop accessing a device during an lvm
opoeration after 3 errors. This can cause lvm to block 3 times when
trying to access inaccessible device. With current iscsi settings, each
block can be 120 seconds, total 360 seconds. We have seen lvm block for
couple of minutes in such cases.
The retries seems uneeded when working with multiple paths, as multipath
already retry all available paths after SCSI errors on one path. However
when working with single path, multipath should fail after one try.
According to lvm developer this may decrease the time lvm is blocked
when devices are not accesible.
(Not tested yet)
Change-Id: I5d11abaaff45ce86e88c6589264e162318ac1f1d
Relates-To: https://bugzilla.redhat.com/880738
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/lvm.py
1 file changed, 1 insertion(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/32356/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index 86edf55..f760f7b 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -106,7 +106,7 @@
preferred_names = ["^/dev/mapper/"]
ignore_suspended_devices=1
write_cache_state=0
-disable_after_error_count=3
+disable_after_error_count=1
obtain_device_list_from_udev=0
%s
}
--
To view, visit http://gerrit.ovirt.org/32356
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5d11abaaff45ce86e88c6589264e162318ac1f1d
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
8 years, 5 months
Change in vdsm[master]: clusterlock: Remove uneeded workaround
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: clusterlock: Remove uneeded workaround
......................................................................
clusterlock: Remove uneeded workaround
Sanlock version XXXX had a off-by-one bug when calling get_hosts with a
host id, returning info for the next host. This bug is fixed in version
XXX. Now we can use the hostId parameter, making the call more efficient
and simpligying clusterlock code.
Change-Id: Ide75e749fbc2916540c2b526b78fedc247b5c6f9
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm.spec.in
M vdsm/storage/clusterlock.py
2 files changed, 5 insertions(+), 16 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/62/31162/1
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 5ba6fc6..fc39eb9 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -167,7 +167,7 @@
Requires: iscsi-initiator-utils >= 6.2.0.873-21
%endif
-Requires: sanlock >= 2.8, sanlock-python
+Requires: sanlock >= XXX, sanlock-python
%if 0%{?rhel}
Requires: python-ethtool >= 0.6-3
diff --git a/vdsm/storage/clusterlock.py b/vdsm/storage/clusterlock.py
index 24a5d81..17cdd53 100644
--- a/vdsm/storage/clusterlock.py
+++ b/vdsm/storage/clusterlock.py
@@ -265,26 +265,15 @@
return False
def getHostStatus(self, hostId):
- # Note: get_hosts has off-by-one bug when asking for particular host
- # id, so get all hosts info and filter.
- # See https://bugzilla.redhat.com/1111210
try:
- hosts = sanlock.get_hosts(self._sdUUID)
+ hosts = sanlock.get_hosts(self._sdUUID, hostId)
except sanlock.SanlockException as e:
self.log.debug("Unable to get host %d status in lockspace %s: %s",
hostId, self._sdUUID, e)
return HOST_STATUS_UNAVAILABLE
-
- for info in hosts:
- if info['host_id'] == hostId:
- status = info['flags']
- return self.STATUS_NAME[status]
-
- # get_hosts with host_id=0 returns only hosts with timestamp != 0,
- # which means that no host is using this host id now. If there a
- # timestamp, sanlock will return HOST_UNKNOWN and then HOST_LIVE or
- # HOST_FAIL.
- return HOST_STATUS_FREE
+ else:
+ status = hosts[0]['flags']
+ return self.STATUS_NAME[status]
# The hostId parameter is maintained here only for compatibility with
# ClusterLock. We could consider to remove it in the future but keeping it
--
To view, visit http://gerrit.ovirt.org/31162
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ide75e749fbc2916540c2b526b78fedc247b5c6f9
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
8 years, 5 months