vdsm-patches November 2012

vdsm-patches@lists.fedorahosted.org

50 participants
416 discussions

Change in vdsm[master]: Include vds_name parameter for node registration
by fgranha＠linux.vnet.ibm.com 26 Nov '13

26 Nov '13

Fernando Granha Jeronimo has uploaded a new change for review. Change subject: Include vds_name parameter for node registration ...................................................................... Include vds_name parameter for node registration Motivation: in ovirt, the vds_name does not need to be resolvable. In order to allow an arbitrary name, a new kernel parameter was created (node_name). If this paramter is not passed, vdsm-reg will stick to the hostname of the node (the previous behavior). Change-Id: Ic1b0bcc473ef1a034736f80e1424e211ddf02946 Signed-off-by: Fernando Granha Jeronimo <fgranha(a)linux.vnet.ibm.com> --- M vdsm_reg/config.py.in M vdsm_reg/vdsm-config M vdsm_reg/vdsm-reg-cmdline M vdsm_reg/vdsm-reg-setup.in M vdsm_reg/vdsm-reg.conf.in 5 files changed, 19 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/36/8036/1 diff --git a/vdsm_reg/config.py.in b/vdsm_reg/config.py.in index 6519ec2..d00b427 100644 --- a/vdsm_reg/config.py.in +++ b/vdsm_reg/config.py.in @@ -35,5 +35,6 @@ config.set('vars', 'vdc_host_port', '80') config.set('vars', 'vdc_reg_port', '54321') config.set('vars', 'vdc_reg_uri', '/SolidICE/VdsAutoRegistration.aspx') +config.set('vars', 'vds_name', 'None') config.set('vars', 'vdsm_conf_file', '/etc/vdsm/vdsm.conf') config.set('vars', 'vdsm_dir', '@VDSMDIR@') diff --git a/vdsm_reg/vdsm-config b/vdsm_reg/vdsm-config index 1cb4c2c..6e0f7f3 100755 --- a/vdsm_reg/vdsm-config +++ b/vdsm_reg/vdsm-config @@ -65,6 +65,9 @@ rhevm_admin_password=*) rhevm_admin_password=${i#rhevm_admin_password=} ;; + node_name=*) + node_name=${i#node_name=} + ;; *) ;; esac @@ -75,6 +78,7 @@ vdc_managment_server=${managment_server%:*} vdc_managment_port=${managment_server#*:} strRHEVMAddress=$vdc_managment_server + vds_name=${node_name} tmp_vdsm_reg_conf=`mktemp` cat /etc/vdsm-reg/vdsm-reg.conf > "$tmp_vdsm_reg_conf" echo "checkpoint 3::management_server: $vdc_managment_server, management_port: $vdc_managment_port" >> $LOG 2>&1 @@ -108,6 +112,11 @@ echo "No management_server_fingerprint found." | tee -a $LOG fi + if [ ! -z "$vds_name" ]; then + sed --copy -i "s/$^vds_name=$$..*$$/\1${vds_name}/" \ + /etc/vdsm-reg/vdsm-reg.conf + fi + ## Set new configuration fWriteConfig=1 else diff --git a/vdsm_reg/vdsm-reg-cmdline b/vdsm_reg/vdsm-reg-cmdline index 92ed468..e7f0047 100644 --- a/vdsm_reg/vdsm-reg-cmdline +++ b/vdsm_reg/vdsm-reg-cmdline @@ -3,4 +3,5 @@ management_server management_server_fingerprint rhevm_admin_password +node_name ovirt_vdsm_disable diff --git a/vdsm_reg/vdsm-reg-setup.in b/vdsm_reg/vdsm-reg-setup.in index 492e38d..8652293 100644 --- a/vdsm_reg/vdsm-reg-setup.in +++ b/vdsm_reg/vdsm-reg-setup.in @@ -48,7 +48,12 @@ self.vdsmDir = config.get('vars', 'vdsm_dir') if self.vdcURL != "None": self.ovirtURL = deployUtil.getMGTIP(self.vdsmDir, self.vdcName) - self.ovirtName = socket.gethostname() + + # use the name from config file, otherwise get the fqdn + self.ovirtName = config.get('vars', 'vds_name') + if self.ovirtName is None or self.ovirtName == "None": + self.ovirtName = socket.gethostname() + self.ovirtUID = deployUtil.getHostID() else: self.ovirtURL = "None" diff --git a/vdsm_reg/vdsm-reg.conf.in b/vdsm_reg/vdsm-reg.conf.in index 2e9df0d..c38401c 100644 --- a/vdsm_reg/vdsm-reg.conf.in +++ b/vdsm_reg/vdsm-reg.conf.in @@ -13,6 +13,8 @@ vdc_host_port=443 #The registrtion uri vdc_reg_uri=/OvirtEngineWeb/register +# The name of this node (it does not need to be resolvable) +vds_name=None #Upgrade settings upgrade_iso_file=@VDSMUPGRADEDIR@/ovirt-node-image.iso upgrade_mount_point=/live -- To view, visit http://gerrit.ovirt.org/8036 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ic1b0bcc473ef1a034736f80e1424e211ddf02946 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Fernando Granha Jeronimo <fgranha(a)linux.vnet.ibm.com>

4 4

Change in vdsm[master]: Added utility to ensure that files written to the file syste...
by asegurap＠redhat.com 18 Nov '13

18 Nov '13

Antoni Segura Puimedon has uploaded a new change for review. Change subject: Added utility to ensure that files written to the file system happen atomically. ...................................................................... Added utility to ensure that files written to the file system happen atomically. The only way in python to make a file be written in an atomic way is to take advantage of the fact that in POSIX filesystems the rename operation is atomic. Thus, we write the context to a temporary file on the same filesystem, sync on the file descriptor and finally do the atomic file rename to the original file we wanted to write. For more info: http://stackoverflow.com/questions/2333872/atomic-writing-to-file-with-pyth… http://stackoverflow.com/questions/7433057/is-rename-without-fsync-safe Change-Id: Ibecd61d6746231a5a8cb17bad9a3302b01454f27 Signed-off-by: Antoni S. Puimedon <asegurap(a)redhat.com> --- M vdsm/configNetwork.py M vdsm/utils.py 2 files changed, 17 insertions(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/7656/1 diff --git a/vdsm/configNetwork.py b/vdsm/configNetwork.py index 58117f8..b943574 100755 --- a/vdsm/configNetwork.py +++ b/vdsm/configNetwork.py @@ -369,7 +369,8 @@ '''Backs up the previous contents of the file referenced by fileName writes the new configuration and sets the specified access mode.''' self._backup(fileName) - open(fileName, 'w').write(configuration) + with utils.atomicWrite(fileName) as f: + f.write(configuration) os.chmod(fileName, 0664) try: selinux.restorecon(fileName) diff --git a/vdsm/utils.py b/vdsm/utils.py index 5e2d4e5..bb8a842 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -36,6 +36,7 @@ import fcntl import functools import stat +from contextlib import contextmanager import ethtool @@ -829,3 +830,17 @@ def __unicode__(self): return unicode(self.cmd) + +@contextmanager +def atomicWrite(file): + '''Context manager that makes the write happen to a temporary file and if + and only it is successful, overwrites the original file. It creates the + temporary file in the same directory of the destination file, as rename is + only atomic within the same filesystem.''' + tempFile = file+'vdsm_temp' + f = open(tempFile, 'w') + yield f + f.flush() + os.fsync(f.fileno()) + f.close() + os.rename(tempFile, file) -- To view, visit http://gerrit.ovirt.org/7656 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ibecd61d6746231a5a8cb17bad9a3302b01454f27 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Antoni Segura Puimedon <asegurap(a)redhat.com>

4 10

Change in vdsm[master]: Add deathSignal options to better peopen
by smizrahi＠redhat.com 17 Nov '13

17 Nov '13

Saggi Mizrahi has uploaded a new change for review. Change subject: Add deathSignal options to better peopen ...................................................................... Add deathSignal options to better peopen This allows VDSM to specify as signal that will be sent by the kernel to the child when VDSM dies. - Also fixes problem in the makefile that would cause make not to recompile the SO when the C file was changed Change-Id: I9f987129cea112e2a75d6f02477369417cc50dc7 Signed-off-by: Saggi Mizrahi <smizrahi(a)redhat.com> --- M vdsm/betterPopen/Makefile.am M vdsm/betterPopen/__init__.py M vdsm/betterPopen/createprocess.c 3 files changed, 35 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/41/6241/1 -- To view, visit http://gerrit.ovirt.org/6241 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9f987129cea112e2a75d6f02477369417cc50dc7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Saggi Mizrahi <smizrahi(a)redhat.com>

8 18

Change in vdsm[master]: Add build-dep packages so yum-builddep is effective
by Ryan Harper 13 Nov '13

13 Nov '13

Ryan Harper has uploaded a new change for review. Change subject: Add build-dep packages so yum-builddep is effective ...................................................................... Add build-dep packages so yum-builddep is effective yum-builddep doesn't get all of the required packages needed to build vdsm and vdsm rpms. Update the specfile Build-Requires to include the needed packages. Change-Id: Ia5b96b5b30a9efea06261f40d1f6df61754b8253 Signed-off-by: Ryan Harper <ryanh(a)us.ibm.com> --- M vdsm.spec.in 1 file changed, 3 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/55/7955/1 diff --git a/vdsm.spec.in b/vdsm.spec.in index 56a42b4..300dbe5 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -29,6 +29,9 @@ BuildRequires: pyflakes BuildRequires: python-nose BuildRequires: python-pep8 +BuildRequires: gcc +BuildRequires: automake +BuildRequires: rpm-build # BuildRequires needed by the tests during the build BuildRequires: python-ethtool -- To view, visit http://gerrit.ovirt.org/7955 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5b96b5b30a9efea06261f40d1f6df61754b8253 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Ryan Harper <ryanh(a)us.ibm.com>

5 7

Change in vdsm[master]: Add ut to supervdsmServer zombie reaper
by lvroyce＠linux.vnet.ibm.com 10 Nov '13

10 Nov '13

Royce Lv has uploaded a new change for review. Change subject: Add ut to supervdsmServer zombie reaper ...................................................................... Add ut to supervdsmServer zombie reaper Change-Id: I7fb8f3dcd575266febe11967e3f8df7d3588acb8 Signed-off-by: Royce Lv<lvroyce(a)linux.vnet.ibm.com> --- M tests/superVdsmTests.py M vdsm/supervdsmServer.py 2 files changed, 12 insertions(+), 7 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/54/9354/1 diff --git a/tests/superVdsmTests.py b/tests/superVdsmTests.py index 20b1591..cb52ff1 100644 --- a/tests/superVdsmTests.py +++ b/tests/superVdsmTests.py @@ -4,6 +4,7 @@ import tempfile from vdsm import utils import os +from time import sleep class TestSuperVdsm(TestCaseBase): @@ -16,8 +17,11 @@ self.pidfd, pidfile = tempfile.mkstemp() self.timefd, timestamp = tempfile.mkstemp() self.addfd, address = tempfile.mkstemp() + self.pidfile = pidfile self._proxy.setIPCPaths(pidfile, timestamp, address) + self._proxy.ping() + self.assertTrue(self._proxy.isRunning()) def tearDown(self): for fd in (self.pidfd, self.timefd, self.addfd): @@ -25,19 +29,19 @@ self._proxy.kill() # cleanning old temp files def testIsSuperUp(self): - self._proxy.ping() # this call initiate svdsm - - self.assertTrue(self._proxy.isRunning()) + pass def testKillSuper(self): - self._proxy.ping() self._proxy.kill() self.assertFalse(self._proxy.isRunning()) self._proxy.ping() # Launching vdsm after kill self.assertTrue(self._proxy.isRunning()) def testNoPidFile(self): - self._proxy.ping() # svdsm is up - self.assertTrue(self._proxy.isRunning()) utils.rmFile(self._proxy.timestamp) self.assertFalse(self._proxy.isRunning()) + + def testZombieReaper(self): + reapedPid = self._proxy.ping() + sleep(10) # supervdsmServer checks SIGCHLD every 5 secs + self.assertRaises(OSError, os.waitpid, reapedPid, os.WNOHANG) diff --git a/vdsm/supervdsmServer.py b/vdsm/supervdsmServer.py index 1287fef..56dd5fd 100755 --- a/vdsm/supervdsmServer.py +++ b/vdsm/supervdsmServer.py @@ -92,7 +92,8 @@ @logDecorator def ping(self, *args, **kwargs): # This method exists for testing purposes - return True + res = self._runAs(DISKIMAGE_USER, (DISKIMAGE_GROUP,), os.getpid) + return res @logDecorator def getDevicePartedInfo(self, *args, **kwargs): -- To view, visit http://gerrit.ovirt.org/9354 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7fb8f3dcd575266febe11967e3f8df7d3588acb8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Royce Lv <lvroyce(a)linux.vnet.ibm.com>

5 15

Change in vdsm[master]: Get oop from object instance
by shuming＠linux.vnet.ibm.com 06 Nov '13

06 Nov '13

Shu Ming has uploaded a new change for review. Change subject: Get oop from object instance ...................................................................... Get oop from object instance In FileStorageDomain non-class method, self.oop should be used instead of oop Change-Id: Ia6b86f2c5e382774003772837e32bd62f8ad29ce Signed-off-by: Shu Ming <shuming(a)linux.vnet.ibm.com> --- M vdsm/storage/fileSD.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/20/8920/1 diff --git a/vdsm/storage/fileSD.py b/vdsm/storage/fileSD.py index f92ed98..7d6e1ab 100644 --- a/vdsm/storage/fileSD.py +++ b/vdsm/storage/fileSD.py @@ -177,7 +177,7 @@ def getReadDelay(self): t = time.time() - oop.getProcessPool(self.sdUUID).directReadLines(self.metafile) + self.oop.getProcessPool(self.sdUUID).directReadLines(self.metafile) return time.time() - t def getFileList(self, pattern, caseSensitive): -- To view, visit http://gerrit.ovirt.org/8920 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia6b86f2c5e382774003772837e32bd62f8ad29ce Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shu Ming <shuming(a)linux.vnet.ibm.com>

6 8

Change in vdsm[master]: AdvancedStatsThread: Throttle duplicated exception log
by zhshzhou＠linux.vnet.ibm.com 30 Oct '13

30 Oct '13

Zhou Zheng Sheng has uploaded a new change for review. Change subject: AdvancedStatsThread: Throttle duplicated exception log ...................................................................... AdvancedStatsThread: Throttle duplicated exception log When the same exception is raised in the stats function many times, the log message is the same as well, so extra message does not help us on the problem. If the problem is not solved, the same exception will be raised again and again, then the log will contain a lot of garbages. This patch proposes a way to throttle duplicated exception log in stats functions. For the same exception in the same stats function, if it is raised too many times, just suppress the log. Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=853725 Change-Id: I2ee04d8d82e2a14b0a003627981c28e5e64a46ab Signed-off-by: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com> --- M vdsm/utils.py 1 file changed, 45 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/84/8884/1 diff --git a/vdsm/utils.py b/vdsm/utils.py index 002094a..7fab19a 100644 --- a/vdsm/utils.py +++ b/vdsm/utils.py @@ -291,6 +291,45 @@ return bgn_sample, end_sample, (end_time - bgn_time) + +class _ExceptionLogThrottle(object): + ''' + Record previous exception for each statsFunction, throttle the log + if the same exception is raised too many times. + ''' + + def __init__(self, log, dupThreshold=3): + self._log = log + self._dupThreshold = dupThreshold + self._exceptions = {} + + def _exceptionEqual(self, exc1, exc2): + return exc1.__class__ == exc2.__class__ and exc1.args == exc2.args + + def needSuppress(self, statsFunction, exceptObj): + if (statsFunction not in self._exceptions or + not self._exceptionEqual( + self._exceptions[statsFunction]['exception'], exceptObj)): + + self._exceptions[statsFunction] = {'exception': exceptObj, + 'dupCount': 0} + return False + + dupCount = self._exceptions[statsFunction]['dupCount'] + + if dupCount <= self._dupThreshold: + dupCount += 1 + if dupCount == self._dupThreshold: + self._log.debug( + "Log will be throttled for the same exception: %s" % + exceptObj.message) + self._exceptions[statsFunction] = {'exception': exceptObj, + 'dupCount': dupCount} + return False + + return True + + class AdvancedStatsThread(threading.Thread): """ A thread that runs the registered AdvancedStatsFunction objects @@ -311,6 +350,8 @@ self._statsTime = None self._statsFunctions = [] + + self._logThrottle = _ExceptionLogThrottle(log) def addStatsFunction(self, *args): """ @@ -395,8 +436,10 @@ statsFunction() except Exception, e: if not self.handleStatsException(e): - self._log.error("Stats function failed: %s", - statsFunction, exc_info=True) + if not self._logThrottle.needSuppress( + statsFunction, e): + self._log.error("Stats function failed: %s", + statsFunction, exc_info=True) self._stopEvent.wait(waitInterval) intervalAccum = (intervalAccum + waitInterval) % maxInterval -- To view, visit http://gerrit.ovirt.org/8884 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I2ee04d8d82e2a14b0a003627981c28e5e64a46ab Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Zhou Zheng Sheng <zhshzhou(a)linux.vnet.ibm.com>

6 34

Change in vdsm[master]: Fix vm block stats lost after recovery
by wudxw＠linux.vnet.ibm.com 29 Oct '13

29 Oct '13

Mark Wu has uploaded a new change for review. Change subject: Fix vm block stats lost after recovery ...................................................................... Fix vm block stats lost after recovery We don't need call preparePaths() in the code path of vm recovery, because the 'path' has been prepared on vm creatation and saved in recovery file. But the flag _volumesPrepared still needs to be set to make the block stats can be collected. Change-Id: I59a90cf909a07967f642d348e47b4928f7516c61 Signed-off-by: Mark Wu <wudxw(a)linux.vnet.ibm.com> --- M vdsm/libvirtvm.py 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/56/6056/1 -- To view, visit http://gerrit.ovirt.org/6056 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I59a90cf909a07967f642d348e47b4928f7516c61 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Mark Wu <wudxw(a)linux.vnet.ibm.com>

3 6

Change in vdsm[master]: Source image exception when querying source image
by shuming＠linux.vnet.ibm.com 29 Oct '13

29 Oct '13

Shu Ming has uploaded a new change for review. Change subject: Source image exception when querying source image ...................................................................... Source image exception when querying source image se.StorageException exception should also be converted to se.SourceImageActionError exception Change-Id: I6abb905f1042477aa6757d6e10d6989923e955b4 Signed-off-by: Shu Ming <shuming(a)linux.vnet.ibm.com> --- M vdsm/storage/image.py 1 file changed, 0 insertions(+), 3 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/79/8479/1 diff --git a/vdsm/storage/image.py b/vdsm/storage/image.py index 9a22863..f9cb3a0 100644 --- a/vdsm/storage/image.py +++ b/vdsm/storage/image.py @@ -457,9 +457,6 @@ try: # Find all volumes of source image srcChain = self.getChain(srcSdUUID, imgUUID) - except se.StorageException: - self.log.error("Unexpected error", exc_info=True) - raise except Exception, e: self.log.error("Unexpected error", exc_info=True) raise se.SourceImageActionError(imgUUID, srcSdUUID, str(e)) -- To view, visit http://gerrit.ovirt.org/8479 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6abb905f1042477aa6757d6e10d6989923e955b4 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shu Ming <shuming(a)linux.vnet.ibm.com>

3 3

Change in vdsm[master]: '0' is a valid master version
by shuming＠linux.vnet.ibm.com 29 Oct '13

29 Oct '13

Shu Ming has uploaded a new change for review. Change subject: '0' is a valid master version ...................................................................... '0' is a valid master version When connecting the storage pool first time with master domain version '0', exception happens not finding the temporary pool file for pool parameters. Change-Id: I736d917bfa7d3a5ac59cddcaf1a18846f53d85b0 Signed-off-by: Shu Ming <shuming(a)linux.vnet.ibm.com> --- M vdsm/storage/hsm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/48/7948/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 1e9311b..f7f8b49 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -849,7 +849,7 @@ return pool = sp.StoragePool(spUUID, self.taskMng) - if not hostID or not scsiKey or not msdUUID or not masterVersion: + if not hostID or not scsiKey or not msdUUID or (masterVersion == None): hostID, scsiKey, msdUUID, masterVersion = pool.getPoolParams() res = pool.connect(hostID, scsiKey, msdUUID, masterVersion) if res: -- To view, visit http://gerrit.ovirt.org/7948 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I736d917bfa7d3a5ac59cddcaf1a18846f53d85b0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Shu Ming <shuming(a)linux.vnet.ibm.com>

4 5

← Newer
1
...
7
8
9
10
11
12
13
...
42
Older →

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches November 2012