Change in vdsm[master]: storage: exec mount in different cgroup
by Sandro Bonazzola
Sandro Bonazzola has uploaded a new change for review.
Change subject: storage: exec mount in different cgroup
......................................................................
storage: exec mount in different cgroup
Always execute mount command in a differenti
cgroup avoiding mounts using user space drivers
like glusterfs to die on VDSM restart.
Change-Id: I8dd6d6076af46ec3e2bf2400afb72ad8ec3fc3dd
Bug-Url: https://bugzilla.redhat.com/1201355
Signed-off-by: Sandro Bonazzola <sbonazzo(a)redhat.com>
---
M configure.ac
M lib/vdsm/constants.py.in
M vdsm.spec.in
M vdsm/storage/mount.py
4 files changed, 8 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/79/40179/1
diff --git a/configure.ac b/configure.ac
index 0a30c3e..ea5eb4f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -280,6 +280,7 @@
AC_PATH_PROG([BLKID_PATH], [blkid], [/sbin/blkid])
AC_PATH_PROG([BRCTL_PATH], [brctl], [/usr/sbin/brctl])
AC_PATH_PROG([CAT_PATH], [cat], [/bin/cat])
+AC_PATH_PROG([CGEXEC_PATH], [cgexec], [/usr/bin/cgexec])
AC_PATH_PROG([CHKCONFIG_PATH], [chkconfig], [/sbin/chkconfig])
AC_PATH_PROG([CHMOD_PATH], [chmod], [/bin/chmod])
AC_PATH_PROG([CHOWN_PATH], [chown], [/bin/chown])
diff --git a/lib/vdsm/constants.py.in b/lib/vdsm/constants.py.in
index 60c272c..2c8c02b 100644
--- a/lib/vdsm/constants.py.in
+++ b/lib/vdsm/constants.py.in
@@ -101,6 +101,7 @@
EXT_BRCTL = '@BRCTL_PATH@'
EXT_CAT = '@CAT_PATH@'
+EXT_CGEXEC = '@CGEXEC_PATH@'
EXT_CHOWN = '@CHOWN_PATH@'
EXT_CP = '@CP_PATH@'
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 25506cf..47e3a45 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -91,6 +91,7 @@
%endif
BuildRequires: cyrus-sasl-lib
+BuildRequires: libcgroup-tools
BuildRequires: gcc
BuildRequires: python
BuildRequires: python-devel
@@ -163,6 +164,7 @@
Requires: curl
Requires: %{name}-xmlrpc = %{version}-%{release}
Requires: %{name}-jsonrpc = %{version}-%{release}
+Requires: libcgroup-tools
%if 0%{?rhev_build}
Requires: mom >= 0.4.1-4
diff --git a/vdsm/storage/mount.py b/vdsm/storage/mount.py
index fefa5df..464445a 100644
--- a/vdsm/storage/mount.py
+++ b/vdsm/storage/mount.py
@@ -213,7 +213,10 @@
return hsh
def mount(self, mntOpts=None, vfstype=None, timeout=None):
- cmd = [constants.EXT_MOUNT]
+ # Always execute mount command in a different cgroup avoiding
+ # mounts using user space drivers like glusterfs to die on
+ # VDSM restart.
+ cmd = [constants.EXT_CGEXEC, '-g', 'cpu:/', constants.EXT_MOUNT]
if vfstype is not None:
cmd.extend(("-t", vfstype))
--
To view, visit https://gerrit.ovirt.org/40179
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8dd6d6076af46ec3e2bf2400afb72ad8ec3fc3dd
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Sandro Bonazzola <sbonazzo(a)redhat.com>
9 years
Change in vdsm[master]: ssl: m2crypto removal
by Piotr Kliczewski
Piotr Kliczewski has posted comments on this change.
Change subject: ssl: m2crypto removal
......................................................................
Patch Set 3:
(1 comment)
https://gerrit.ovirt.org/#/c/39990/3/tests/sslTests.py
File tests/sslTests.py:
Line 369: "-key", self.keyfile,
Line 370: ])
Line 371: self.assertEquals(rc, 0)
Line 372:
Line 373: @brokentest
> The python-native implementation of ssl is unaware of ssl sessions, and it
Done
Line 374: def testSessionIsCached(self):
Line 375: """
Line 376: Verify that SSL the session identifier is preserved when
Line 377: connecting two times without stopping the server.
--
To view, visit https://gerrit.ovirt.org/39990
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I3f2688b6c00eadd3f15be0ced926a397b55c1f33
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com>
Gerrit-Reviewer: Alon Bar-Lev <alonbl(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Dima Kuznetsov <dkuznets(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Ido Barkan <ibarkan(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Juan Hernandez <juan.hernandez(a)redhat.com>
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Sandro Bonazzola <sbonazzo(a)redhat.com>
Gerrit-Reviewer: Simone Tiraboschi <stirabos(a)redhat.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: Yeela Kaplan <ykaplan(a)redhat.com>
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-HasComments: Yes
9 years
Change in vdsm[master]: Teach utils.rmFile to handle ovirtNode
by ibarkan＠redhat.com
Ido Barkan has uploaded a new change for review.
Change subject: Teach utils.rmFile to handle ovirtNode
......................................................................
Teach utils.rmFile to handle ovirtNode
We need to handle files in other places in vdsm so moving it down
the file hierarchy.
Change-Id: I080c4f34f8962ac5589f0a2c54f26e808d65ec33
Signed-off-by: Ido Barkan <ibarkan(a)redhat.com>
---
M lib/vdsm/utils.py
M vdsm/network/configurators/ifcfg.py
2 files changed, 24 insertions(+), 5 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/73/40573/1
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 4856cf9..520542b 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -135,6 +135,26 @@
If the file doesn't exist it's assumed that it was already removed.
"""
+ if isOvirtNode():
+ try:
+ _rmFileInNode(fileToRemove)
+ except OSError as e:
+ if e.errno == errno.ENOENT:
+ logging.warning("File: %s already removed", fileToRemove)
+ else:
+ logging.error("Removing file: %s failed", fileToRemove,
+ exc_info=True)
+ raise
+ else:
+ _rmFile(fileToRemove)
+
+
+def _rmFileInNode(fileToRemove):
+ """unpersists and shreds the file"""
+ node_fs.Config().delete(fileToRemove)
+
+
+def _rmFile(fileToRemove):
try:
os.unlink(fileToRemove)
except OSError as e:
@@ -1088,6 +1108,9 @@
bool(glob.glob('/etc/ovirt-node-*-release')))
+if isOvirtNode():
+ from ovirt.node.utils import fs as node_fs
+
# Copied from
# http://docs.python.org/2.6/library/itertools.html?highlight=grouper#recipes
def grouper(iterable, n, fillvalue=None):
diff --git a/vdsm/network/configurators/ifcfg.py b/vdsm/network/configurators/ifcfg.py
index 6d35c0a..cdb22c8 100644
--- a/vdsm/network/configurators/ifcfg.py
+++ b/vdsm/network/configurators/ifcfg.py
@@ -280,11 +280,7 @@
@staticmethod
def _removeFile(filename):
- """Remove file (directly or using oVirt node's library)"""
- if utils.isOvirtNode():
- node_fs.Config().delete(filename) # unpersists and shreds the file
- else:
- utils.rmFile(filename)
+ utils.rmFile(filename)
logging.debug("Removed file %s", filename)
def createLibvirtNetwork(self, network, bridged=True, iface=None,
--
To view, visit https://gerrit.ovirt.org/40573
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I080c4f34f8962ac5589f0a2c54f26e808d65ec33
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Ido Barkan <ibarkan(a)redhat.com>
9 years
Change in vdsm[ovirt-3.5]: net: Force blocking DHCP when restoring networks
by ibarkan＠redhat.com
Ido Barkan has uploaded a new change for review.
Change subject: net: Force blocking DHCP when restoring networks
......................................................................
net: Force blocking DHCP when restoring networks
This is done right before restoring the network configuration, and
before calling setupNetworks. It forces the configurator to wait for
an IP address to be configured on the devices before restoration is
completed. This prevents VDSM from possibly report missing IP
addresses on interfaces that had been restored right before it was
started.
Change-Id: Ibc8b84a82794ac97eba6d34ec9d54430e387b659
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1218637
Signed-off-by: Ido Barkan <ibarkan(a)redhat.com>
Change-Id: Ib9eb07f7dc54d9167460ea9936d68b77b96564d6
Signed-off-by: Ido Barkan <ibarkan(a)redhat.com>
---
M vdsm/vdsm-restore-net-config
1 file changed, 16 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/67/40567/1
diff --git a/vdsm/vdsm-restore-net-config b/vdsm/vdsm-restore-net-config
index d7cb8c7..515a03a 100755
--- a/vdsm/vdsm-restore-net-config
+++ b/vdsm/vdsm-restore-net-config
@@ -73,11 +73,27 @@
persistentConfig = PersistentConfig()
nets, bonds = _filter_nets_bonds(persistentConfig.networks,
persistentConfig.bonds)
+ _convert_to_blocking_dhcp(nets)
logging.debug('Calling setupNetworks with networks (%s) and bond (%s).',
nets, bonds)
setupNetworks(nets, bonds, connectivityCheck=False, _inRollback=True)
+def _convert_to_blocking_dhcp(networks):
+ """
+ This function changes DHCP configuration, if present, to be blocking.
+
+ This is done right before restoring the network configuration, and forces
+ the configurator to wait for an IP address to be configured on the devices
+ before restoration is completed. This prevents VDSM to possibly report
+ missing IP address on interfaces that had been restored right before it was
+ started.
+ """
+ for net, net_attr in networks.iteritems():
+ if net_attr.get('bootproto') == 'dhcp':
+ net_attr['blockingdhcp'] = True
+
+
def _filter_nets_bonds(nets, bonds):
"""Returns only nets and bonds that can be configured with the devices
present in the system"""
--
To view, visit https://gerrit.ovirt.org/40567
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib9eb07f7dc54d9167460ea9936d68b77b96564d6
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-3.5
Gerrit-Owner: Ido Barkan <ibarkan(a)redhat.com>
9 years
Change in vdsm[master]: utils: add CommandStream class
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: utils: add CommandStream class
......................................................................
utils: add CommandStream class
Change-Id: Ie015368bb9c5992e5c73a149277c59fc4ffbd570
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M lib/vdsm/utils.py
M tests/utilsTests.py
2 files changed, 188 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/09/33909/1
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 244f0e2..4b0ccb7 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -313,6 +313,90 @@
timeout = max(0, endtime - time.time())
+class CommandStream(object):
+ def __init__(self, command, stdoutcb, stderrcb, cwd=None,
+ deathSignal=0):
+ self.command = command
+ self.returncode = None
+
+ self.child = CPopen(self.command, cwd=cwd, close_fds=True,
+ deathSignal=deathSignal)
+
+ self.epoll = select.epoll()
+
+ self.iocb = {
+ self.child.stdout.fileno(): stdoutcb,
+ self.child.stderr.fileno(): stderrcb,
+ }
+
+ for fd in self.iocb.keys():
+ self.epoll.register(fd, select.EPOLLIN)
+
+ def terminate(self):
+ self.child.terminate()
+
+ def kill(self):
+ self.child.kill()
+
+ def write(self, data):
+ self.child.stdin.write(data)
+
+ def flush(self):
+ self.child.stdin.flush()
+
+ def close(self):
+ self.child.stdin.close()
+
+ def _epoll_input(self, fileno):
+ self.iocb[fileno](os.read(fileno, io.DEFAULT_BUFFER_SIZE))
+
+ def _epoll_event(self, fileno):
+ self.epoll.unregister(fileno)
+ del self.iocb[fileno]
+
+ def _epoll_timeout(self, timeout):
+ fdevents = NoIntrPoll(self.epoll.poll, timeout)
+
+ for fileno, event in fdevents:
+ if event & select.EPOLLIN:
+ self._epoll_input(fileno)
+ elif event & (select.EPOLLHUP | select.EPOLLERR):
+ self._epoll_event(fileno)
+ # Trying to collect the child status in case the
+ # file descriptor was closed because the process
+ # terminated.
+ self.returncode = self.child.poll()
+
+ def wait(self, timeout=None):
+ if timeout is None:
+ epoll_remaining = -1
+ else:
+ endtime = os.times()[4] + timeout
+
+ while self.returncode is None:
+ if timeout is not None:
+ epoll_remaining = endtime - os.times()[4]
+ if epoll_remaining <= 0:
+ break
+
+ if len(self.iocb):
+ self._epoll_timeout(epoll_remaining)
+ else:
+ # This is a busy-loop taken from issue5673, and
+ # python 3.4 still uses this implementation.
+ # A smarter solution would be using signalfd or
+ # sigtimedwait but they don't seem to mix well
+ # with multithreading (especially under heavy
+ # load: tens of children from tens of threads).
+ # Anyway we reach this only when both stdout and
+ # stderr are closed, which means that in most of
+ # the cases the child is about to die.
+ time.sleep(0.0005)
+ self.returncode = self.child.poll()
+
+ return self.returncode
+
+
class AsyncProc(object):
"""
AsyncProc is a funky class. It wraps a standard subprocess.Popen
diff --git a/tests/utilsTests.py b/tests/utilsTests.py
index 6e66c02..c279f04 100644
--- a/tests/utilsTests.py
+++ b/tests/utilsTests.py
@@ -22,8 +22,12 @@
import contextlib
import errno
import logging
+import operator
+import signal
import sys
import threading
+
+from contextlib import contextmanager
from testlib import VdsmTestCase as TestCaseBase
from testlib import permutations, expandPermutations
@@ -634,3 +638,103 @@
def test_empty(self):
self.assertEquals(utils._list2cmdline([]), "")
+
+
+class CommandStreamTests(TestCaseBase):
+
+ @contextmanager
+ def assertElapsed(self, limit):
+ start = os.times()[4]
+
+ yield
+
+ elapsed = os.times()[4] - start
+
+ if elapsed < limit:
+ raise AssertionError("Operation time: %s" % elapsed)
+
+ def assertNoOutput(self, data):
+ raise AssertionError("Unexpected data: " + repr(data))
+
+ def test_output(self):
+ text = "Hello World"
+ received = bytearray()
+
+ def recv_stdout(buffer):
+ # cannot use received += buffer with a variable
+ # defined in the parent function.
+ operator.iadd(received, buffer)
+
+ p = utils.CommandStream(["echo", "-n", text],
+ recv_stdout,
+ self.assertNoOutput)
+
+ retcode = p.wait()
+
+ self.assertEqual(retcode, 0)
+ self.assertEqual(text, str(received))
+
+ def test_write(self):
+ text = "Hello World"
+ received = bytearray()
+
+ def recv_stdout(buffer):
+ # cannot use received += buffer with a variable
+ # defined in the parent function.
+ operator.iadd(received, buffer)
+
+ p = utils.CommandStream(["cat"], recv_stdout,
+ self.assertNoOutput)
+
+ p.write(text)
+ p.flush()
+ p.close()
+
+ retcode = p.wait()
+
+ self.assertEqual(retcode, 0)
+ self.assertEqual(text, str(received))
+
+ def test_timeout(self):
+ p = utils.CommandStream(["sleep", "3"],
+ self.assertNoOutput,
+ self.assertNoOutput)
+
+ with self.assertElapsed(2):
+ retcode = p.wait(2)
+
+ self.assertEqual(retcode, None)
+
+ retcode = p.wait()
+ self.assertEqual(retcode, 0)
+
+ def test_terminate(self):
+ p = utils.CommandStream(["sleep", "2"],
+ self.assertNoOutput,
+ self.assertNoOutput)
+
+ p.terminate()
+
+ retcode = p.wait()
+ self.assertEqual(retcode, -signal.SIGTERM)
+
+ def test_kill(self):
+ p = utils.CommandStream(["sleep", "2"],
+ self.assertNoOutput,
+ self.assertNoOutput)
+
+ p.kill()
+
+ retcode = p.wait()
+ self.assertEqual(retcode, -signal.SIGKILL)
+
+ def test_early_close(self):
+ p = utils.CommandStream(["bash", "-c",
+ "exec 1>&-; exec 2>&-; exec sleep 2"],
+ self.assertNoOutput,
+ self.assertNoOutput)
+
+ with self.assertElapsed(2):
+ retcode = p.wait()
+
+ self.assertEqual(retcode, 0)
--
To view, visit http://gerrit.ovirt.org/33909
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie015368bb9c5992e5c73a149277c59fc4ffbd570
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
9 years
Change in vdsm[master]: ssl: m2crypto removal
by Dan Kenigsberg
Dan Kenigsberg has posted comments on this change.
Change subject: ssl: m2crypto removal
......................................................................
Patch Set 3:
(1 comment)
https://gerrit.ovirt.org/#/c/39990/3/tests/sslTests.py
File tests/sslTests.py:
Line 369: "-key", self.keyfile,
Line 370: ])
Line 371: self.assertEquals(rc, 0)
Line 372:
Line 373: @brokentest
> This is the test that checks that sessions are cached, in order to avoid pe
The python-native implementation of ssl is unaware of ssl sessions, and it does not cache them.
However, since you introduced ssl session caching, the need for caching is much weaker, as we use long-living TCP connection, thanks to html1.1.
I admit the verification of this patch must make sure that there is no typical degradation of performance due to this patch. Piotr - are you aware if CPU consuption under load was tested?
Also please explain why the test is broken:
@brokentest('support for SSL sessions discontinued and may be reintroduced when we move to python 3')
https://docs.python.org/3/library/ssl.html#ssl-contexts
Line 374: def testSessionIsCached(self):
Line 375: """
Line 376: Verify that SSL the session identifier is preserved when
Line 377: connecting two times without stopping the server.
--
To view, visit https://gerrit.ovirt.org/39990
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I3f2688b6c00eadd3f15be0ced926a397b55c1f33
Gerrit-PatchSet: 3
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Alon Bar-Lev <alonbl(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Gerrit-Reviewer: Dima Kuznetsov <dkuznets(a)redhat.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Jenkins CI
Gerrit-Reviewer: Juan Hernandez <juan.hernandez(a)redhat.com>
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Sandro Bonazzola <sbonazzo(a)redhat.com>
Gerrit-Reviewer: Simone Tiraboschi <stirabos(a)redhat.com>
Gerrit-Reviewer: Yaniv Bronhaim <ybronhei(a)redhat.com>
Gerrit-Reviewer: Yeela Kaplan <ykaplan(a)redhat.com>
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-HasComments: Yes
9 years
Change in vdsm[master]: WIP: statscache: support multiple removals
by fromani＠redhat.com
Francesco Romani has uploaded a new change for review.
Change subject: WIP: statscache: support multiple removals
......................................................................
WIP: statscache: support multiple removals
There is a still obscure bug that causes multiple removals
from cache, and everyone but the first will fail (as expected).
This patch handles this case, until either we pinpoint the bug,
or the actual need for supporting multiple removals.
Change-Id: I1748cba3646341d2be799aaf30b81432b009359c
Signed-off-by: Francesco Romani <fromani(a)redhat.com>
---
M vdsm/virt/sampling.py
1 file changed, 4 insertions(+), 1 deletion(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/39/40439/1
diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py
index bb2f462..455f49e 100644
--- a/vdsm/virt/sampling.py
+++ b/vdsm/virt/sampling.py
@@ -425,7 +425,10 @@
Remove any data from the cache related to the given VM.
"""
with self._lock:
- del self._vm_last_timestamp[vmid]
+ ts = self._vm_last_timestamp.pop(vmid, None)
+
+ if ts is None:
+ self._log.warn('timestamp for vm %s already removed', vmid)
def get(self, vmid):
"""
--
To view, visit https://gerrit.ovirt.org/40439
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I1748cba3646341d2be799aaf30b81432b009359c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>
9 years
Change in vdsm[master]: hooks: introduce before/after_get_stats hook points
by Dan Kenigsberg
Dan Kenigsberg has uploaded a new change for review.
Change subject: hooks: introduce before/after_get_stats hook points
......................................................................
hooks: introduce before/after_get_stats hook points
This patches adds a hook point that allows to modify the host statistics
which Vdsm reports on its getVdsStats API. This hook can become useful
if a hook writer would like to provide fake usage statistics to a fake
network that is reported by an after_get_caps hook script.
Change-Id: Id1ff93837abb73f88ececc7571115645b95486cf
Signed-off-by: Dan Kenigsberg <danken(a)redhat.com>
---
M debian/vdsm.dirs
M vdsm.spec.in
M vdsm/API.py
M vdsm/hooks.py
M vdsm/vdsmd.8.in
M vdsm_hooks/Makefile.am
6 files changed, 29 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/03/40403/1
diff --git a/debian/vdsm.dirs b/debian/vdsm.dirs
index 87703cd..82ba929 100644
--- a/debian/vdsm.dirs
+++ b/debian/vdsm.dirs
@@ -17,6 +17,7 @@
usr/libexec/vdsm/hooks/after_disk_hotunplug
usr/libexec/vdsm/hooks/after_get_all_vm_stats
usr/libexec/vdsm/hooks/after_get_caps
+usr/libexec/vdsm/hooks/after_get_stats
usr/libexec/vdsm/hooks/after_get_vm_stats
usr/libexec/vdsm/hooks/after_hostdev_list_by_caps
usr/libexec/vdsm/hooks/after_network_setup
@@ -45,6 +46,7 @@
usr/libexec/vdsm/hooks/before_disk_hotunplug
usr/libexec/vdsm/hooks/before_get_all_vm_stats
usr/libexec/vdsm/hooks/before_get_caps
+usr/libexec/vdsm/hooks/before_get_stats
usr/libexec/vdsm/hooks/before_get_vm_stats
usr/libexec/vdsm/hooks/before_network_setup
usr/libexec/vdsm/hooks/before_nic_hotplug
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 76ed135..212155e 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -1264,6 +1264,8 @@
%dir %{_libexecdir}/%{vdsm_name}/hooks/after_get_all_vm_stats
%dir %{_libexecdir}/%{vdsm_name}/hooks/before_get_caps
%dir %{_libexecdir}/%{vdsm_name}/hooks/after_get_caps
+%dir %{_libexecdir}/%{vdsm_name}/hooks/before_get_stats
+%dir %{_libexecdir}/%{vdsm_name}/hooks/after_get_stats
%dir %{_libexecdir}/%{vdsm_name}/hooks/after_hostdev_list_by_caps
%{_datadir}/%{vdsm_name}/addNetwork
%{_datadir}/%{vdsm_name}/delNetwork
diff --git a/vdsm/API.py b/vdsm/API.py
index 3cdc465..3050735 100644
--- a/vdsm/API.py
+++ b/vdsm/API.py
@@ -1321,6 +1321,7 @@
meminfo = utils.readMemInfo()
return meminfo['SwapTotal'] / 1024, meminfo['SwapFree'] / 1024
+ hooks.before_get_stats()
stats = {}
decStats = self._cif._hostStats.get()
@@ -1354,6 +1355,7 @@
# For backwards compatibility, will be removed in the future
stats['haScore'] = stats['haStats']['score']
+ stats = hooks.after_get_stats(stats)
return {'status': doneCode, 'info': stats}
def setLogLevel(self, level):
diff --git a/vdsm/hooks.py b/vdsm/hooks.py
index cedf2be..965459d 100644
--- a/vdsm/hooks.py
+++ b/vdsm/hooks.py
@@ -371,6 +371,16 @@
hookType=_JSON_HOOK)
+def before_get_stats():
+ return _runHooksDir({}, 'before_get_stats', raiseError=True,
+ hookType=_JSON_HOOK)
+
+
+def after_get_stats(caps):
+ return _runHooksDir(caps, 'after_get_stats', raiseError=False,
+ hookType=_JSON_HOOK)
+
+
def after_hostdev_list_by_caps(devices):
return _runHooksDir(devices, 'after_hostdev_list_by_caps',
raiseError=False, hookType=_JSON_HOOK)
diff --git a/vdsm/vdsmd.8.in b/vdsm/vdsmd.8.in
index f6e7158..78989cd 100644
--- a/vdsm/vdsmd.8.in
+++ b/vdsm/vdsmd.8.in
@@ -61,7 +61,9 @@
before_set_num_of_cpus, after_set_num_of_cpus,
before_get_vm_stats, after_get_vm_stats,
before_get_all_vm_stats, after_get_all_vm_stats,
- before_get_caps, after_get_caps, after_hostdev_list_by_caps.
+ before_get_caps, after_get_caps,
+ before_get_stats, after_get_stats,
+ after_hostdev_list_by_caps.
Each hook executes the scripts under
.FN /usr/libexec/vdsm/hooks/<hook-name>/
@@ -71,7 +73,9 @@
Each hook script (except before_vdsm_start, after_vdsm_stop,
before_network_setup and after_network_setup, before_get_vm_stats,
after_get_vm_stats, before_get_all_vm_stats, after_get_all_vm_stats,
-before_get_caps, after_get_caps and after_hostdev_list_by_caps) inherit
+before_get_caps, after_get_caps,
+before_get_stats, after_get_stats,
+and after_hostdev_list_by_caps) inherit
the environment of the VDSM process, with an additional variable
.B _hook_domxml
which holds the path of libvirt's
@@ -162,6 +166,11 @@
a getVdsCapabilities API request.
after_get_caps receives the complete capabilities dictionary within _hook_json.
+before_get_stats and after_get_stats are called before (and after)
+a getVdsStats API request.
+after_get_stats receives the complete host statistics dictionary within
+_hook_json.
+
.SS Hook execution
before_vdsm_start and after_vdsm_stop scripts are executed as user
.I root.
diff --git a/vdsm_hooks/Makefile.am b/vdsm_hooks/Makefile.am
index 337284b..d1aa8b6 100644
--- a/vdsm_hooks/Makefile.am
+++ b/vdsm_hooks/Makefile.am
@@ -124,6 +124,8 @@
after_get_all_vm_stats \
before_get_caps \
after_get_caps \
+ before_get_stats \
+ after_get_stats \
after_hostdev_list_by_caps \
$(NULL)
--
To view, visit https://gerrit.ovirt.org/40403
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id1ff93837abb73f88ececc7571115645b95486cf
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Dan Kenigsberg <danken(a)redhat.com>
9 years
Change in vdsm[master]: block: use the contextmanager for mapMetaOffset
by Federico Simoncelli
Federico Simoncelli has uploaded a new change for review.
Change subject: block: use the contextmanager for mapMetaOffset
......................................................................
block: use the contextmanager for mapMetaOffset
Change-Id: Id1496238ea4f9a1902f816b20959ce3379e87871
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
---
M vdsm/storage/blockSD.py
M vdsm/storage/blockVolume.py
M vdsm/storage/imageRepository/formatConverter.py
3 files changed, 13 insertions(+), 10 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/88/25988/1
diff --git a/vdsm/storage/blockSD.py b/vdsm/storage/blockSD.py
index cac59fa..02c7dfd 100644
--- a/vdsm/storage/blockSD.py
+++ b/vdsm/storage/blockSD.py
@@ -29,6 +29,7 @@
import time
import functools
from collections import namedtuple
+from contextlib import contextmanager
from operator import itemgetter
from vdsm.config import config
@@ -732,11 +733,15 @@
newsize = self.metaSize(self.sdUUID)
lvm.extendLV(self.sdUUID, sd.METADATA, newsize)
+ _mdSlotLock = threading.Lock()
+
+ @contextmanager
def mapMetaOffset(self, vol_name, slotSize):
- if self.getVersion() in VERS_METADATA_LV:
- return self.getVolumeMetadataOffsetFromPvMapping(vol_name)
- else:
- return self.getFreeMetadataSlot(slotSize)
+ with self._mdSlotLock:
+ if self.getVersion() in VERS_METADATA_LV:
+ yield self.getVolumeMetadataOffsetFromPvMapping(vol_name)
+ else:
+ yield self.getFreeMetadataSlot(slotSize)
def _getOccupiedMetadataSlots(self):
stripPrefix = lambda s, pfx: s[len(pfx):]
diff --git a/vdsm/storage/blockVolume.py b/vdsm/storage/blockVolume.py
index 36bfa1f..9a7fa81 100644
--- a/vdsm/storage/blockVolume.py
+++ b/vdsm/storage/blockVolume.py
@@ -73,7 +73,6 @@
class BlockVolume(volume.Volume):
""" Actually represents a single volume (i.e. part of virtual disk).
"""
- _tagCreateLock = threading.Lock()
def __init__(self, repoPath, sdUUID, imgUUID, volUUID):
self.metaoff = None
@@ -169,8 +168,7 @@
imgUUID, volUUID, srcImgUUID, srcVolUUID)
volParent.clone(imgPath, volUUID, volFormat, preallocate)
- with cls._tagCreateLock:
- mdSlot = dom.mapMetaOffset(volUUID, VOLUME_MDNUMBLKS)
+ with dom.mapMetaOffset(volUUID, VOLUME_MDNUMBLKS) as mdSlot:
mdTags = ["%s%s" % (TAG_PREFIX_MD, mdSlot),
"%s%s" % (TAG_PREFIX_PARENT, srcVolUUID),
"%s%s" % (TAG_PREFIX_IMAGE, imgUUID)]
diff --git a/vdsm/storage/imageRepository/formatConverter.py b/vdsm/storage/imageRepository/formatConverter.py
index c701163..11a0039 100644
--- a/vdsm/storage/imageRepository/formatConverter.py
+++ b/vdsm/storage/imageRepository/formatConverter.py
@@ -180,9 +180,9 @@
metaOffset, vol.volUUID)
metaContent = vol.getMetadata()
- with vol._tagCreateLock:
- newMetaOffset = domain.mapMetaOffset(
- vol.volUUID, blockVolume.VOLUME_MDNUMBLKS)
+ with domain.mapMetaOffset(
+ vol.volUUID, blockVolume.VOLUME_MDNUMBLKS) \
+ as newMetaOffset:
if newMetaOffset > metaMaxSlot:
raise se.NoSpaceLeftOnDomain(domain.sdUUID)
--
To view, visit http://gerrit.ovirt.org/25988
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id1496238ea4f9a1902f816b20959ce3379e87871
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Federico Simoncelli <fsimonce(a)redhat.com>
9 years
Change in vdsm[master]: vm: Set disk replica only if it can be prepared
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: vm: Set disk replica only if it can be prepared
......................................................................
vm: Set disk replica only if it can be prepared
We used to set drive replicating state, and then try to prepare the
volume. If the preparing the volume fails, we revert the disk state.
To support LSM between mixed domain types, we need to check if the
replica disk is a block device, and keep this information in
Drive.diskReplicate dict. So we must prepare the volume before setting
the disk in replication state. This also makes sense anyway, since if we
cannot prepare the volume we cannot start the replication.
With the new order, we can simplify logging, and log only one exception
in the outer try block, instead of 3 different exceptions. Since
Vm._delDiskReplica() may raise, we keep now the original error and
re-raise it.
Change-Id: I43814fa41c9cdc830fee9e929baa2e8249b18720
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 12 insertions(+), 14 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/22/40022/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 4e64728..56f43a5 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -25,6 +25,7 @@
from xml.dom.minidom import parseString as _domParseStr
import logging
import os
+import sys
import tempfile
import threading
import time
@@ -3432,27 +3433,24 @@
# prepareVolumePath
dstDiskCopy['device'] = srcDrive.device
+ dstDiskCopy['path'] = self.cif.prepareVolumePath(dstDiskCopy)
try:
self._setDiskReplica(srcDrive, dstDisk)
- except Exception:
- self.log.error("Unable to set the replication for disk '%s' with "
- "destination '%s'", srcDrive.name, dstDisk)
- return errCode['replicaErr']
-
- try:
- dstDiskCopy['path'] = self.cif.prepareVolumePath(dstDiskCopy)
try:
self._startDriveReplication(srcDrive, dstDiskCopy)
except Exception:
- self.log.exception("Unable to start the replication"
- " for %s to %s",
- srcDrive.name, dstDiskCopy)
- self.cif.teardownVolumePath(dstDiskCopy)
- raise
+ t, v, tb = sys.exc_info()
+ try:
+ self._delDiskReplica(srcDrive)
+ except Exception:
+ self.log.exception("Error unsetting disk replica %s",
+ dstDisk)
+ raise t, v, tb
except Exception:
- self.log.exception("Cannot complete the disk replication process")
- self._delDiskReplica(srcDrive)
+ self.log.exception("Unable to start replication for %s to %s",
+ srcDrive.name, dstDiskCopy)
+ self.cif.teardownVolumePath(dstDiskCopy)
return errCode['replicaErr']
if srcDrive.chunked:
--
To view, visit https://gerrit.ovirt.org/40022
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I43814fa41c9cdc830fee9e929baa2e8249b18720
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
9 years