vdsm-patches July 2015

vdsm-patches@lists.fedorahosted.org

44 participants
919 discussions

Change in vdsm[master]: sampling: add 'ncpus' property to HostSample
by fromani＠redhat.com 03 Nov '15

03 Nov '15

Francesco Romani has uploaded a new change for review. Change subject: sampling: add 'ncpus' property to HostSample ...................................................................... sampling: add 'ncpus' property to HostSample to decouple HostStatsThread and hoststats.produce(). Change-Id: I0e7d549b7772e3f38eec2dd9b91bbc6416b549bf Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/hoststats.py M vdsm/virt/sampling.py 2 files changed, 7 insertions(+), 6 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/35/42035/1 diff --git a/vdsm/virt/hoststats.py b/vdsm/virt/hoststats.py index 57c6954..7a23743 100644 --- a/vdsm/virt/hoststats.py +++ b/vdsm/virt/hoststats.py @@ -26,7 +26,7 @@ import v2v -def produce(ncpus, first_sample, last_sample): +def produce(first_sample, last_sample): stats = _empty_stats() if first_sample is None: @@ -45,10 +45,10 @@ jiffies = ( last_sample.totcpu.user - first_sample.totcpu.user) % (2 ** 32) - stats['cpuUser'] = jiffies / interval / ncpus + stats['cpuUser'] = jiffies / interval / last_sample.ncpus jiffies = ( last_sample.totcpu.sys - first_sample.totcpu.sys) % (2 ** 32) - stats['cpuSys'] = jiffies / interval / ncpus + stats['cpuSys'] = jiffies / interval / last_sample.ncpus stats['cpuIdle'] = max(0.0, 100.0 - stats['cpuUser'] - stats['cpuSys']) stats['memUsed'] = last_sample.memUsed diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index 5729a54..401bae9 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -232,7 +232,7 @@ d[p] = {'free': str(free)} return d - def __init__(self, pid): + def __init__(self, pid, ncpus): """ Initialize a HostSample. @@ -242,6 +242,7 @@ super(HostSample, self).__init__() self.interfaces = _get_interfaces_and_samples() self.pidcpu = PidCpuSample(pid) + self.ncpus = ncpus self.totcpu = TotalCpuSample() meminfo = utils.readMemInfo() freeOrCached = (meminfo['MemFree'] + @@ -553,7 +554,7 @@ time.sleep(self._sampleInterval) while not self._stopEvent.isSet(): try: - sample = HostSample(self._pid) + sample = HostSample(self._pid, self._ncpus) self._samples.append(sample) second_last = self._samples.last(nth=2) if second_last is None: @@ -571,7 +572,7 @@ def get(self): first_sample, last_sample, _ = self._samples.stats() - return hoststats.produce(self._ncpus, first_sample, last_sample) + return hoststats.produce(first_sample, last_sample) def _getLinkSpeed(dev): -- To view, visit https://gerrit.ovirt.org/42035 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0e7d549b7772e3f38eec2dd9b91bbc6416b549bf Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

5 33

Change in vdsm[master]: WIP: rename stats cache to vm stats cache
by fromani＠redhat.com 02 Nov '15

02 Nov '15

Francesco Romani has uploaded a new change for review. Change subject: WIP: rename stats cache to vm stats cache ...................................................................... WIP: rename stats cache to vm stats cache Future patch want to modernize HostStats* and port it to periodic Operations infrastructure. To make room for this change and reduce ambiguity, re-introduced the ubiquitous vm* prefix. No changes besides naming. Change-Id: If38c49f686dfc2bc0994d444ff24c7736f2e951b Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M tests/samplingTests.py M tests/vmfakelib.py M vdsm/virt/periodic.py M vdsm/virt/sampling.py M vdsm/virt/vm.py 5 files changed, 16 insertions(+), 16 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/71/40371/1 diff --git a/tests/samplingTests.py b/tests/samplingTests.py index 2624dd7..1f2a65c 100644 --- a/tests/samplingTests.py +++ b/tests/samplingTests.py @@ -250,13 +250,13 @@ self.assertTrue(self._sampleCount >= self.STOP_SAMPLE) -class StatsCacheTests(TestCaseBase): +class VmStatsCacheTests(TestCaseBase): FAKE_CLOCK_STEP = 1 def setUp(self): self.clock = 0 - self.cache = sampling.StatsCache(clock=self.fake_monotonic_time) + self.cache = sampling.VmStatsCache(clock=self.fake_monotonic_time) def fake_monotonic_time(self): self.clock += self.FAKE_CLOCK_STEP diff --git a/tests/vmfakelib.py b/tests/vmfakelib.py index d8ce6ce..544eb5a 100644 --- a/tests/vmfakelib.py +++ b/tests/vmfakelib.py @@ -201,7 +201,7 @@ fake._guestCpuRunning = runCpu if status is not None: fake._lastStatus = status - sampling.stats_cache.add(fake.id) + sampling.vm_stats_cache.add(fake.id) yield fake diff --git a/vdsm/virt/periodic.py b/vdsm/virt/periodic.py index a3f8cc6..00615c3 100644 --- a/vdsm/virt/periodic.py +++ b/vdsm/virt/periodic.py @@ -94,7 +94,7 @@ sampling.VMBulkSampler( libvirtconnection.get(cif), cif.getVMs, - sampling.stats_cache), + sampling.vm_stats_cache), config.getint('vars', 'vm_sample_interval')), # we do this only until we get high water mark notifications diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index 6804406..89f109b 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -375,7 +375,7 @@ EMPTY_SAMPLE = StatsSample(None, None, None, None) -class StatsCache(object): +class VmStatsCache(object): """ Cache for bulk stats samples. Provide facilities to retrieve per-vm samples, and the glue code to deal @@ -391,7 +391,7 @@ VDSM has countermeasures for those cases. Stuck threads are replaced, thanks to Executor. But still, before to be destroyed, a replaced - thread can mistakenly try to add a sample to a StatsCache. + thread can mistakenly try to add a sample to a VmStatsCache. Because of worker thread replacement, that sample from stuck thread can be stale. @@ -402,7 +402,7 @@ between a well behaving call and an unblocked stuck call. """ - _log = logging.getLogger("sampling.StatsCache") + _log = logging.getLogger("sampling.VmStatsCache") def __init__(self, clock=utils.monotonic_time): self._clock = clock @@ -478,7 +478,7 @@ self._vm_last_timestamp[vmid] = monotonic_ts -stats_cache = StatsCache() +vm_stats_cache = VmStatsCache() # this value can be tricky to tune. @@ -492,18 +492,18 @@ class VMBulkSampler(object): - def __init__(self, conn, get_vms, stats_cache, + def __init__(self, conn, get_vms, vm_stats_cache, stats_flags=0, timeout=_TIMEOUT): self._conn = conn self._get_vms = get_vms - self._stats_cache = stats_cache + self._vm_stats_cache = vm_stats_cache self._stats_flags = stats_flags self._skip_doms = ExpiringCache(timeout) self._sampling = Stage() self._log = logging.getLogger("sampling.VMBulkSampler") def __call__(self): - timestamp = self._stats_cache.clock() + timestamp = self._vm_stats_cache.clock() # we are deep in the hot path. bool(ExpiringCache) # *is* costly so we should avoid it if we can. fast_path = (self._sampling.empty and not self._skip_doms) @@ -514,7 +514,7 @@ # If everything's ok, we can skip all the costly checks. bulk_stats = self._conn.getAllDomainStats( self._stats_flags) - self._stats_cache.put(_translate(bulk_stats), timestamp) + self._vm_stats_cache.put(_translate(bulk_stats), timestamp) else: # A previous call got stuck, or not every domain # has properly recovered. Thus we must whitelist domains. @@ -523,7 +523,7 @@ if doms: bulk_stats = self._conn.domainListGetStats( doms, self._stats_flags) - self._stats_cache.put(_translate(bulk_stats), timestamp) + self._vm_stats_cache.put(_translate(bulk_stats), timestamp) def _get_responsive_doms(self): vms = self._get_vms() diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index dcc215b..2b7e3eb 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -1308,7 +1308,7 @@ decStats = {} try: - vm_sample = sampling.stats_cache.get(self.id) + vm_sample = sampling.vm_stats_cache.get(self.id) decStats = vmstats.produce(self, vm_sample.first_value, vm_sample.last_value, @@ -1675,7 +1675,7 @@ nic.name) self._guestEventTime = self._startTime - sampling.stats_cache.add(self.id) + sampling.vm_stats_cache.add(self.id) try: self.guestAgent.connect() except Exception: @@ -3454,7 +3454,7 @@ self.lastStatus = vmstatus.POWERING_DOWN # Terminate the VM's creation thread. self._incomingMigrationFinished.set() - sampling.stats_cache.remove(self.id) + sampling.vm_stats_cache.remove(self.id) self.guestAgent.stop() if self._dom: result = self._destroyVmGraceful() -- To view, visit https://gerrit.ovirt.org/40371 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: If38c49f686dfc2bc0994d444ff24c7736f2e951b Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

3 24

Change in vdsm[master]: sampling: introduce expensive checks
by fromani＠redhat.com 02 Nov '15

02 Nov '15

Francesco Romani has uploaded a new change for review. Change subject: sampling: introduce expensive checks ...................................................................... sampling: introduce expensive checks To improve troubleshooting, introduce more expensive sanity checks, with a default-off tunable to enable them. These sanity checks will make no attempt to recover, will only add logs to report possible issues. Change-Id: I7b3bb707dd60de194eedfc2e3de1efbf05574ff7 Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M lib/vdsm/config.py.in M vdsm/virt/sampling.py 2 files changed, 19 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/91/40391/1 diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in index e213839..b60e836 100644 --- a/lib/vdsm/config.py.in +++ b/lib/vdsm/config.py.in @@ -325,6 +325,10 @@ ('periodic_task_per_worker', '100', 'Max number of tasks which can be queued on workers.' ' This is for internal usage and may change without warning'), + + ('expensive_checks', 'false', + 'Perform additional sanity checks and does additional debug logs' + 'which are expensive performance-wise'), ]), # Section: [devel] diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index b0628a4..3c1510f 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -504,6 +504,7 @@ self._skip_doms = ExpiringCache(timeout) self._sampling = Stage() self._log = logging.getLogger("sampling.VMBulkSampler") + self._extra_check = config.getboolean('sampling', 'expensive_checks') def __call__(self): timestamp = self._vm_stats_cache.clock() @@ -517,7 +518,6 @@ # If everything's ok, we can skip all the costly checks. bulk_stats = self._conn.getAllDomainStats( self._stats_flags) - self._vm_stats_cache.put(_translate(bulk_stats), timestamp) else: # A previous call got stuck, or not every domain # has properly recovered. Thus we must whitelist domains. @@ -526,7 +526,12 @@ if doms: bulk_stats = self._conn.domainListGetStats( doms, self._stats_flags) - self._vm_stats_cache.put(_translate(bulk_stats), timestamp) + else: + bulk_stats = [] + + stats = _translate(bulk_stats) + self._vm_stats_cache.put(stats, timestamp) + self._log_missing_vms(self._get_vms(), stats, timestamp) def _get_responsive_doms(self): vms = self._get_vms() @@ -541,6 +546,14 @@ doms.append(vm_obj._dom._dom) return doms + def _log_missing_vms(self, expected_vms, retrieved_stats, timestamp): + # costly check. add another layer of check before to embark on it. + if self._extra_check: + for vm_id in expected_vms: + if vm_id not in retrieved_stats: + self._log.debug('VM %s not updated in bulk at %f', + vm_id, timestamp) + class HostStatsThread(threading.Thread): """ -- To view, visit https://gerrit.ovirt.org/40391 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I7b3bb707dd60de194eedfc2e3de1efbf05574ff7 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

3 20

Change in vdsm[master]: sampling: simplify flows
by fromani＠redhat.com 02 Nov '15

02 Nov '15

Francesco Romani has uploaded a new change for review. Change subject: sampling: simplify flows ...................................................................... sampling: simplify flows Executor already has a fallback net for uncaught exceptions. Failures in SampleVMs.__call__ are expected to be sporadic, and in these cases we actually want to be as much noisy as we can. This patch removes redundant code in SampleVMs.__call__, to make the flow less convoluted. Change-Id: Ide103d0ed9a694cc9ddd9b0b382e2d81a1bd48c0 Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/sampling.py 1 file changed, 17 insertions(+), 21 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/26/40326/1 diff --git a/vdsm/virt/sampling.py b/vdsm/virt/sampling.py index 09ce2b3..5a205f7 100644 --- a/vdsm/virt/sampling.py +++ b/vdsm/virt/sampling.py @@ -507,27 +507,23 @@ # we are deep in the hot path. bool(ExpiringCache) # *is* costly so we should avoid it if we can. fast_path = (self._sampling.empty and not self._skip_doms) - try: - with self._sampling.performer(): - if fast_path: - # This is expected to be the common case. - # If everything's ok, we can skip all the costly checks. - bulk_stats = self._conn.getAllDomainStats( - self._stats_flags) - else: - # A previous call got stuck, or not every domain - # has properly recovered. Thus we must whitelist domains. - doms = self._get_responsive_doms() - self._log.debug('sampling %d domains', len(doms)) - if doms: - bulk_stats = self._conn.domainListGetStats( - doms, self._stats_flags) - else: - bulk_stats = [] - except Exception: - self._log.exception("vm sampling failed") - else: - self._stats_cache.put(_translate(bulk_stats), timestamp) + + with self._sampling.performer(): + if fast_path: + # This is expected to be the common case. + # If everything's ok, we can skip all the costly checks. + bulk_stats = self._conn.getAllDomainStats( + self._stats_flags) + self._stats_cache.put(_translate(bulk_stats), timestamp) + else: + # A previous call got stuck, or not every domain + # has properly recovered. Thus we must whitelist domains. + doms = self._get_responsive_doms() + self._log.debug('sampling %d domains', len(doms)) + if doms: + bulk_stats = self._conn.domainListGetStats( + doms, self._stats_flags) + self._stats_cache.put(_translate(bulk_stats), timestamp) def _get_responsive_doms(self): vms = self._get_vms() -- To view, visit https://gerrit.ovirt.org/40326 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ide103d0ed9a694cc9ddd9b0b382e2d81a1bd48c0 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

3 18

Change in vdsm[master]: hsm: prepareForShutdown - operations order
by laravot＠redhat.com 01 Nov '15

01 Nov '15

Liron Aravot has uploaded a new change for review. Change subject: hsm: prepareForShutdown - operations order ...................................................................... hsm: prepareForShutdown - operations order Currently cleanupMasterMount() is executed before taskMgr.prepareForShutdown() and before the domain monitor is stopped, that causes to errors during the tasks abortion (becasue of failure to access the path) and to erros when stopping the domain monitoring thread on that domain. The solution introduced in that patch is to change the order of the operations - so that the cleanupMasterMount() will be executed only after the other operations are executed. Change-Id: I9edd84317b08a17db80e265053edaf69582c2a51 Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1161934 Signed-off-by: Liron Aravot <laravot(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/62/36162/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index c8aaf93..8f71d71 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -3433,7 +3433,6 @@ # stop spm tasks if spm etc.) try: self._connectionMonitor.stopMonitoring() - sp.StoragePool.cleanupMasterMount() self.__releaseLocks() for spUUID in self.pools: @@ -3454,6 +3453,7 @@ exc_info=True) self.taskMng.prepareForShutdown() + sp.StoragePool.cleanupMasterMount() except: pass -- To view, visit http://gerrit.ovirt.org/36162 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9edd84317b08a17db80e265053edaf69582c2a51 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Liron Aravot <laravot(a)redhat.com>

5 17

Change in vdsm[master]: vdsm: added functionality to teardownImage when disk has bee...
by cshereme＠redhat.com 01 Nov '15

01 Nov '15

Candace Sheremeta has uploaded a new change for review. Change subject: vdsm: added functionality to teardownImage when disk has been deleted ...................................................................... vdsm: added functionality to teardownImage when disk has been deleted added code to teardownImage in hsm.py so that teardownImage reports "Volume does not exist" for a previously deleted disk, where it previously simply reported "OK" - teardownImage now checks to see if volume exists before attempting to delete it Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1184718 Change-Id: Ia929dfdc78ccaa736033e41a77bce861d5a27769 Signed-off-by: Candace Sheremeta <cshereme(a)redhat.com> --- M vdsm/storage/hsm.py 1 file changed, 7 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/41/38241/1 diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py index 8c75277..45e6634 100644 --- a/vdsm/storage/hsm.py +++ b/vdsm/storage/hsm.py @@ -3271,6 +3271,13 @@ vars.task.getSharedLock(STORAGE, sdUUID) dom = sdCache.produce(sdUUID) + allVols = dom.getAllVolumes() + # Filter volumes related to this image + imgVolumes = sd.getVolsOfImage(allVols, imgUUID).keys() + + if volUUID not in imgVolumes: + raise se.VolumeDoesNotExist(volUUID) + dom.deactivateImage(imgUUID) @public -- To view, visit https://gerrit.ovirt.org/38241 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia929dfdc78ccaa736033e41a77bce861d5a27769 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Candace Sheremeta <cshereme(a)redhat.com>

5 7

Change in vdsm[master]: tests: AsyncDispatcher tests suite
by Piotr Kliczewski 29 Oct '15

29 Oct '15

Piotr Kliczewski has uploaded a new change for review. Change subject: tests: AsyncDispatcher tests suite ...................................................................... tests: AsyncDispatcher tests suite Change-Id: Ie18a58d1d8ffe4f8ea2cd8dec9bb2e508c73c7f8 Signed-off-by: pkliczewski <piotr.kliczewski(a)gmail.com> --- M lib/yajsonrpc/stomp.py M tests/Makefile.am A tests/stompAsyncDispatcherTests.py 3 files changed, 172 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/94/43394/1 diff --git a/lib/yajsonrpc/stomp.py b/lib/yajsonrpc/stomp.py index a02a06f..4a2364d 100644 --- a/lib/yajsonrpc/stomp.py +++ b/lib/yajsonrpc/stomp.py @@ -19,7 +19,7 @@ from uuid import uuid4 from collections import deque -from vdsm.utils import monotonic_time +from vdsm import utils import re # REQUIRED_FOR: engine-3.5 @@ -361,10 +361,11 @@ return self._parser.popFrame() def _update_outgoing_heartbeat(self): - self._lastOutgoingTimeStamp = monotonic_time() + self._lastOutgoingTimeStamp = utils.monotonic_time() def _outgoing_heartbeat_expiration_interval(self): - since_last_update = (monotonic_time() - self._lastOutgoingTimeStamp) + now = utils.monotonic_time() + since_last_update = (now - self._lastOutgoingTimeStamp) return (self._outgoing_heartbeat_in_milis / 1000.0) - since_last_update def next_check_interval(self): @@ -411,7 +412,7 @@ return True def _milis(self): - return int(round(monotonic_time() * 1000)) + return int(round(utils.monotonic_time() * 1000)) def handle_close(self, dispatcher): self.connection.close() diff --git a/tests/Makefile.am b/tests/Makefile.am index 4d42b55..15dbd7b 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -96,6 +96,7 @@ sslTests.py \ stompAdapterTests.py \ stompAsyncClientTests.py \ + stompAsyncDispatcherTests.py \ storageMailboxTests.py \ storageMonitorTests.py \ storageServerTests.py \ diff --git a/tests/stompAsyncDispatcherTests.py b/tests/stompAsyncDispatcherTests.py new file mode 100644 index 0000000..9d50cee --- /dev/null +++ b/tests/stompAsyncDispatcherTests.py @@ -0,0 +1,166 @@ +# +# Copyright 2015 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# +import itertools +from collections import deque +from monkeypatch import MonkeyPatch + +from testlib import VdsmTestCase as TestCaseBase +from yajsonrpc.stomp import AsyncDispatcher, Command, Frame, Headers +from vdsm import utils + + +class TestConnection(object): + + def __init__(self): + self.closed = False + + def close(self): + self.closed = True + + +class TestFrameHandler(object): + + def __init__(self): + self.handle_connect_called = False + self._outbox = deque() + + def handle_connect(self, dispatcher): + self.handle_connect_called = True + + def handle_frame(self, dispatcher, frame): + self.queue_frame(frame) + + def peek_message(self): + return self._outbox[0] + + def pop_message(self): + return self._outbox.popleft() + + @property + def has_outgoing_messages(self): + return (len(self._outbox) > 0) + + def queue_frame(self, frame): + self._outbox.append(frame) + + +class TestDispatcher(object): + + def __init__(self, data): + self._data = data + + def recv(self, buffer_size): + return self._data + + def send(self, data): + return len(data) + + +class FakeTimeGen(object): + + def __init__(self, list): + self._chain = itertools.chain(list) + + def get_fake_time(self): + return self._chain.next() + + +class AsyncDispatcherTest(TestCaseBase): + + def test_handle_connect(self): + frame_handler = TestFrameHandler() + dispatcher = AsyncDispatcher(TestConnection(), frame_handler) + + dispatcher.handle_connect(None) + + self.assertTrue(frame_handler.handle_connect_called) + + def test_handle_read(self): + frame_handler = TestFrameHandler() + headers = {Headers.CONTENT_LENGTH: '78', + Headers.DESTINATION: 'jms.topic.vdsm_responses', + Headers.CONTENT_TYPE: 'application/json', + Headers.SUBSCRIPTION: 'ad052acb-a934-4e10-8ec3-00c7417ef8d'} + body = ('{"jsonrpc": "2.0", "id": "e8a936a6-d886-4cfa-97b9-2d54209053f' + 'f", "result": []}') + frame = Frame(command=Command.MESSAGE, headers=headers, body=body) + dispatcher = AsyncDispatcher(TestConnection(), frame_handler) + + dispatcher.handle_read(TestDispatcher(frame.encode())) + + self.assertTrue(frame_handler.has_outgoing_messages) + recv_frame = frame_handler.pop_message() + self.assertEquals(Command.MESSAGE, recv_frame.command) + self.assertEquals(body, recv_frame.body) + + @MonkeyPatch(utils, 'monotonic_time', + FakeTimeGen([4000000.0, 4000002.0]).get_fake_time) + def test_heartbeat_calc(self): + dispatcher = AsyncDispatcher(TestConnection(), TestFrameHandler()) + dispatcher.setHeartBeat(8000, 0) + + self.assertEquals(6, dispatcher.next_check_interval()) + + @MonkeyPatch(utils, 'monotonic_time', + FakeTimeGen([4000000.0, 4000012.0]).get_fake_time) + def test_heartbeat_exceeded(self): + frame_handler = TestFrameHandler() + dispatcher = AsyncDispatcher(TestConnection(), frame_handler) + dispatcher.setHeartBeat(8000, 0) + + self.assertTrue(dispatcher.writable(None)) + self.assertTrue(frame_handler.has_outgoing_messages) + + def test_no_incoming_heartbeat(self): + dispatcher = AsyncDispatcher(TestConnection(), TestFrameHandler()) + + with self.assertRaises(ValueError): + dispatcher.setHeartBeat(8000, 8000) + + def test_no_heartbeat(self): + dispatcher = AsyncDispatcher(TestConnection(), TestFrameHandler()) + dispatcher.setHeartBeat(0, 0) + + self.assertIsNone(dispatcher.next_check_interval()) + + def test_handle_write(self): + headers = {Headers.CONTENT_LENGTH: '78', + Headers.DESTINATION: 'jms.topic.vdsm_responses', + Headers.CONTENT_TYPE: 'application/json', + Headers.SUBSCRIPTION: 'ad052acb-a934-4e10-8ec3-00c7417ef8d'} + body = ('{"jsonrpc": "2.0", "id": "e8a936a6-d886-4cfa-97b9-2d54209053f' + 'f", "result": []}') + frame = Frame(command=Command.MESSAGE, headers=headers, body=body) + frame_handler = TestFrameHandler() + frame_handler.handle_frame(None, frame) + + dispatcher = AsyncDispatcher(TestConnection(), frame_handler) + self.assertTrue(dispatcher.writable(None)) + + dispatcher.handle_write(TestDispatcher('')) + self.assertFalse(frame_handler.has_outgoing_messages) + + def test_handle_close(self): + connection = TestConnection() + dispatcher = AsyncDispatcher(connection, TestFrameHandler()) + + dispatcher.handle_close(None) + + self.assertTrue(connection.closed) -- To view, visit https://gerrit.ovirt.org/43394 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie18a58d1d8ffe4f8ea2cd8dec9bb2e508c73c7f8 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>

4 14

Change in vdsm[master]: janitorial: vm: remove _reportError
by fromani＠redhat.com 29 Oct '15

29 Oct '15

Francesco Romani has uploaded a new change for review. Change subject: janitorial: vm: remove _reportError ...................................................................... janitorial: vm: remove _reportError Vm._reportError was an early attempt to build what we have now with response.error(). With the new code in place there is no more reason to use it, so this patch removes it. Change-Id: Ie23167c0289bdb0326dcfe48903cd1bae905e9b5 Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/virt/vm.py 1 file changed, 15 insertions(+), 27 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/69/38269/1 diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py index cd392c7..4be63ce 100644 --- a/vdsm/virt/vm.py +++ b/vdsm/virt/vm.py @@ -42,7 +42,7 @@ from vdsm import utils from vdsm.compat import pickle from vdsm.config import config -from vdsm.define import ERROR, NORMAL, doneCode, errCode +from vdsm.define import ERROR, NORMAL, doneCode from vdsm.netinfo import DUMMY_BRIDGE from storage import outOfProcess as oop from storage import sd @@ -2650,15 +2650,15 @@ if not params: self.log.error("updateVmPolicy got an empty policy.") - return self._reportError(key='MissParam', - msg="updateVmPolicy got an empty policy.") + return response.error('MissParam', + 'updateVmPolicy got an empty policy.') # # Get the current QoS block metadata_modified = False qos = self._getVmPolicy() if qos is None: - return self._reportError(key='updateVmPolicyErr') + return response.error('updateVmPolicyErr') # # Process provided properties, remove property after it is processed @@ -2712,8 +2712,7 @@ if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: return response.error('noVM') else: - return self._reportError(key='updateVmPolicyErr', - msg=e.message) + return response.error('updateVmPolicyErr', e.message) return {'status': doneCode} @@ -2771,9 +2770,9 @@ found_device = self._findDeviceByNameOrPath(device_name, device_path) if found_device is None: - return self._reportError( - key='updateIoTuneErr', - msg="Device {} not found".format(device_name)) + return response.error( + 'updateIoTuneErr', + "Device {} not found".format(device_name)) # Merge the update with current values dom = found_device.getXML() @@ -2799,8 +2798,7 @@ if e.get_error_code() == libvirt.VIR_ERR_NO_DOMAIN: return response.error('noVM') else: - return self._reportError(key='updateIoTuneErr', - msg=e.message) + return response.error('updateIoTuneErr', e.message) # Update both the ioTune arguments and device xml DOM # so we are still up-to-date @@ -3956,7 +3954,7 @@ def setBalloonTarget(self, target): if self._dom is None: - return self._reportError(key='balloonErr') + return response.error('balloonErr') try: target = int(target) self._dom.setMemory(target) @@ -3980,8 +3978,8 @@ try: self._dom.setSchedulerParameters({'vcpu_quota': int(quota)}) except ValueError: - return self._reportError(key='cpuTuneErr', - msg='an integer is required for period') + return response.error('cpuTuneErr', + 'an integer is required for period') except libvirt.libvirtError as e: return self._reportException(key='cpuTuneErr', msg=e.message) return {'status': doneCode} @@ -3990,21 +3988,11 @@ try: self._dom.setSchedulerParameters({'vcpu_period': int(period)}) except ValueError: - return self._reportError(key='cpuTuneErr', - msg='an integer is required for period') + return response.error('cpuTuneErr', + 'an integer is required for period') except libvirt.libvirtError as e: return self._reportException(key='cpuTuneErr', msg=e.message) return {'status': doneCode} - - def _reportError(self, key, msg=None): - """ - Produce an error status. - """ - if msg is None: - error = errCode[key] - else: - error = response.error(key, msg) - return error def _reportException(self, key, msg=None): """ @@ -4012,7 +4000,7 @@ This method should be called only within exception-handling context. """ self.log.exception("Operation failed") - return self._reportError(key, msg) + return response.error(key, msg) def _getUnderlyingDeviceAddress(self, devXml): """ -- To view, visit https://gerrit.ovirt.org/38269 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ie23167c0289bdb0326dcfe48903cd1bae905e9b5 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>

6 78

Change in vdsm[master]: API: use modern locking around VM.destroy
by fromani＠redhat.com 29 Oct '15

29 Oct '15

Francesco Romani has uploaded a new change for review. Change subject: API: use modern locking around VM.destroy ...................................................................... API: use modern locking around VM.destroy cleanup a bit VM.destroy() without functional changes: - log outside the lock - use context managers for lock Change-Id: I799b3b0d3e992df988efc6ab8e9263909062db0c Signed-off-by: Francesco Romani <fromani(a)redhat.com> --- M vdsm/API.py 1 file changed, 5 insertions(+), 5 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/89/42889/1 diff --git a/vdsm/API.py b/vdsm/API.py index 242783d..55eaa80 100644 --- a/vdsm/API.py +++ b/vdsm/API.py @@ -332,9 +332,11 @@ """ Destroy the specified VM. """ - self._cif.vmContainerLock.acquire() - self.log.info("vmContainerLock acquired by vm %s", self._UUID) - try: + self.log.debug('About to destroy VM %s (reason=%s)', + self._UUID, + 'N/A' if reason == vmexitreason.SUCCESS + else str(reason)) + with self._cif.vmContainerLock: v = self._cif.vmContainer.get(self._UUID) if not v: return errCode['noVM'] @@ -343,8 +345,6 @@ if status['status']['code'] == 0: status['status']['message'] = "Machine destroyed" return status - finally: - self._cif.vmContainerLock.release() def getMigrationStatus(self): """ -- To view, visit https://gerrit.ovirt.org/42889 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I799b3b0d3e992df988efc6ab8e9263909062db0c Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: automation(a)ovirt.org

4 14

Change in vdsm[master]: ssl: change default protocol
by Piotr Kliczewski 29 Oct '15

29 Oct '15

Piotr Kliczewski has uploaded a new change for review. Change subject: ssl: change default protocol ...................................................................... ssl: change default protocol This setting breaks backward compatibility with older engines. Change-Id: I40267cb07b19d444c7d85aba6d1160c27e8fe3a6 Signed-off-by: pkliczewski <piotr.kliczewski(a)gmail.com> Bug-Url: https://bugzilla.redhat.com/1229765 --- M lib/vdsm/config.py.in 1 file changed, 1 insertion(+), 1 deletion(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/57/43457/1 diff --git a/lib/vdsm/config.py.in b/lib/vdsm/config.py.in index 3221aab..fbee478 100644 --- a/lib/vdsm/config.py.in +++ b/lib/vdsm/config.py.in @@ -196,7 +196,7 @@ ('transient_disks_repository', '@VDSMLIBDIR@/transient', 'Local path to the transient disks repository.'), - ('ssl_protocol', 'sslv23', + ('ssl_protocol', 'tlsv1', 'SSL protocol used by encrypted connection'), ('connection_stats_timeout', '3600', -- To view, visit https://gerrit.ovirt.org/43457 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I40267cb07b19d444c7d85aba6d1160c27e8fe3a6 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>

5 15

← Newer
1
...
6
7
8
9
10
11
12
...
92
Older →

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches July 2015