Change in vdsm[master]: doc: add basic quickstart for containers
by fromani＠redhat.com
Francesco Romani has uploaded a new change for review.
Change subject: doc: add basic quickstart for containers
......................................................................
doc: add basic quickstart for containers
Change-Id: I0fb768ea97dd719cde9bd5e57e1b7cabe4b0f0ae
Signed-off-by: Francesco Romani <fromani(a)redhat.com>
---
M README
1 file changed, 57 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/59824/1
diff --git a/README b/README
index 955dfa9..370028e 100644
--- a/README
+++ b/README
@@ -32,6 +32,63 @@
The 'vdsm.spec' file demonstrates how to distribute Vdsm as an RPM
package.
+Containers support
+==================
+
+While Vdsm focus is on managing KVM virtual machines, it could also run
+containers alongside virtual machines using popular formats and runtimes,
+such as rkt and appc images.
+
+Containers are reported as special-purpose VMs to the clients, and responds
+to the Vdsm API invoked on them.
+If a particular container runtime doesn't support an operation, this will
+fail with a standard Vdsm error.
+
+To try this out, you just need to install the 'vdsm-containers subpackage'.
+Make sure to restart *both* supervdsmd and vdsmd once that package is installed.
+You'll also need to have the container runtime you wish to use installed on
+the same host which runs Vdsm. The supported runtimes are:
+
+- rkt (main supported runtime)
+- runc (work in progress)
+- docker (very experimental)
+
+The supported image container formats depend on the container runtime.
+For example, modern rkt could run docker images transparently.
+
+To check if the Vdsm is properly configured to run containers, just do:
+
+# vdsClient -s 0 getVdsCaps | grep containers
+
+ containers = ['rkt']
+
+This means that this Vdsm could also run containers using `rkt`.
+
+Any Engine >= 3.6 could handle containers - they are just VMs from its perspective.
+You just need to set few custom properties. Run this command
+on your Engine host:
+
+# engine-config -s UserDefinedVMProperties='volumeMap=^[a-zA-Z_-]+:[a-zA-Z_-]+$;containerImage=^[a-zA-Z]+(://|)[a-zA-Z]+$;containerType=^(rkt|docker)$' --cver=3.6
+
+replace --cver=3.6 with the version of the Engine you are using.
+Now restart Ovirt Engine, and log in.
+
+You can now run any container. The user defined VM properties define
+the key settings which are not (yet) exposed in the engine UI.
+
+- volumeMap allows you to mount any disk inside the container, should you
+ need any persistence. It is a mapping between disks (e.g. vda)
+ and mountpoint (e.g. data). The mountpoints are just container-dependent labels.
+
+- containerImage is the URL or path of any container image supported by your
+ runtime. E.g. 'docker://redis'
+
+- containerType allows to select the runtime you want to use (e.g. rkt)
+
+Please be aware that many settings are ignored by containers, like all
+the device configurations. Only memory and CPU settings are honoured.
+
+
Getting Help
============
--
To view, visit https://gerrit.ovirt.org/59824
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0fb768ea97dd719cde9bd5e57e1b7cabe4b0f0ae
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani <fromani(a)redhat.com>
6 years, 4 months
Change in vdsm[master]: py3 tests: ported imagetickets_test.py to python3
by igoihman＠redhat.com
Irit Goihman has uploaded a new change for review.
Change subject: py3 tests: ported imagetickets_test.py to python3
......................................................................
py3 tests: ported imagetickets_test.py to python3
ported relevant code to python3 in order that imagetickets_test.py
will be python3 compatible.
Change-Id: If13fb30966d98fa0381da25f0f2907899df12e96
Signed-off-by: Irit Goihman <igoihman(a)redhat.com>
---
M tests/Makefile.am
M tests/imagetickets_test.py
M vdsm/storage/imagetickets.py
3 files changed, 6 insertions(+), 5 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/59/59859/1
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 352f5be..16119bd 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -179,7 +179,6 @@
hooksTests.py \
hostdevTests.py \
hoststatsTests.py \
- imagetickets_test.py \
iscsiTests.py \
lvmTests.py \
main.py \
diff --git a/tests/imagetickets_test.py b/tests/imagetickets_test.py
index 39a2800..f2ea111 100644
--- a/tests/imagetickets_test.py
+++ b/tests/imagetickets_test.py
@@ -18,15 +18,17 @@
# Refer to the README and COPYING files for full details of the license
#
-import httplib
import json
import socket
import io
+
+from six.moves import http_client
from monkeypatch import MonkeyPatch
from testlib import VdsmTestCase
from testlib import expandPermutations, permutations
from testlib import recorded
+
from vdsm.storage import exception as se
from storage import imagetickets
@@ -158,7 +160,7 @@
self.assertTrue(err_msg in e.value)
@MonkeyPatch(imagetickets, 'uhttp', FakeUHTTP())
- @permutations([[httplib.HTTPException], [socket.error], [OSError]])
+ @permutations([[http_client.HTTPException], [socket.error], [OSError]])
def test_image_tickets_error(self, exc_type):
ticket = create_ticket(uuid="uuid")
diff --git a/vdsm/storage/imagetickets.py b/vdsm/storage/imagetickets.py
index 281a0e9..476f249 100644
--- a/vdsm/storage/imagetickets.py
+++ b/vdsm/storage/imagetickets.py
@@ -19,12 +19,12 @@
#
import functools
-import httplib
import json
import logging
import os
from contextlib import closing
+from six.moves import http_client
try:
from ovirt_imageio_daemon import uhttp
@@ -74,7 +74,7 @@
try:
con.request(method, "/tickets/%s" % uuid, body=body)
res = con.getresponse()
- except (httplib.HTTPException, EnvironmentError) as e:
+ except (http_client.HTTPException, EnvironmentError) as e:
raise se.ImageTicketsError("Error communicating with "
"ovirt-imageio-daemon: "
"{error}".format(error=e))
--
To view, visit https://gerrit.ovirt.org/59859
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If13fb30966d98fa0381da25f0f2907899df12e96
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Irit Goihman <igoihman(a)redhat.com>
6 years, 4 months
Change in vdsm[master]: virt: Always execute after_hibernation event
by Vinzenz Feenstra
Vinzenz Feenstra has uploaded a new change for review.
Change subject: virt: Always execute after_hibernation event
......................................................................
virt: Always execute after_hibernation event
Previously we had no way that the after_hibernation event got executed.
The enableGuestEvents value was never set to true and we're now also
dropping it on the migration destination.
Since there is also no parameter for enabling guest events on the suspend
call it, we decided that it is much more beneficial to let the VM always
know that it is going to be hibernated and/or has been resumed from
hibernation.
Since purely from theory this information can be also determined without
the notifications there's no security issue that the Virtual Machine would
know too much about it's environment. However it's beneficial for a range of
applications which need to resynchronize, adjust the time or re-accquire an
IP address via DHCP etc.
Change-Id: Ic52b8c22d57028fb6459c7faa2bcfebd8f32d6c4
Bug-Url: https://bugzilla.redhat.com/1349907
Signed-off-by: Vinzenz Feenstra <vfeenstr(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 12 insertions(+), 7 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/14/59914/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index c2b1b93..d8e4239 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -1866,14 +1866,19 @@
except Exception:
self.log.exception("Failed to connect to guest agent channel")
- try:
- if self.conf.get('enableGuestEvents', False):
- if self.lastStatus == vmstatus.MIGRATION_DESTINATION:
+ if self.lastStatus == vmstatus.RESTORING_STATE:
+ try:
+ self.guestAgent.events.after_hibernation()
+ except Exception:
+ self.log.exception("Unexpected error on guest after "
+ "hibernation notification")
+ elif self.conf.get('enableGuestEvents', False):
+ if self.lastStatus == vmstatus.MIGRATION_DESTINATION:
+ try:
self.guestAgent.events.after_migration()
- elif self.lastStatus == vmstatus.RESTORING_STATE:
- self.guestAgent.events.after_hibernation()
- except Exception:
- self.log.exception("Unexpected error on guest event notification")
+ except Exception:
+ self.log.exception("Unexpected error on guest after "
+ "migration notification")
# Drop enableGuestEvents from conf - Not required from here anymore
self.conf.pop('enableGuestEvents', None)
--
To view, visit https://gerrit.ovirt.org/59914
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ic52b8c22d57028fb6459c7faa2bcfebd8f32d6c4
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Vinzenz Feenstra <vfeenstr(a)redhat.com>
6 years, 4 months
Change in vdsm[master]: virt: enable libgfapi
by ahino＠redhat.com
Ala Hino has uploaded a new change for review.
Change subject: virt: enable libgfapi
......................................................................
virt: enable libgfapi
This change is based on Federico's changes:
https://gerrit.ovirt.org/33768/
Change-Id: I54b81e87b959b0b49c0f06810f88410e7c75de1d
Signed-off-by: Federico Simoncelli <fsimonce(a)redhat.com>
Signed-off-by: Ala Hino <ahino(a)redhat.com>
---
M vdsm/storage/glusterVolume.py
M vdsm/storage/hsm.py
M vdsm/virt/vm.py
3 files changed, 22 insertions(+), 5 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/61/44061/1
diff --git a/vdsm/storage/glusterVolume.py b/vdsm/storage/glusterVolume.py
index 8f701ba..109d1c9 100644
--- a/vdsm/storage/glusterVolume.py
+++ b/vdsm/storage/glusterVolume.py
@@ -54,4 +54,5 @@
return {'volType': VmVolumeInfo.TYPE_NETWORK, 'path': glusterPath,
'protocol': 'gluster', 'volPort': volPort,
'volTransport': volTrans,
- 'volfileServer': volfileServer}
+ 'volfileServer': volfileServer,
+ 'bricks': volInfo[volname]['bricks']}
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index f68d3bb..109cdfa 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -3209,8 +3209,7 @@
path = os.path.join(dom.domaindir, sd.DOMAIN_IMAGES, imgUUID,
volUUID)
volInfo = {'domainID': sdUUID, 'imageID': imgUUID,
- 'volumeID': volUUID, 'path': path,
- 'volType': "path"}
+ 'volumeID': volUUID, 'path': path}
leasePath, leaseOffset = dom.getVolumeLease(imgUUID, volUUID)
@@ -3221,8 +3220,8 @@
})
imgVolumesInfo.append(volInfo)
- if volUUID == leafUUID:
- leafInfo = volInfo
+
+ leafInfo = dom.produceVolume(imgUUID, leafUUID).getVmVolumeInfo()
return {'path': leafPath, 'info': leafInfo,
'imgVolumesInfo': imgVolumesInfo}
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index 71a74b3..2976f3b 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -805,6 +805,23 @@
# A destroy request has been issued, exit early
break
drive['path'] = self.cif.prepareVolumePath(drive, self.id)
+ if drive.get('diskType') == DISK_TYPE.NETWORK:
+ volinfo = drive.get('volumeInfo')
+ drive['path'] = volinfo['path']
+ drive['protocol'] = volinfo['protocol']
+ if drive.get('hosts') is None:
+ """
+ THIS CODE COMMENTED OUT DUE TO A BUG IN LIBVIRT,
+ CANNOT WORK WITH MULTIPLE HOSTS
+ """
+ """
+ hosts = [dict(name=brick.split(":")[0],
+ port='0', transport='tcp')
+ for brick in volinfo['bricks']]
+ """
+ hosts = [dict(name=volinfo['bricks'][0].split(":")[0],
+ port='0', transport='tcp')]
+ drive['hosts'] = hosts
else:
# Now we got all the resources we needed
--
To view, visit https://gerrit.ovirt.org/44061
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I54b81e87b959b0b49c0f06810f88410e7c75de1d
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Ala Hino <ahino(a)redhat.com>
6 years, 5 months
Change in vdsm[master]: tests: Add basic tests for RWLock
by alitke＠redhat.com
Adam Litke has uploaded a new change for review.
Change subject: tests: Add basic tests for RWLock
......................................................................
tests: Add basic tests for RWLock
Change-Id: I77b9ea1cd6d378738fbf14de119c88bf0ebc94e2
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M tests/miscTests.py
1 file changed, 18 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/72/42772/1
diff --git a/tests/miscTests.py b/tests/miscTests.py
index 976b503..c6d9690 100644
--- a/tests/miscTests.py
+++ b/tests/miscTests.py
@@ -1268,3 +1268,21 @@
def _run(self):
self.result = self._func()
+
+
+class RWLockTests(TestCaseBase):
+
+ def test_multiple_acquire(self):
+ lock = misc.RWLock()
+ self.assertTrue(lock.acquire(exclusive=True))
+ self.assertTrue(lock.acquire(exclusive=True))
+
+ def test_demote(self):
+ lock = misc.RWLock()
+ self.assertTrue(lock.acquire(exclusive=True))
+ self.assertTrue(lock.acquire(exclusive=False))
+
+ def test_promote(self):
+ lock = misc.RWLock()
+ self.assertTrue(lock.acquire(False))
+ self.assertRaises(RuntimeError, lock.acquire, True)
--
To view, visit https://gerrit.ovirt.org/42772
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I77b9ea1cd6d378738fbf14de119c88bf0ebc94e2
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
6 years, 5 months
Change in vdsm[master]: lvm: Fail loudly if called with unexptected input.
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: lvm: Fail loudly if called with unexptected input.
......................................................................
lvm: Fail loudly if called with unexptected input.
When creating pvs with the force option, we are very carefull to accept
only True. When using the jsonrpc transport, engine was sending "true"
and "false", causing the call to fail misteiously.
Now we are also carefull about rejecting invlid input, making debugging
easier.
Change-Id: If9e6754d4aa2efaf894a9309cfaa4595d710063b
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/lvm.py
1 file changed, 5 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/29/37329/1
diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py
index aa3c04b..549839a 100644
--- a/vdsm/storage/lvm.py
+++ b/vdsm/storage/lvm.py
@@ -724,6 +724,11 @@
else:
raise
+ # We must be very carefull here; any value execpt True or False is a user
+ # error.
+ if type(force) != bool:
+ raise ValueError("Invalid value for 'force': %r" % force)
+
if force is True:
options = ("-y", "-ff")
_initpvs_removeHolders()
--
To view, visit http://gerrit.ovirt.org/37329
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: If9e6754d4aa2efaf894a9309cfaa4595d710063b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
6 years, 5 months
Change in vdsm[master]: hsm: Report vg name in getDeviceList
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: hsm: Report vg name in getDeviceList
......................................................................
hsm: Report vg name in getDeviceList
Hosted engine needs the iscsi session info used by the hosted engine
storage domain. getDeviceList() seems to include the needed info, but it
does not report the vg name for each device, making it hard to match the
iscsi session info and the hosted engine storage domain.
We return now the vg name of each device, which seems to be useful info
regardless of hosted engine needs, and can be used on the engine side
for reconstructing host state or validating engine view vs host view.
Change-Id: I116714cb5143ea92f5cb54c3f80f895c07ada536
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/rpc/vdsmapi-schema.json
M vdsm/storage/hsm.py
2 files changed, 11 insertions(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/23/45823/1
diff --git a/vdsm/rpc/vdsmapi-schema.json b/vdsm/rpc/vdsmapi-schema.json
index e0e95b9..1b2a171 100644
--- a/vdsm/rpc/vdsmapi-schema.json
+++ b/vdsm/rpc/vdsmapi-schema.json
@@ -1473,6 +1473,10 @@
#
# @status: The device status (free/used/unknown)
#
+# @vgname: The LVM volume group name, if this device is used as
+# a physical volume. This is typically a storage domain
+# UUID.
+#
# Since: 4.10.0
#
# Notes: The value of @serial may be dependent on the current host so this
@@ -1490,7 +1494,8 @@
'pathstatus': ['BlockDevicePathInfo'],
'pathlist': ['IscsiSessionInfo'], 'logicalblocksize': 'uint',
'physicalblocksize': 'uint', 'partitioned': 'bool',
- 'pvsize': 'uint', 'status': 'BlockDeviceStatus'}}
+ 'pvsize': 'uint', 'status': 'BlockDeviceStatus',
+ 'vgname': 'str'}}
##
# @Host.getDeviceList:
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index 1b8c064..32e16c3 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -2019,14 +2019,18 @@
pvuuid = pv.uuid
pvsize = pv.size
vguuid = pv.vg_uuid
+ vgname = pv.vg_name
else:
pvuuid = ""
pvsize = ""
vguuid = ""
+ vgname = ""
devInfo = {'GUID': dev.get("guid", ""), 'pvUUID': pvuuid,
'pvsize': str(pvsize),
- 'vgUUID': vguuid, 'vendorID': dev.get("vendor", ""),
+ 'vgUUID': vguuid,
+ 'vgname': vgname,
+ 'vendorID': dev.get("vendor", ""),
'productID': dev.get("product", ""),
'fwrev': dev.get("fwrev", ""),
"serial": dev.get("serial", ""),
--
To view, visit https://gerrit.ovirt.org/45823
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I116714cb5143ea92f5cb54c3f80f895c07ada536
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
6 years, 5 months
Change in vdsm[master]: utils: Consider sleep time in deadline calculation
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: utils: Consider sleep time in deadline calculation
......................................................................
utils: Consider sleep time in deadline calculation
This patches fixes 2 utils.retry() broken tests:
- The special case when deadline has reached when an operation was done.
Previously we use to sleep and perform another retry, now we bail out.
- The special case when deadline was not reached when an operation was
done, but we don't have time for sleep (sleeping will reach or exceed
the deadline). Previously we used to sleep and perform another retry,
now we bail out.
Change-Id: I7f41c6b21e3432159c13d46cfe75d1f6236cbb8c
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M lib/vdsm/utils.py
M tests/utilsTests.py
2 files changed, 3 insertions(+), 8 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/00/46400/1
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index caf4cc2..4a4b153 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -938,10 +938,7 @@
if tries in [0, None]:
tries = -1
- if timeout in [0, None]:
- timeout = -1
-
- startTime = monotonic_time()
+ deadline = monotonic_time() + timeout if timeout else None
while True:
tries -= 1
@@ -951,7 +948,7 @@
if tries == 0:
raise
- if (timeout > 0) and ((monotonic_time() - startTime) > timeout):
+ if deadline and monotonic_time() + sleep >= deadline:
raise
if stopCallback is not None and stopCallback():
diff --git a/tests/utilsTests.py b/tests/utilsTests.py
index d213de5..fd4f42e 100644
--- a/tests/utilsTests.py
+++ b/tests/utilsTests.py
@@ -44,7 +44,7 @@
from testlib import permutations, expandPermutations
from testlib import VdsmTestCase as TestCaseBase
from testValidation import checkSudo
-from testValidation import brokentest, stresstest
+from testValidation import stresstest
from multiprocessing import Process
EXT_SLEEP = "sleep"
@@ -86,7 +86,6 @@
# Make sure we had the proper amount of iterations before failing
self.assertEquals(counter[0], limit)
- @brokentest("deadline is not respected")
@MonkeyPatch(utils, 'monotonic_time', FakeTime(0))
@MonkeyPatch(time, 'sleep', fake_sleep)
def testTimeoutDeadlineReached(self):
@@ -104,7 +103,6 @@
timeout=3, sleep=1)
self.assertEqual(utils.monotonic_time.now, 3)
- @brokentest("sleep is not considered in deadline calculation")
@MonkeyPatch(utils, 'monotonic_time', FakeTime(0))
@MonkeyPatch(time, 'sleep', fake_sleep)
def testTimeoutNoTimeForSleep(self):
--
To view, visit https://gerrit.ovirt.org/46400
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7f41c6b21e3432159c13d46cfe75d1f6236cbb8c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
6 years, 5 months
Change in vdsm[master]: hsm: Pep8ize function name
by Nir Soffer
Nir Soffer has uploaded a new change for review.
Change subject: hsm: Pep8ize function name
......................................................................
hsm: Pep8ize function name
Remove unneeded _private prefix for inner function and use lowercase
name.
Change-Id: I54715971378319a8501a49cb89b24f27e50a07f4
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
---
M vdsm/storage/hsm.py
1 file changed, 3 insertions(+), 3 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/06/39306/1
diff --git a/vdsm/storage/hsm.py b/vdsm/storage/hsm.py
index a4bb858..8f75a39 100644
--- a/vdsm/storage/hsm.py
+++ b/vdsm/storage/hsm.py
@@ -2070,7 +2070,7 @@
boolean
:rtype: dict
"""
- def _isVisible(guid):
+ def is_visible(guid):
path = os.path.join('/dev/mapper', guid)
try:
st = os.stat(path)
@@ -2081,10 +2081,10 @@
else:
return (st.st_mode & stat.S_IRUSR) != 0
- visibility = [_isVisible(guid) for guid in guids]
+ visibility = [is_visible(guid) for guid in guids]
if not all(visibility):
multipath.rescan()
- visibility = [_isVisible(guid) for guid in guids]
+ visibility = [is_visible(guid) for guid in guids]
visibility = dict(zip(guids, visibility))
# After multipath.rescan, existing devices may disapper, and new
--
To view, visit https://gerrit.ovirt.org/39306
To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I54715971378319a8501a49cb89b24f27e50a07f4
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>
6 years, 5 months
Change in vdsm[master]: Live Merge: Restore watermark tracking
by alitke＠redhat.com
Adam Litke has uploaded a new change for review.
Change subject: Live Merge: Restore watermark tracking
......................................................................
Live Merge: Restore watermark tracking
Change-Id: I632f31e7795ec5d8c6f52a480116b14470c3163f
Signed-off-by: Adam Litke <alitke(a)redhat.com>
---
M vdsm/virt/vm.py
1 file changed, 108 insertions(+), 10 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/24/36924/1
diff --git a/vdsm/virt/vm.py b/vdsm/virt/vm.py
index f22610d..09080b9 100644
--- a/vdsm/virt/vm.py
+++ b/vdsm/virt/vm.py
@@ -1512,12 +1512,94 @@
with self._confLock:
self.conf['timeOffset'] = newTimeOffset
+ def _getWriteWatermarks(self):
+ def pathToVolID(drive, path):
+ for vol in drive.volumeChain:
+ if os.path.realpath(vol['path']) == os.path.realpath(path):
+ return vol['volumeID']
+ raise LookupError("Unable to find VolumeID for path '%s'", path)
+
+ volAllocMap = {}
+ statsFlags = self._libvirtBackingChainStatsFlag()
+ conn = libvirtconnection.get()
+ blkStats = conn.domainListGetStats([self._dom._dom],
+ libvirt.VIR_DOMAIN_STATS_BLOCK,
+ statsFlags)[0][1]
+ for i in xrange(0, blkStats['block.count']):
+ name = blkStats['block.%i.name' % i]
+ try:
+ drive = self._findDriveByName(name)
+ except LookupError:
+ continue
+ if not drive.blockDev or drive.format != 'cow':
+ continue
+
+ try:
+ path = blkStats['block.%i.path' % i]
+ alloc = blkStats['block.%i.allocation' % i]
+ except KeyError as e:
+ self.log.debug("Block stats are missing expected key '%s', "
+ "skipping volume", e.args[0])
+ continue
+ volID = pathToVolID(drive, path)
+ volAllocMap[volID] = alloc
+ return volAllocMap
+
+ def _getLiveMergeExtendCandidates(self):
+ # The common case is that there are no active jobs.
+ if not self.conf['_blockJobs'].values():
+ return {}
+
+ candidates = {}
+ watermarks = self._getWriteWatermarks()
+ for job in self.conf['_blockJobs'].values():
+ try:
+ drive = self._findDriveByUUIDs(job['disk'])
+ except LookupError:
+ # After an active layer merge completes the vdsm metadata will
+ # be out of sync for a brief period. If we cannot find the old
+ # disk then it's safe to skip it.
+ continue
+
+ if not drive.blockDev:
+ continue
+
+ if job['strategy'] == 'commit':
+ volumeID = job['baseVolume']
+ else:
+ self.log.debug("Unrecognized merge strategy '%s'",
+ job['strategy'])
+ continue
+ res = self.cif.irs.getVolumeInfo(drive.domainID, drive.poolID,
+ drive.imageID, volumeID)
+ if res['status']['code'] != 0:
+ self.log.error("Unable to get the info of volume %s (domain: "
+ "%s image: %s)", volumeID, drive.domainID,
+ drive.imageID)
+ continue
+ volInfo = res['info']
+
+ if volInfo['format'].lower() != 'cow':
+ continue
+
+ if volumeID in watermarks:
+ self.log.debug("Adding live merge extension candidate: "
+ "volume=%s allocation=%i", volumeID,
+ watermarks[volumeID])
+ candidates[drive.imageID] = {
+ 'alloc': watermarks[volumeID],
+ 'physical': int(volInfo['truesize']),
+ 'capacity': int(volInfo['apparentsize']),
+ 'volumeID': volumeID}
+ else:
+ self.log.warning("No watermark info available for %s",
+ volumeID)
+ return candidates
+
def _getExtendCandidates(self):
ret = []
- # FIXME: mergeCandidates should be a dictionary of candidate volumes
- # once libvirt starts reporting watermark information for all volumes.
- mergeCandidates = {}
+ mergeCandidates = self._getLiveMergeExtendCandidates()
for drive in self._devices[hwclass.DISK]:
if not drive.blockDev or drive.format != 'cow':
continue
@@ -4771,6 +4853,14 @@
jobsRet[jobID] = entry
return jobsRet
+ def _libvirtBackingChainStatsFlag(self):
+ # Since libvirt 1.2.13, the virConnectGetAllDomainStats API will return
+ # block statistics for all volumes in the chain when using a new flag.
+ try:
+ return libvirt.VIR_CONNECT_GET_ALL_DOMAINS_STATS_BACKING
+ except AttributeError:
+ return 0
+
def merge(self, driveSpec, baseVolUUID, topVolUUID, bandwidth, jobUUID):
if not caps.getLiveMergeSupport():
self.log.error("Live merge is not supported on this host")
@@ -4815,6 +4905,8 @@
if res['info']['voltype'] == 'SHARED':
self.log.error("merge: Refusing to merge into a shared volume")
return errCode['mergeErr']
+ baseSize = int(res['info']['apparentsize'])
+ baseCow = bool(res['info']['format'].lower() == 'cow')
# Indicate that we expect libvirt to maintain the relative paths of
# backing files. This is necessary to ensure that a volume chain is
@@ -4865,13 +4957,19 @@
# blockCommit will cause data to be written into the base volume.
# Perform an initial extension to ensure there is enough space to
- # copy all the required data. Normally we'd use monitoring to extend
- # the volume on-demand but internal watermark information is not being
- # reported by libvirt so we must do the full extension up front. In
- # the worst case, we'll need to extend 'base' to the same size as 'top'
- # plus a bit more to accomodate additional writes to 'top' during the
- # live merge operation.
- self.extendDriveVolume(drive, baseVolUUID, topSize)
+ # copy all the required data. If libvirt supports monitoring of
+ # backing chain volumes, just extend by one chunk now and monitor
+ # during the rest of the operation. Otherwise, extend now to
+ # accomodate the worst case scenario: no intersection between the
+ # allocated blocks in the base volume and the top volume.
+ if drive.blockDev and baseCow:
+ if self._libvirtBackingChainStatsFlag():
+ self.extendDrivesIfNeeded()
+ else:
+ extendSize = baseSize + topSize
+ self.log.debug("Preemptively extending volume %s with size %i"
+ "(job: %s)", baseVolUUID, extendSize, jobUUID)
+ self.extendDriveVolume(drive, baseVolUUID, extendCurSize)
# Trigger the collection of stats before returning so that callers
# of getVmStats after this returns will see the new job
--
To view, visit http://gerrit.ovirt.org/36924
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I632f31e7795ec5d8c6f52a480116b14470c3163f
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Adam Litke <alitke(a)redhat.com>
6 years, 5 months