vdsm-patches October 2016

vdsm-patches@lists.fedorahosted.org

33 participants
468 discussions

Change in vdsm[master]: guest-lvs: Add lvm bootstrap tests
by Nir Soffer 01 Oct '16

01 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: guest-lvs: Add lvm bootstrap tests ...................................................................... guest-lvs: Add lvm bootstrap tests Add fake_env for creating lvm tests using loop devices instead of shared storage, and start lvm.bootstrap tests. The first test is verifying that unused ovirt lvs are deactivated. Change-Id: I314544930aa28cf1d929a23842a7e011d461706d Bug-Url: https://bugzilla.redhat.com/1374545 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/storage_lvm_test.py 1 file changed, 86 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/67/64367/1 diff --git a/tests/storage_lvm_test.py b/tests/storage_lvm_test.py index 6792266..67ed799 100644 --- a/tests/storage_lvm_test.py +++ b/tests/storage_lvm_test.py @@ -19,9 +19,25 @@ # Refer to the README and COPYING files for full details of the license # -from testlib import VdsmTestCase as TestCaseBase +import collections +import os +import logging -import storage.lvm as lvm +from contextlib import contextmanager + +from vdsm import commands +from vdsm import cmdutils + +from storage import blockSD +from storage import lvm + +import loop +from monkeypatch import MonkeyPatchScope +from testlib import VdsmTestCase as TestCaseBase +from testlib import namedTemporaryDir +from testValidation import ValidateRunningAsRoot + +log = logging.getLogger("test") class LvmTests(TestCaseBase): @@ -37,3 +53,71 @@ "\\\\x22\\\\x28|\', \'r|.*|\' ]" ) self.assertEqual(expectedFilter, filter) + + +class TestBootstrap(TestCaseBase): + + @ValidateRunningAsRoot + def test_deactivate_unused_ovirt_lvs(self): + with fake_env() as env: + log.debug("Creating ovirt lvs") + run("pvcreate", "-ff", env.device) + run("vgcreate", "ovirt-vg", env.device) + run("vgchange", "--addtag", blockSD.STORAGE_DOMAIN_TAG) + run("lvcreate", "-n", "ovirt-lv-1", "-L", "128m", "ovirt-vg") + run("lvcreate", "-n", "ovirt-lv-2", "-L", "128m", "ovirt-vg") + try: + lvm.bootstrap() + # ovirt-lv must be inactive + self.check_no_active_lvs("ovirt-vg") + finally: + run("lvchange", "-an", "ovirt-vg") + + # TODO: do not modify vgs without STORAGE_DOMAIN_TAG + + def check_no_active_lvs(self, vgname): + out = run("lvs", "--noheadings", "-o", "name", + "--select", "lv_active=active", vgname)[0] + lvs = [line.strip() for line in out.splitlines()] + self.assertEqual(lvs, []) + + +Env = collections.namedtuple("Env", "tmpdir, device") + + +@contextmanager +def fake_env(): + """ + Create test environment for lvm tests. + """ + with namedTemporaryDir() as tmpdir: + backing_file = os.path.join(tmpdir, "backing_file") + truncate(backing_file, 1024**3) + with loop.Device(backing_file) as device: + log.debug("Using loop device %s", device.path) + with MonkeyPatchScope([ + # Created during bootstrap. + (lvm, "VDSM_LVM_SYSTEM_DIR", os.path.join(tmpdir, "lvm")), + # Saved each time conf is changed. + (lvm, "VDSM_LVM_CONF", os.path.join(tmpdir, "lvm", "lvm.conf")), + # Allow detection of test device, otherwise only + # /dev/mapper/guid devices are detected. + (lvm, "USER_DEV_LIST", [device.path]), + ]): + try: + yield Env(tmpdir, device.path) + finally: + # Update lvmetad to keep the environment sane + run("pvscan", "--cache") + + +def run(*cmd): + rc, out, err = commands.execCmd(cmd, raw=True) + if rc != 0: + raise cmdutils.Error(cmd, rc, out, err) + return out, err + + +def truncate(path, size): + with open(path, "w") as f: + f.truncate(size) -- To view, visit https://gerrit.ovirt.org/64367 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I314544930aa28cf1d929a23842a7e011d461706d Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

2 17

Change in vdsm[master]: tests: Add loopback module
by Nir Soffer 01 Oct '16

01 Oct '16

Nir Soffer has posted comments on this change. Change subject: tests: Add loopback module ...................................................................... Patch Set 7: Verified+1 This version address the issue I mention in the previous version. -- To view, visit https://gerrit.ovirt.org/64329 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I37184887930ad9ec1234036fdcdeb6cee8ccac42 Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Freddy Rolland <frolland(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: tests: Rename lvmTests to new naming convention
by Nir Soffer 01 Oct '16

01 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: tests: Rename lvmTests to new naming convention ...................................................................... tests: Rename lvmTests to new naming convention Change-Id: Ia266cfde3308f2a842a4fe3be4b6e2bbf2265386 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M tests/Makefile.am R tests/storage_lvm_test.py 2 files changed, 2 insertions(+), 2 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/28/64328/1 diff --git a/tests/Makefile.am b/tests/Makefile.am index 38ab95c..ba958de 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -78,7 +78,6 @@ jobsTests.py \ libvirtconnectionTests.py \ logutils_test.py \ - lvmTests.py \ miscTests.py \ mkimageTests.py \ modprobe.py \ @@ -125,6 +124,7 @@ storage_directio_test.py \ storage_guarded_test.py \ storage_hsm_test.py \ + storage_lvm_test.py \ storage_monitor_test.py \ storage_rwlock_test.py \ storage_sdm_copy_data_test.py \ @@ -184,7 +184,6 @@ hoststatsTests.py \ imagetickets_test.py \ iscsiTests.py \ - lvmTests.py \ miscTests.py \ mkimageTests.py \ momPolicyTests.py \ @@ -222,6 +221,7 @@ storageMailboxTests.py \ storage_guarded_test.py \ storage_hsm_test.py \ + storage_lvm_test.py \ storage_monitor_test.py \ storageServerTests.py \ storage_rwlock_test.py \ diff --git a/tests/lvmTests.py b/tests/storage_lvm_test.py similarity index 100% rename from tests/lvmTests.py rename to tests/storage_lvm_test.py -- To view, visit https://gerrit.ovirt.org/64328 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia266cfde3308f2a842a4fe3be4b6e2bbf2265386 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

2 11

Change in vdsm[master]: guest-lvs: Deactivate guest lvs during bootstrap
by Nir Soffer 01 Oct '16

01 Oct '16

Nir Soffer has uploaded a new change for review. Change subject: guest-lvs: Deactivate guest lvs during bootstrap ...................................................................... guest-lvs: Deactivate guest lvs during bootstrap Add a new pass during deactivation, detecting open lvs used by other lvs, and deactivate the guest lvs using them. After deactivating the guest lvs the host lvs are invalidated. If the lvs are still open in the next pass, they are likely used by a vm. We don't have enough information at this point so we skip them. If the lvs are closed, they will be deactivated. Change-Id: I0b0eed7a61ff11593700c2c5ef0e7d5d4f8be784 Bug-Url: https://bugzilla.redhat.com/1374545 Signed-off-by: Nir Soffer <nsoffer(a)redhat.com> --- M lib/vdsm/storage/devicemapper.py M vdsm/storage/lvm.py 2 files changed, 33 insertions(+), 4 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/68/64368/1 diff --git a/lib/vdsm/storage/devicemapper.py b/lib/vdsm/storage/devicemapper.py index 90ba81a..c5d061c 100644 --- a/lib/vdsm/storage/devicemapper.py +++ b/lib/vdsm/storage/devicemapper.py @@ -117,7 +117,10 @@ def removeMapping(deviceName): - return getProxy().removeDeviceMapping(deviceName) + if os.geteuid() == 0: + return _removeMapping(deviceName) + else: + return getProxy().removeDeviceMapping(deviceName) def _removeMapping(deviceName): diff --git a/vdsm/storage/lvm.py b/vdsm/storage/lvm.py index 68d3d98..c501105 100644 --- a/vdsm/storage/lvm.py +++ b/vdsm/storage/lvm.py @@ -654,12 +654,17 @@ refresh = [] for lv in _lvminfo.getLv(vg.name): + if lv.opened and lv.name not in refreshlvs: + _remove_lv_holders(vg.name, lv.name) + + for lv in _lvminfo.getLv(vg.name): if lv.active: if lv.name in refreshlvs: refresh.append(lv.name) elif lv.opened: - log.debug("Skipping open lv: vg=%s lv=%s", vg.name, - lv.name) + # This LV may be used by a vm, we don't have enough + # information about that at this point. + log.debug("Skipping open lv %s/%s", vg.name, lv.name) else: deactivate.append(lv.name) @@ -680,6 +685,27 @@ except se.LogicalVolumeRefreshError: log.error("Error refreshing lvs: vg=%s lvs=%s", vg.name, refresh) + + +def _remove_lv_holders(vg_name, lv_name): + """ + This may be needed when a raw volume is used as a pv in the guest without + creating partition table. LVM scans active lvs and activate guest lvs on + the host. Active guest lvs allow unwanted access to guest data on the + host, and prevent deactivation of the host lvs. + + See https://bugzilla.redhat.com/1374545 + """ + dm_name = getLvDmName(vg_name, lv_name) + holders = devicemapper.getHolders(dm_name) + if holders: + devices = [devicemapper.getDevName(dev) for dev in holders] + log.info("LV %s/%s is used by %s, removing mappings", + vg_name, lv_name, devices) + for dev in devices: + devicemapper.removeMapping(dev) + # This invalidates lv.attr, force reload on next access. + _lvminfo._invalidatelvs(vg_name, [lv_name]) def invalidateCache(): @@ -742,7 +768,7 @@ def getLvDmName(vgName, lvName): - return "%s-%s" % (vgName.replace("-", "--"), lvName) + return "%s-%s" % (vgName.replace("-", "--"), lvName.replace("-", "--")) def removeVgMapping(vgName): -- To view, visit https://gerrit.ovirt.org/64368 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I0b0eed7a61ff11593700c2c5ef0e7d5d4f8be784 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com>

2 6

Change in vdsm[master]: guest-lvs: Add failing test for guest lvs
by automation＠ovirt.org 01 Oct '16

01 Oct '16

gerrit-hooks has posted comments on this change. Change subject: guest-lvs: Add failing test for guest lvs ...................................................................... Patch Set 11: * #1374545::Update tracker: OK * Check Bug-Url::OK * Check Public Bug::#1374545::OK, public bug * Check Product::#1374545::OK, Correct product Red Hat Enterprise Virtualization Manager * Check TM::SKIP, not in a monitored branch (ovirt-3.6 ovirt-4.0) * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/64330 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I80d37278225262bc5692e00aed15654e84119590 Gerrit-PatchSet: 11 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Freddy Rolland <frolland(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: tests: Add loopback module
by automation＠ovirt.org 01 Oct '16

01 Oct '16

gerrit-hooks has posted comments on this change. Change subject: tests: Add loopback module ...................................................................... Patch Set 7: * #1374545::Update tracker: OK * Check Bug-Url::OK * Check Public Bug::#1374545::OK, public bug * Check Product::#1374545::OK, Correct product Red Hat Enterprise Virtualization Manager * Check TM::SKIP, not in a monitored branch (ovirt-3.6 ovirt-4.0) * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/64329 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I37184887930ad9ec1234036fdcdeb6cee8ccac42 Gerrit-PatchSet: 7 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Freddy Rolland <frolland(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: guest-lvs: Add failing test for guest lvs
by automation＠ovirt.org 01 Oct '16

01 Oct '16

gerrit-hooks has posted comments on this change. Change subject: guest-lvs: Add failing test for guest lvs ...................................................................... Patch Set 10: * #1374545::Update tracker: OK * Check Bug-Url::OK * Check Public Bug::#1374545::OK, public bug * Check Product::#1374545::OK, Correct product Red Hat Enterprise Virtualization Manager * Check TM::SKIP, not in a monitored branch (ovirt-3.6 ovirt-4.0) * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/64330 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I80d37278225262bc5692e00aed15654e84119590 Gerrit-PatchSet: 10 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Freddy Rolland <frolland(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Change in vdsm[master]: tests: Add loopback module
by automation＠ovirt.org 01 Oct '16

01 Oct '16

gerrit-hooks has posted comments on this change. Change subject: tests: Add loopback module ...................................................................... Patch Set 6: * #1374545::Update tracker: OK * Check Bug-Url::OK * Check Public Bug::#1374545::OK, public bug * Check Product::#1374545::OK, Correct product Red Hat Enterprise Virtualization Manager * Check TM::SKIP, not in a monitored branch (ovirt-3.6 ovirt-4.0) * Check merged to previous::IGNORE, Not in stable branch (['ovirt-3.6', 'ovirt-4.0']) -- To view, visit https://gerrit.ovirt.org/64329 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: I37184887930ad9ec1234036fdcdeb6cee8ccac42 Gerrit-PatchSet: 6 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: Adam Litke <alitke(a)redhat.com> Gerrit-Reviewer: Ala Hino <ahino(a)redhat.com> Gerrit-Reviewer: Allon Mureinik <amureini(a)redhat.com> Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com> Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com> Gerrit-Reviewer: Freddy Rolland <frolland(a)redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com> Gerrit-Reviewer: gerrit-hooks <automation(a)ovirt.org> Gerrit-HasComments: No

1 0

Jump to page:

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

vdsm-patches October 2016