Francesco Romani has posted comments on this change.
Change subject: vm: sync device update during creation of a vm
......................................................................
Patch Set 2:
We have quite some things mixed here.
Most likely (but not 100% sure) this hasn't happened before because
the XML marshaller (as Piotr found) used a different approach when iterating on python dictionaries, which kept things (quite) safe.
This changed in the 3.5 timeframe due mostly to json-rpc and live-merge. Both of those changes are not to blame, their interaction with the existing codebase however broke up things.
Simply to revert to the old method is feasible as short term solution, not as long term one.
The deeper problem is how badly Vm is intertwined. To fix this will cost time - first approach here: http://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:st…
As first-aid patch I'll see if it is possible to revert to the old path (items() vs iteritems()), but this solution is not very good either.
--
To view, visit http://gerrit.ovirt.org/34599
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I477551500ccc2297eb0c05d6562710bc420363a5
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Michal Skrivanek <michal.skrivanek(a)redhat.com>
Gerrit-Reviewer: Michal Skrivanek <mskrivan(a)redhat.com>
Gerrit-Reviewer: Oved Ourfali <oourfali(a)redhat.com>
Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
Saggi Mizrahi has posted comments on this change.
Change subject: vm: sync device update during creation of a vm
......................................................................
Patch Set 2: Code-Review+1
--
To view, visit http://gerrit.ovirt.org/34599
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: I477551500ccc2297eb0c05d6562710bc420363a5
Gerrit-PatchSet: 2
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Piotr Kliczewski <piotr.kliczewski(a)gmail.com>
Gerrit-Reviewer: Francesco Romani <fromani(a)redhat.com>
Gerrit-Reviewer: Michal Skrivanek <michal.skrivanek(a)redhat.com>
Gerrit-Reviewer: Michal Skrivanek <mskrivan(a)redhat.com>
Gerrit-Reviewer: Oved Ourfali <oourfali(a)redhat.com>
Gerrit-Reviewer: Saggi Mizrahi <smizrahi(a)redhat.com>
Gerrit-Reviewer: automation(a)ovirt.org
Gerrit-Reviewer: oVirt Jenkins CI Server
Gerrit-HasComments: No
Hello Antoni Segura Puimedon, Dan Kenigsberg,
I'd like you to do a code review. Please visit
http://gerrit.ovirt.org/34617
to review the following change.
Change subject: caps: Additional ppc64 hardware information
......................................................................
caps: Additional ppc64 hardware information
Includes extra information (manufacturer and product name) about ppc64
hosts in the getVdsHardwareInfo command. This extra information is
obtained from the device tree and skipped in case it is missing.
Change-Id: I8f67a830740b64bc246f680f2c7a18a4293f4cc2
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1149262
Signed-off-by: Vitor de Lima <vdelima(a)redhat.com>
Reviewed-on: http://gerrit.ovirt.org/33857
Reviewed-by: Antoni Segura Puimedon <asegurap(a)redhat.com>
Reviewed-by: Dan Kenigsberg <danken(a)redhat.com>
---
M vdsm/ppc64HardwareInfo.py
1 file changed, 16 insertions(+), 9 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/17/34617/1
diff --git a/vdsm/ppc64HardwareInfo.py b/vdsm/ppc64HardwareInfo.py
index 1a4b30e..029eaa4 100644
--- a/vdsm/ppc64HardwareInfo.py
+++ b/vdsm/ppc64HardwareInfo.py
@@ -21,14 +21,21 @@
import os
+def _getFromDeviceTree(treeProperty):
+ path = '/proc/device-tree/%s' % treeProperty
+ if os.path.exists(path):
+ with open(path) as f:
+ value = f.readline().rstrip('\0').replace(',', '')
+ return value
+ else:
+ return 'unavailable'
+
+
@utils.memoized
def getHardwareInfoStructure():
- infoStructure = {'systemProductName': 'unavailable',
- 'systemSerialNumber': 'unavailable',
+ infoStructure = {'systemSerialNumber': 'unavailable',
'systemFamily': 'unavailable',
- 'systemVersion': 'unavailable',
- 'systemUUID': 'unavailable',
- 'systemManufacturer': 'unavailable'}
+ 'systemVersion': 'unavailable'}
for line in file('/proc/cpuinfo'):
if line.strip() == '':
@@ -42,11 +49,11 @@
elif key == 'machine':
infoStructure['systemVersion'] = value
- if os.path.exists('/proc/device-tree/system-id'):
- with open('/proc/device-tree/system-id') as f:
- vdsmId = f.readline().rstrip('\0').replace(',', '')
+ infoStructure['systemUUID'] = _getFromDeviceTree('system-id')
- infoStructure['systemUUID'] = vdsmId
+ infoStructure['systemProductName'] = _getFromDeviceTree('model-name')
+
+ infoStructure['systemManufacturer'] = _getFromDeviceTree('vendor')
return infoStructure
--
To view, visit http://gerrit.ovirt.org/34617
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8f67a830740b64bc246f680f2c7a18a4293f4cc2
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-3.5
Gerrit-Owner: Vitor de Lima <vdelima(a)redhat.com>
Gerrit-Reviewer: Antoni Segura Puimedon <asegurap(a)redhat.com>
Gerrit-Reviewer: Dan Kenigsberg <danken(a)redhat.com>
Allon Mureinik has uploaded a new change for review.
Change subject: volume: Log the correct error when creating a volume fails
......................................................................
volume: Log the correct error when creating a volume fails
When a volume creation failed because of CannotCreateLogicalVolume
exception, we used to lie and log "volume already exists". This log
confused and wasted many developers hours. Now we log the exception
value instead.
Change-Id: I603b055658950dae5ccc3806b8b7a9e53762c5ef
Bug-Url: https://bugzilla.redhat.com/1143830
Relates-To: https://bugzilla.redhat.com/1142710
Signed-off-by: Nir Soffer <nsoffer(a)redhat.com>
Reviewed-on: http://gerrit.ovirt.org/33301
Reviewed-by: Yoav Kleinberger <ykleinbe(a)redhat.com>
Reviewed-by: Dan Kenigsberg <danken(a)redhat.com>
(cherry picked from commit 3e17f9828576f16e4ef95f805c2e7ce27b63d812)
---
M vdsm/storage/volume.py
1 file changed, 1 insertion(+), 2 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/39/34639/1
diff --git a/vdsm/storage/volume.py b/vdsm/storage/volume.py
index 13bd256..75fa1c4 100644
--- a/vdsm/storage/volume.py
+++ b/vdsm/storage/volume.py
@@ -434,8 +434,7 @@
preallocate, volParent, srcImgUUID,
srcVolUUID, volPath)
except (se.VolumeAlreadyExists, se.CannotCreateLogicalVolume) as e:
- cls.log.error("Failed to create volume: %s, volume already "
- "exists", volPath)
+ cls.log.error("Failed to create volume %s: %s", volPath, e)
vars.task.popRecovery()
raise e
# When the volume format is raw what the guest sees is the apparent
--
To view, visit http://gerrit.ovirt.org/34639
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I603b055658950dae5ccc3806b8b7a9e53762c5ef
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-3.5
Gerrit-Owner: Allon Mureinik <amureini(a)redhat.com>
Gerrit-Reviewer: Nir Soffer <nsoffer(a)redhat.com>
Petr Horáček has uploaded a new change for review.
Change subject: netinfo:nicSpeed(): fix nicSpeed condition
......................................................................
netinfo:nicSpeed(): fix nicSpeed condition
In `if s not in (2 ** 16 - 1, 2 ** 32 - 1) or s > 0` first part doesn't
make sense with OR. Changed to AND.
I changed conditions to make the function more readable and created unit
test, to be sure that function returns correct values.
Change-Id: I34cfca16909f9695441e26d3ddd508e7a4210c12
Bug-Url: https://bugzilla.redhat.com/show_bug.cgi?id=1157224
Signed-off-by: Petr Horáček <phoracek(a)redhat.com>
---
M lib/vdsm/netinfo.py
M tests/netinfoTests.py
2 files changed, 28 insertions(+), 13 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/01/34701/1
diff --git a/lib/vdsm/netinfo.py b/lib/vdsm/netinfo.py
index 174e5e4..74c9f6c 100644
--- a/lib/vdsm/netinfo.py
+++ b/lib/vdsm/netinfo.py
@@ -246,19 +246,17 @@
def nicSpeed(nicName):
- """Returns the nic speed if it is a legal value and nicName refers to a
- nic, 0 otherwise."""
+ """Returns the nic speed if it is a legal value, nicName refers to a
+ nic and nic is UP, 0 otherwise."""
try:
- # if the device is not up we must report 0
- if operstate(nicName) != OPERSTATE_UP:
- return 0
- with open('/sys/class/net/%s/speed' % nicName) as speedFile:
- s = int(speedFile.read())
- # the device may have been disabled/downed after checking
- # so we validate the return value as sysfs may return
- # special values to indicate the device is down/disabled
- if s not in (2 ** 16 - 1, 2 ** 32 - 1) or s > 0:
- return s
+ if operstate(nicName) == OPERSTATE_UP:
+ with open('/sys/class/net/%s/speed' % nicName) as speedFile:
+ s = int(speedFile.read())
+ # the device may have been disabled/downed after checking
+ # so we validate the return value as sysfs may return
+ # special values to indicate the device is down/disabled
+ if s not in (2 ** 16 - 1, 2 ** 32 - 1) and s > 0:
+ return s
except IOError as ose:
if ose.errno == errno.EINVAL:
return _ibHackedSpeed(nicName)
diff --git a/tests/netinfoTests.py b/tests/netinfoTests.py
index 8900b24..62c46b6 100644
--- a/tests/netinfoTests.py
+++ b/tests/netinfoTests.py
@@ -18,9 +18,11 @@
#
# Refer to the README and COPYING files for full details of the license
#
+import __builtin__
import os
from datetime import datetime
from functools import partial
+import io
import time
import ethtool
@@ -29,7 +31,7 @@
from vdsm import netconfpersistence
from vdsm import netinfo
from vdsm.netinfo import (getBootProtocol, getDhclientIfaces, BONDING_MASTERS,
- BONDING_OPT, _getBondingOptions)
+ BONDING_OPT, _getBondingOptions, OPERSTATE_UP)
from vdsm.tool.dump_bonding_defaults import _random_iface_name
from functional import dummy, veth
@@ -93,6 +95,21 @@
for s, addr in zip(inputs, ip):
self.assertEqual(addr, netinfo.ipv6StrToAddress(s))
+ def testValidNicSpeed(self):
+ values = ((0, OPERSTATE_UP, 0),
+ (-10, OPERSTATE_UP, 0),
+ (2 ** 16 - 1, OPERSTATE_UP, 0),
+ (2 ** 32 - 1, OPERSTATE_UP, 0)
+ (123, OPERSTATE_UP, 123),
+ (123, 'unknown', 0))
+
+ for passed, operstate, expected in values:
+ with MonkeyPatchScope([(__builtin__, 'open',
+ lambda x: io.BytesIO(str(passed))),
+ (netinfo, 'operstate',
+ lambda x: operstate)]):
+ self.assertEqual(netinfo.nicSpeed('fake_nic'), expected)
+
@MonkeyPatch(ipwrapper.Link, '_detectType',
partial(_fakeTypeDetection, ipwrapper.Link))
@MonkeyPatch(netinfo, 'networks', lambda: {'fake': {'bridged': True}})
--
To view, visit http://gerrit.ovirt.org/34701
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I34cfca16909f9695441e26d3ddd508e7a4210c12
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: ovirt-3.5
Gerrit-Owner: Petr Horáček <phoracek(a)redhat.com>
Petr Horáček has uploaded a new change for review.
Change subject: persistence: fails while removing non-existing network
......................................................................
persistence: fails while removing non-existing network
vdsm-restore-netconfig:unified_restoration() removes current networks
(listed in runningConfig) before it tries to restore persisted ones.
Problem occurs when current network (listed in runningConfig) does not
exist in the system (for example when we manualy delete its nic). In
this case, setupNetwork() returns exception (network does not exist)
and restoration can not continue.
I added new condition to setupNetworks(), it checks if network is in
runningConfig in case it was not listed in netinfo and libvirt_nets. If
so, invalid network is removed from runningConfig.
Change-Id: I0c507626705d7ead84db2f3aa15e4032f9558d12
Signed-off-by: Petr Horáček <phoracek(a)redhat.com>
---
M vdsm/network/api.py
M vdsm/vdsm-restore-net-config
2 files changed, 11 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/95/33995/1
diff --git a/vdsm/network/api.py b/vdsm/network/api.py
index e70d954..dc372a7 100755
--- a/vdsm/network/api.py
+++ b/vdsm/network/api.py
@@ -719,6 +719,16 @@
del networks[network]
del libvirt_nets[network]
_netinfo.updateDevices()
+ elif network in configurator.runningConfig.networks:
+ # If the network was not in _netinfo or libvirt_nets but is in
+ # the networks returned by running configurator, we might
+ # remove it from runningConfig.
+ logger.debug('Removing non-existing network %r from '
+ 'runningConfig', network)
+ configurator.runningConfig.removeNetwork(network)
+ if 'remove' in networkAttrs:
+ del networks[network]
+ _netinfo.updateDevices()
elif 'remove' in networkAttrs:
raise ConfigNetworkError(ne.ERR_BAD_BRIDGE, "Cannot delete "
"network %r: It doesn't exist in the "
diff --git a/vdsm/vdsm-restore-net-config b/vdsm/vdsm-restore-net-config
index f9ca589..6c9681a 100755
--- a/vdsm/vdsm-restore-net-config
+++ b/vdsm/vdsm-restore-net-config
@@ -31,6 +31,7 @@
# Unified persistence restoration
from network.api import setupNetworks
from network import configurators
+from network import errors as ne
from vdsm.netconfpersistence import RunningConfig, PersistentConfig
import pkgutil
--
To view, visit http://gerrit.ovirt.org/33995
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I0c507626705d7ead84db2f3aa15e4032f9558d12
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Petr Horáček <phoracek(a)redhat.com>
Yoav Kleinberger has uploaded a new change for review.
Change subject: vdsm: documentation for new storage functional tests
......................................................................
vdsm: documentation for new storage functional tests
Added a markdown README file that explains the design and usage of the
new functional tests. This will help future developers who wish to
extend the tests.
Change-Id: I7858c5ce6e17f2dbf11bf7ec6d041e4dbef0457c
Signed-off-by: Yoav Kleinberger <ykleinbe(a)redhat.com>
---
A tests/functional/testlib/README.md
1 file changed, 144 insertions(+), 0 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/78/34478/1
diff --git a/tests/functional/testlib/README.md b/tests/functional/testlib/README.md
new file mode 100644
index 0000000..3a2847c
--- /dev/null
+++ b/tests/functional/testlib/README.md
@@ -0,0 +1,144 @@
+# New Functional Tests for VDSM
+
+By Yoav Kleinberger (ykleinbe(a)redhat.com ; haarcuba(a)gmail.com)
+
+## Introduction
+This document describes new functional tests for VDSM storage,
+intended to replace the old tests at `tests/functional/storageTests.py`. The old
+tests need replacing, because they
+
+1. Are hard to understand.
+2. Do not really verify VDSM behaviour.
+
+The new tests *do* verify VDSM behaviour, and are, so I hope, easier to understand.
+
+## Running the Tests
+
+First thing's first: to run the new functional tests, change into the `tests` directory and then
+
+ $ ./run_functional_storage_tests.sh
+
+You can use
+
+ $ ./run_functional_storage_tests.sh --verbose
+
+to see verbose logging.
+
+## The Basic Test: Create Volume
+
+There is currently one basic flow we test
+
+1. Creation of a Storage Domain
+2. Creation of a Virtual Disk on that Domain
+
+We want to test this flow for various storage backends, e.g. for NFS, iSCSI and
+Fibre Channel. This requires that the test framework support different storage
+backends. The problem is solved at the price of a non-trivial design of test
+"Storage Contexts", which will now be explained. The test itself resides in
+`tests/functional/basicStorageTest.py`.
+
+## Storage Contexts
+
+### iSCSI example
+
+As explained above, our create-volume test needs to run in different contexts:
+an iSCSI context, an NFS context, and so on. These contexts are represented by
+modules residing in
+
+ functional/testlib/storagecontexts
+
+To take a concrete example, let's look at the iSCSI context, located at
+`functional/testlib/storagecontexts/iscsi.py`. This file contains two classes:
+`ISCSI` and `Verify`. The usage is as follows:
+
+ with iscsi.ISCSI() as (vdsm, verify):
+ storageServerID = vdsm.connectStorageServer()
+ verify.storageServerConnected()
+
+ domainID = vdsm.createStorageDomain()
+ verify.storageDomainCreated(domainID)
+
+ poolID = vdsm.createStoragePool()
+ verify.storagePoolCreated(poolID, masterDomainID=domainID)
+ ...
+
+Note the structure of action/verification. The `vdsm` object lets you give VDSM
+a command [just like Engine does]. The `verify` object checks for observable
+changes induced by the command, e.g. the existence of logical volumes, etc.
+
+The ISCSI storage context makes sure that the `(vdsm, verify)` pair acts on an
+actual iSCSI storage set up for the purpose of this test. The ISCSI object will
+set up an iSCSI storage (on the host where the test runs), and will clean it up
+when the test is finished. In the example above, `vdsm.connectStorageServer()`
+tells vdsm to connect to the iSCSI storage, and the `verify` object then checks
+inside the `sysfs` interface exposed by the linux kernel that the connection
+actually exists.
+
+### Under the Hood of the iSCSI Storage Context
+
+As a further clarification of the workings of these tests, let's look at what
+happens when the `vdsm.connectStorageServer()` call above is executed. If you
+look inside, you'll find that this gets translated into calling the VDSM API
+with something like
+
+
+ connectStorageServer( storage.sd.ISCSI_DOMAIN,
+ '00000000-0000-0000-0000-000000000000',
+ [{ 'connection': '127.0.0.1',
+ 'iqn': 'iqn.1970-01.functional.test:3243',
+ 'user': '',
+ 'tpgt': '1',
+ 'password': '',
+ 'id': '00000000-0000-0000-0000-000000000000',
+ 'port': '3260' }])
+
+Thus, the ISCSI storage context object translates the essence of
+`connectStorageServer` to proper iSCSI terms.
+
+The same goes for the `verify`
+object. The `verify.storageServerConnected` call above will result in globbing
+files under `/sys/devices/platform/host*/session*/iscsi_session/*/targetname`
+and looking for the IQN inside.
+
+### Other Storage Contexts
+
+As demostrated above, the `ISCSI` class takes care of setting up iSCSI storage,
+and cleaning up afterwards. Similarly, if you want to test the same operations
+on NFS, you would use an NFS storage context (to be found in
+`functional/testlib/storagecontexts/nfs.py`) in exactly the same way:
+
+ with nfs.NFS() as (vdsm, verify):
+ storageServerID = vdsm.connectStorageServer()
+ verify.storageServerConnected()
+ ...
+
+This time, the `NFS` object will set up NFS storage, use the proper NFS
+parameters for calling VDSM and for verifying its behaviour, and remove it
+afterwards.
+
+Currently, we have iSCSI, LocalFS and NFS support. This may be extented by
+writing new storage contexts, e.g. for Fibre Channel.
+
+## VDSM Service Shutdown and Bringup
+
+Between tests, we shutdown the VDSM service, clean up the `/rhev/data-center`
+directory, and restart the service. The class in charge of this is
+`ControlVDSM`, located in `tests/functional/testlib/controlvdsm.py`. It also
+checks that VDSM is listening for commands before returning.
+
+## Randomness
+
+The storage contexts include a fair amount of ranomizing, e.g. the `iqn` iSCSI
+parameter is different every time the test is run. This helps us avoid cases in
+which some leftovers from manual checks or previous test runs mess with our
+test results. I recommend continuing this practice.
+
+## Known Issues
+
+1. Currently, cleanup after the tests is not perfect: if you work with the
+ tests you'll see the problems. The tests work well enough that it's not an
+ immediate concern, but there is room for improvement. At this stage I wanted
+ to get the tests into wider acceptance and worry about improving this later.
+2. The storage context classes are obviously quite tailored to the
+ create volume test, and will probably require some changes when adding
+ future tests.
--
To view, visit http://gerrit.ovirt.org/34478
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7858c5ce6e17f2dbf11bf7ec6d041e4dbef0457c
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Yoav Kleinberger <ykleinbe(a)redhat.com>