From: Christos Sfakianakis csfakian@redhat.com
Hello,
this patch set includes changes required for the recipe set to be compatible with commits d62821 to 7bde5c inclusive, which introduce a new recipe class hierarchy and inheritance model. It also addresses py3 specific-issues as well as some of the framework-specific bugs revealed during testing. Finally, it adds info getters to the OvsBridgeDevice class, useful for the description of the test configuration.
For SimplePerfRecipe, ping endpoints are discarded.
For the multipoint ping tests, there is logic applied to predict the result (pass/fail) where this is handy (e.g vlan_id comparison for vlans), but this preciction is hardcoded in some of the recipes for which more complex network path analysis would be needed.
No getters are added to the MacsecDevice class as, in the relevant recipe, the devices are created during the apply_sub_config stage and are thus not visible to the description stage.
Christos
Christos Sfakianakis (8): lnst.Recipes.ENRT.ConfigMixins: specialize the h/w device list lnst.RecipeCommon.Perf.Measurements.IperfFlowMeasurement: modify cpupin check lnst.Devices.VxlanDevice: handle missing realdev lnst.Controller.MessageDispatcher: orig kwargs to DeviceRef lnst.Tests.PacketAssert: decode bytes to str for py3 lnst.Recipes.ENRT.XfrmTools: adopt the division operator to py3 lnst.Devices.OvsBridgeDevice: add config getters lnst.Recipes.ENRT: rework recipes to adopt to 334e2c..7bde5c
lnst/Controller/MessageDispatcher.py | 1 + lnst/Devices/OvsBridgeDevice.py | 111 +++++++ lnst/Devices/VxlanDevice.py | 8 +- .../Perf/Measurements/IperfFlowMeasurement.py | 4 +- lnst/Recipes/ENRT/BondRecipe.py | 137 +++++--- .../ENRT/ConfigMixins/BaseHWConfigMixin.py | 8 +- .../ConfigMixins/CoalescingHWConfigMixin.py | 6 + .../ConfigMixins/DevInterruptHWConfigMixin.py | 8 +- .../ENRT/ConfigMixins/MTUHWConfigMixin.py | 6 +- .../ParallelStreamQDiscHWConfigMixin.py | 6 +- lnst/Recipes/ENRT/DoubleBondRecipe.py | 140 +++++--- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 159 ++++++---- lnst/Recipes/ENRT/IpsecEspAeadRecipe.py | 280 ++++++++-------- lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py | 297 ++++++++--------- lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py | 130 +++++--- lnst/Recipes/ENRT/PingFloodRecipe.py | 21 +- .../ENRT/ShortLivedConnectionsRecipe.py | 87 ++--- lnst/Recipes/ENRT/SimpleMacsecRecipe.py | 252 +++++++-------- lnst/Recipes/ENRT/SimplePerfRecipe.py | 42 +-- lnst/Recipes/ENRT/TeamRecipe.py | 134 +++++--- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 162 ++++++---- lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py | 230 +++++++++++--- .../VirtualBridgeVlanInGuestMirroredRecipe.py | 191 ++++++----- .../ENRT/VirtualBridgeVlanInGuestRecipe.py | 167 ++++++---- .../VirtualBridgeVlanInHostMirroredRecipe.py | 178 +++++++---- .../ENRT/VirtualBridgeVlanInHostRecipe.py | 155 ++++++--- .../ENRT/VirtualBridgeVlansOverBondRecipe.py | 298 +++++++++++++----- ...rtualOvsBridgeVlanInGuestMirroredRecipe.py | 193 +++++++----- .../ENRT/VirtualOvsBridgeVlanInGuestRecipe.py | 164 ++++++---- ...irtualOvsBridgeVlanInHostMirroredRecipe.py | 155 +++++---- .../ENRT/VirtualOvsBridgeVlanInHostRecipe.py | 144 ++++++--- .../VirtualOvsBridgeVlansOverBondRecipe.py | 277 ++++++++++------ lnst/Recipes/ENRT/VlansOverBondRecipe.py | 267 +++++++++++----- lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 269 +++++++++++----- lnst/Recipes/ENRT/VlansRecipe.py | 230 ++++++++++---- lnst/Recipes/ENRT/VxlanMulticastRecipe.py | 155 ++++++--- lnst/Recipes/ENRT/VxlanRemoteRecipe.py | 115 +++++-- lnst/Recipes/ENRT/XfrmTools.py | 2 +- lnst/Tests/PacketAssert.py | 4 +- 39 files changed, 3339 insertions(+), 1854 deletions(-)
From: Christos Sfakianakis csfakian@redhat.com
Replace the generic 'hw_dev_list' in BaseHWConfigMixin class with a specialized list in each subclass. This allows needed flexibility for customized h/w configuration.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Recipes/ENRT/ConfigMixins/BaseHWConfigMixin.py | 8 ++------ lnst/Recipes/ENRT/ConfigMixins/CoalescingHWConfigMixin.py | 6 ++++++ .../ENRT/ConfigMixins/DevInterruptHWConfigMixin.py | 8 ++++++-- lnst/Recipes/ENRT/ConfigMixins/MTUHWConfigMixin.py | 6 +++++- .../ENRT/ConfigMixins/ParallelStreamQDiscHWConfigMixin.py | 6 +++++- 5 files changed, 24 insertions(+), 10 deletions(-)
diff --git a/lnst/Recipes/ENRT/ConfigMixins/BaseHWConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/BaseHWConfigMixin.py index 1550691..633a0fe 100644 --- a/lnst/Recipes/ENRT/ConfigMixins/BaseHWConfigMixin.py +++ b/lnst/Recipes/ENRT/ConfigMixins/BaseHWConfigMixin.py @@ -1,8 +1,4 @@ class BaseHWConfigMixin(object): - @property - def hw_config_dev_list(self): - return [] - def hw_config(self, config): config.hw_config = {}
@@ -12,11 +8,11 @@ class BaseHWConfigMixin(object): def describe_hw_config(self, config): return []
- def _configure_dev_attribute(self, config, attr_name, value): + def _configure_dev_attribute(self, config, dev_list, attr_name, value): hw_config = config.hw_config if value: attr_cfg = hw_config[attr_name + "_configuration"] = {} - for dev in self.hw_config_dev_list: + for dev in dev_list: attr_cfg[dev] = {} attr_cfg[dev]["original"] = getattr(dev, attr_name) setattr(dev, attr_name, value) diff --git a/lnst/Recipes/ENRT/ConfigMixins/CoalescingHWConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/CoalescingHWConfigMixin.py index d2e0eae..23fb208 100644 --- a/lnst/Recipes/ENRT/ConfigMixins/CoalescingHWConfigMixin.py +++ b/lnst/Recipes/ENRT/ConfigMixins/CoalescingHWConfigMixin.py @@ -7,16 +7,22 @@ class CoalescingHWConfigMixin(BaseHWConfigMixin): adaptive_rx_coalescing = BoolParam(mandatory=False) adaptive_tx_coalescing = BoolParam(mandatory=False)
+ @property + def coalescing_hw_config_dev_list(self): + return [] + def hw_config(self, config): super().hw_config(config)
self._configure_dev_attribute( config, + self.coalescing_hw_config_dev_list, "adaptive_rx_coalescing", getattr(self.params, "adaptive_rx_coalescing", None), ) self._configure_dev_attribute( config, + self.coalescing_hw_config_dev_list, "adaptive_tx_coalescing", getattr(self.params, "adaptive_tx_coalescing", None), ) diff --git a/lnst/Recipes/ENRT/ConfigMixins/DevInterruptHWConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/DevInterruptHWConfigMixin.py index 0a47255..4eb6c59 100644 --- a/lnst/Recipes/ENRT/ConfigMixins/DevInterruptHWConfigMixin.py +++ b/lnst/Recipes/ENRT/ConfigMixins/DevInterruptHWConfigMixin.py @@ -9,6 +9,10 @@ from lnst.Recipes.ENRT.ConfigMixins.BaseHWConfigMixin import BaseHWConfigMixin class DevInterruptHWConfigMixin(BaseHWConfigMixin): dev_intr_cpu = IntParam(mandatory=False)
+ @property + def dev_interrupt_hw_config_dev_list(self): + return [] + def hw_config(self, config): super().hw_config(config)
@@ -20,14 +24,14 @@ class DevInterruptHWConfigMixin(BaseHWConfigMixin): intr_cfg["irqbalance_hosts"] = []
hosts = [] - for dev in self.hw_config_dev_list: + for dev in self.dev_interrupt_hw_config_dev_list: if dev.host not in hosts: hosts.append(dev.host) for host in hosts: host.run("service irqbalance stop") intr_cfg["irqbalance_hosts"].append(host)
- for dev in self.hw_config_dev_list: + for dev in self.dev_interrupt_hw_config_dev_list: # TODO better service handling through HostAPI self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) intr_cfg["irq_devs"][dev] = self.params.dev_intr_cpu diff --git a/lnst/Recipes/ENRT/ConfigMixins/MTUHWConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/MTUHWConfigMixin.py index 6bcf0c5..143c0ea 100644 --- a/lnst/Recipes/ENRT/ConfigMixins/MTUHWConfigMixin.py +++ b/lnst/Recipes/ENRT/ConfigMixins/MTUHWConfigMixin.py @@ -6,11 +6,15 @@ from lnst.Recipes.ENRT.ConfigMixins.BaseHWConfigMixin import BaseHWConfigMixin class MTUHWConfigMixin(BaseHWConfigMixin): mtu = IntParam(mandatory=False)
+ @property + def mtu_hw_config_dev_list(self): + return [] + def hw_config(self, config): super().hw_config(config)
self._configure_dev_attribute( - config, "mtu", getattr(self.params, "mtu", None) + config, self.mtu_hw_config_dev_list, "mtu", getattr(self.params, "mtu", None) )
def describe_hw_config(self, config): diff --git a/lnst/Recipes/ENRT/ConfigMixins/ParallelStreamQDiscHWConfigMixin.py b/lnst/Recipes/ENRT/ConfigMixins/ParallelStreamQDiscHWConfigMixin.py index 7ca5741..dc6e011 100644 --- a/lnst/Recipes/ENRT/ConfigMixins/ParallelStreamQDiscHWConfigMixin.py +++ b/lnst/Recipes/ENRT/ConfigMixins/ParallelStreamQDiscHWConfigMixin.py @@ -2,6 +2,10 @@ from lnst.Recipes.ENRT.ConfigMixins.BaseHWConfigMixin import BaseHWConfigMixin
class ParallelStreamQDiscHWConfigMixin(BaseHWConfigMixin): + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [] + def hw_config(self, config): super().hw_config(config)
@@ -10,7 +14,7 @@ class ParallelStreamQDiscHWConfigMixin(BaseHWConfigMixin): parallel_streams = getattr(self.params, "perf_parallel_streams", None) if parallel_streams is not None and parallel_streams > 1: hw_config["parallel_stream_devs"] = [] - for dev in self.hw_config_dev_list: + for dev in self.parallel_stream_qdisc_hw_config_dev_list: dev.host.run("tc qdisc replace dev %s root mq" % dev.name) hw_config["parallel_stream_devs"].append(dev)
From: Christos Sfakianakis csfakian@redhat.com
Check its value against 'None' explicitly at the first 'if' clause.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/RecipeCommon/Perf/Measurements/IperfFlowMeasurement.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lnst/RecipeCommon/Perf/Measurements/IperfFlowMeasurement.py b/lnst/RecipeCommon/Perf/Measurements/IperfFlowMeasurement.py index 9faf7ec..c193987 100644 --- a/lnst/RecipeCommon/Perf/Measurements/IperfFlowMeasurement.py +++ b/lnst/RecipeCommon/Perf/Measurements/IperfFlowMeasurement.py @@ -113,7 +113,7 @@ class IperfFlowMeasurement(BaseFlowMeasurement): server_params = dict(bind = ipaddress(flow.receiver_bind), oneoff = True)
- if flow.cpupin and flow.cpupin >= 0: + if flow.cpupin is not None and flow.cpupin >= 0: if flow.parallel_streams == 1: server_params["cpu_bind"] = flow.cpupin else: @@ -140,7 +140,7 @@ class IperfFlowMeasurement(BaseFlowMeasurement): else: raise RecipeError("Unsupported flow type '{}'".format(flow.type))
- if flow.cpupin and flow.cpupin >= 0: + if flow.cpupin is not None and flow.cpupin >= 0: if flow.parallel_streams == 1: client_params["cpu_bind"] = flow.cpupin else:
From: Christos Sfakianakis csfakian@redhat.com
Return None in case realdev has not been set.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Devices/VxlanDevice.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/lnst/Devices/VxlanDevice.py b/lnst/Devices/VxlanDevice.py index c317913..3129bf3 100644 --- a/lnst/Devices/VxlanDevice.py +++ b/lnst/Devices/VxlanDevice.py @@ -43,9 +43,11 @@ class VxlanDevice(SoftDevice): def realdev(self): if self._nl_msg is None: return None - - if_id = int(self._get_linkinfo_data_attr("IFLA_VXLAN_LINK")) - return self._if_manager.get_device(if_id) + try: + if_id = int(self._get_linkinfo_data_attr("IFLA_VXLAN_LINK")) + return self._if_manager.get_device(if_id) + except: + return None
@realdev.setter def realdev(self, val):
On Fri, Jul 19, 2019 at 01:36:46PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
Return None in case realdev has not been set.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com
lnst/Devices/VxlanDevice.py | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/lnst/Devices/VxlanDevice.py b/lnst/Devices/VxlanDevice.py index c317913..3129bf3 100644 --- a/lnst/Devices/VxlanDevice.py +++ b/lnst/Devices/VxlanDevice.py @@ -43,9 +43,11 @@ class VxlanDevice(SoftDevice): def realdev(self): if self._nl_msg is None: return None
if_id = int(self._get_linkinfo_data_attr("IFLA_VXLAN_LINK"))
return self._if_manager.get_device(if_id)
try:
if_id = int(self._get_linkinfo_data_attr("IFLA_VXLAN_LINK"))
return self._if_manager.get_device(if_id)
except:
I think in this case we should handle just DeviceNotFound exceptions, so we don't accidentaly hide some other error.
return None
@realdev.setter def realdev(self, val):
-- 2.17.1 _______________________________________________ LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
From: Christos Sfakianakis csfakian@redhat.com
For BaseTestModule instances, ensure appropriate DeviceRef objects are created from the original kwargs of the instance to prevent pickle errors.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Controller/MessageDispatcher.py | 1 + 1 file changed, 1 insertion(+)
diff --git a/lnst/Controller/MessageDispatcher.py b/lnst/Controller/MessageDispatcher.py index ed6aa51..d2dc613 100644 --- a/lnst/Controller/MessageDispatcher.py +++ b/lnst/Controller/MessageDispatcher.py @@ -75,6 +75,7 @@ def remote_device_to_deviceref(obj): elif isinstance(obj, BaseTestModule): new_test = copy.deepcopy(obj) new_test.params = remote_device_to_deviceref(new_test.params) + new_test._orig_kwargs = remote_device_to_deviceref(new_test._orig_kwargs) return new_test else: return obj
From: Christos Sfakianakis csfakian@redhat.com
Convert the binary output/error of the packet assert process into text to comply with py3.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Tests/PacketAssert.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/lnst/Tests/PacketAssert.py b/lnst/Tests/PacketAssert.py index 9452a4f..25a92ba 100644 --- a/lnst/Tests/PacketAssert.py +++ b/lnst/Tests/PacketAssert.py @@ -76,8 +76,8 @@ class PacketAssert(BaseTestModule): raise LnstError("Could not handle interrupt properly.")
with packet_assert_process.stdout, packet_assert_process.stderr: - stderr=packet_assert_process.stderr.read() - stdout=packet_assert_process.stdout.read() + stderr=packet_assert_process.stderr.read().decode() + stdout=packet_assert_process.stdout.read().decode()
self._res_data["stderr"] = stderr
From: Christos Sfakianakis csfakian@redhat.com
In the case of 'ipsec_esp_aead', use the new integer division operator for the key construction.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Recipes/ENRT/XfrmTools.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lnst/Recipes/ENRT/XfrmTools.py b/lnst/Recipes/ENRT/XfrmTools.py index 468259e..fd226fc 100644 --- a/lnst/Recipes/ENRT/XfrmTools.py +++ b/lnst/Recipes/ENRT/XfrmTools.py @@ -2,7 +2,7 @@ from lnst.Common.LnstError import LnstError
def generate_key(length): key = "0x" - key = key + (length/8) * "0b" + key = key + (length//8) * "0b" return key
def configure_ipsec_esp_aead(m1, ip1, m2, ip2, algo, algo_key, icv_len,
From: Christos Sfakianakis csfakian@redhat.com
Add getter properties for ports, internal ports, tunnels, flows, bonds. Return values are based on the output of: - for ports/internal/ports/tunnels: 'ovs-vsctl show', 'ovs-ofctl dump-ports-desc <dev>' - for bonds: 'ovs-appctl bond/list' - for flows: 'ovs-ofctl dump-flows <dev>'
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Devices/OvsBridgeDevice.py | 111 ++++++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+)
diff --git a/lnst/Devices/OvsBridgeDevice.py b/lnst/Devices/OvsBridgeDevice.py index 643fa7d..d9bba5a 100644 --- a/lnst/Devices/OvsBridgeDevice.py +++ b/lnst/Devices/OvsBridgeDevice.py @@ -10,6 +10,8 @@ __author__ = """ olichtne@redhat.com (Ondrej Lichtner) """
+import re +import pprint from lnst.Common.Utils import check_process_running from lnst.Common.ExecCmd import exec_cmd from lnst.Common.DeviceError import DeviceError @@ -22,6 +24,8 @@ class OvsBridgeDevice(SoftDevice): def __init__(self, ifmanager, *args, **kwargs): super(OvsBridgeDevice, self).__init__(ifmanager) self._type_init() + self._numbered_ports = {} + self._port_lines = []
@classmethod def _type_init(cls): @@ -112,3 +116,110 @@ class OvsBridgeDevice(SoftDevice):
def flows_del(self, entry): exec_cmd("ovs-ofctl del-flows %s" % (self.name)) + + @property + def ports(self): + if not self._numbered_ports: + self._get_port_info() + ports = {} + + for line in self._port_lines: + if not re.search('type=', line): + self._index(line, ports) + + return ports + + @property + def internal_ports(self): + if not self._numbered_ports: + self._get_port_info() + int_ports = {} + + for line in self._port_lines: + if re.search('type=internal', line): + line = re.sub(r",?\stype=internal", "", line) + self._index(line, int_ports) + + return int_ports + + @property + def tunnels(self): + if not self._numbered_ports: + self._get_port_info() + tunnels = {} + + for line in self._port_lines: + if re.search('type=(?!internal)', line): + self._index(line, tunnels) + + return tunnels + + @property + def bonds(self): + bonds = {} + bond_list = [] + out = exec_cmd("ovs-appctl bond/list", log_outputs=False)[0] + + for line in out.split('\n'): + if line: + bond_list.append(line.split('\t')) + + for bond in bond_list[1:]: + bonds[bond[0]] = {'type' : bond[1], 'slaves' : bond[3]} + + return bonds + + @property + def flows(self): + flows = [] + ignore_exprs = [r"cookie", r"duration", r"n_packets", + r"n_bytes", r"idle_age"] + out = exec_cmd("ovs-ofctl dump-flows %s" % self.name, + log_outputs=False)[0] + + for line in out.split('\n'): + if line: + flows.append(line.split(', ')) + + for flow in flows[1:]: + for entry in list(flow): + for expr in ignore_exprs: + if re.search(expr, entry): + del flow[flow.index(entry)] + break + + return pprint.pformat(flows[1:]) + + def _get_port_info(self): + dumped_ports = exec_cmd("ovs-ofctl dump-ports-desc %s" % + self.name, log_outputs=False)[0] + + for match in re.finditer(r'(\w+)((\w*))', + dumped_ports): + self._numbered_ports[match.groups()[1]] = match.groups()[0] + + ovs_show = exec_cmd("ovs-vsctl show", + log_outputs=False)[0] + regex = r'(Port[\w\W]*?)(?=Port|ovs_version)' + + for match in re.finditer(regex, ovs_show): + line = match.groups()[0].replace('\n', ' ') + line = self._port_format(line) + self._port_lines.append(line) + + def _port_format(self, line): + res = re.sub(r":", "", line) + res = re.sub(r"(\S[^,])\s(\S)", "\1=\2", res) + res = re.sub(r"\s{2,}(?=\S)", ", ", res) + res = re.sub(r"\s*$", "", res) + + return res + + def _index(self, line, result): + name = re.match(r"Port="(\w+)"", line).groups()[0] + + try: + number = self._numbered_ports[name] + result[number] = line + except KeyError: + pass
On Fri, Jul 19, 2019 at 01:36:50PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
Add getter properties for ports, internal ports, tunnels, flows, bonds. Return values are based on the output of:
- for ports/internal/ports/tunnels: 'ovs-vsctl show', 'ovs-ofctl dump-ports-desc <dev>'
- for bonds: 'ovs-appctl bond/list'
- for flows: 'ovs-ofctl dump-flows <dev>'
Signed-off-by: Christos Sfakianakis csfakian@redhat.com
lnst/Devices/OvsBridgeDevice.py | 111 ++++++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+)
diff --git a/lnst/Devices/OvsBridgeDevice.py b/lnst/Devices/OvsBridgeDevice.py index 643fa7d..d9bba5a 100644 --- a/lnst/Devices/OvsBridgeDevice.py +++ b/lnst/Devices/OvsBridgeDevice.py @@ -10,6 +10,8 @@ __author__ = """ olichtne@redhat.com (Ondrej Lichtner) """
+import re +import pprint from lnst.Common.Utils import check_process_running from lnst.Common.ExecCmd import exec_cmd from lnst.Common.DeviceError import DeviceError @@ -22,6 +24,8 @@ class OvsBridgeDevice(SoftDevice): def __init__(self, ifmanager, *args, **kwargs): super(OvsBridgeDevice, self).__init__(ifmanager) self._type_init()
self._numbered_ports = {}
self._port_lines = []
i assume these are supposed to work as a sort of cache for the data? Does it make sense to only load them up once with the get_port_info()? Could the bridge configuration change during recipe execution and we'd end up getting out of date info from this?
@classmethod def _type_init(cls):
@@ -112,3 +116,110 @@ class OvsBridgeDevice(SoftDevice):
def flows_del(self, entry): exec_cmd("ovs-ofctl del-flows %s" % (self.name))
- @property
- def ports(self):
if not self._numbered_ports:
self._get_port_info()
ports = {}
for line in self._port_lines:
if not re.search('type=', line):
self._index(line, ports)
return ports
- @property
- def internal_ports(self):
if not self._numbered_ports:
self._get_port_info()
int_ports = {}
for line in self._port_lines:
if re.search('type=internal', line):
line = re.sub(r",?\stype=internal", "", line)
self._index(line, int_ports)
return int_ports
- @property
- def tunnels(self):
if not self._numbered_ports:
self._get_port_info()
tunnels = {}
for line in self._port_lines:
if re.search('type=(?!internal)', line):
self._index(line, tunnels)
return tunnels
- @property
- def bonds(self):
bonds = {}
bond_list = []
out = exec_cmd("ovs-appctl bond/list", log_outputs=False)[0]
for line in out.split('\n'):
if line:
bond_list.append(line.split('\t'))
for bond in bond_list[1:]:
bonds[bond[0]] = {'type' : bond[1], 'slaves' : bond[3]}
return bonds
- @property
- def flows(self):
flows = []
ignore_exprs = [r"cookie", r"duration", r"n_packets",
r"n_bytes", r"idle_age"]
out = exec_cmd("ovs-ofctl dump-flows %s" % self.name,
log_outputs=False)[0]
for line in out.split('\n'):
if line:
flows.append(line.split(', '))
for flow in flows[1:]:
for entry in list(flow):
for expr in ignore_exprs:
if re.search(expr, entry):
del flow[flow.index(entry)]
break
return pprint.pformat(flows[1:])
The previous properties return a data structure (a dict), but this one returns a formatted string, is that intentional? If so would it make more sense to name the method with something more descriptive e.g. flows_desc or flows_str. Or if not then returning a data structure would be better?
- def _get_port_info(self):
dumped_ports = exec_cmd("ovs-ofctl dump-ports-desc %s" %
self.name, log_outputs=False)[0]
for match in re.finditer(r'(\w+)\((\w*)\)',
dumped_ports):
self._numbered_ports[match.groups()[1]] = match.groups()[0]
ovs_show = exec_cmd("ovs-vsctl show",
log_outputs=False)[0]
regex = r'(Port[\w\W]*?)(?=Port|ovs_version)'
for match in re.finditer(regex, ovs_show):
line = match.groups()[0].replace('\n', ' ')
line = self._port_format(line)
self._port_lines.append(line)
wrt. the first comment, yeah I think saving this method output data into instance attributes is a bit weird, considering that almost immediately after they get used they can be considered "outdated", in such a case it might make more sense to call this method every time you call one of the "ports" properties and return the output values directly instead of going through instance attributes.
- def _port_format(self, line):
res = re.sub(r":", "", line)
res = re.sub(r"(\S[^,])\s(\S)", "\\1=\\2", res)
res = re.sub(r"\s{2,}(?=\S)", ", ", res)
res = re.sub(r"\s*$", "", res)
return res
- def _index(self, line, result):
I'm not sure what this method is doing exactly, could a more descriptive name be used?
name = re.match(r"Port=\"(\w+)\"", line).groups()[0]
try:
number = self._numbered_ports[name]
result[number] = line
except KeyError:
pass
-- 2.17.1 _______________________________________________ LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
Hi,
----- Original Message -----
From: "Ondrej Lichtner" olichtne@redhat.com To: csfakian@redhat.com Cc: lnst-developers@lists.fedorahosted.org Sent: Monday, July 22, 2019 3:48:05 PM Subject: Re: [PATCH-next 7/8] lnst.Devices.OvsBridgeDevice: add config getters
On Fri, Jul 19, 2019 at 01:36:50PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
Add getter properties for ports, internal ports, tunnels, flows, bonds. Return values are based on the output of:
- for ports/internal/ports/tunnels: 'ovs-vsctl show', 'ovs-ofctl dump-ports-desc <dev>'
- for bonds: 'ovs-appctl bond/list'
- for flows: 'ovs-ofctl dump-flows <dev>'
Signed-off-by: Christos Sfakianakis csfakian@redhat.com
lnst/Devices/OvsBridgeDevice.py | 111 ++++++++++++++++++++++++++++++++ 1 file changed, 111 insertions(+)
diff --git a/lnst/Devices/OvsBridgeDevice.py b/lnst/Devices/OvsBridgeDevice.py index 643fa7d..d9bba5a 100644 --- a/lnst/Devices/OvsBridgeDevice.py +++ b/lnst/Devices/OvsBridgeDevice.py @@ -10,6 +10,8 @@ __author__ = """ olichtne@redhat.com (Ondrej Lichtner) """
+import re +import pprint from lnst.Common.Utils import check_process_running from lnst.Common.ExecCmd import exec_cmd from lnst.Common.DeviceError import DeviceError @@ -22,6 +24,8 @@ class OvsBridgeDevice(SoftDevice): def __init__(self, ifmanager, *args, **kwargs): super(OvsBridgeDevice, self).__init__(ifmanager) self._type_init()
self._numbered_ports = {}
self._port_lines = []
i assume these are supposed to work as a sort of cache for the data? Does it make sense to only load them up once with the get_port_info()? Could the bridge configuration change during recipe execution and we'd end up getting out of date info from this?
I will need to ensure these are always up to date (as you said) to avoid outdated info being sent back in case a custom test case is used.
@classmethod def _type_init(cls):
@@ -112,3 +116,110 @@ class OvsBridgeDevice(SoftDevice):
def flows_del(self, entry): exec_cmd("ovs-ofctl del-flows %s" % (self.name))
- @property
- def ports(self):
if not self._numbered_ports:
self._get_port_info()
ports = {}
for line in self._port_lines:
if not re.search('type=', line):
self._index(line, ports)
return ports
- @property
- def internal_ports(self):
if not self._numbered_ports:
self._get_port_info()
int_ports = {}
for line in self._port_lines:
if re.search('type=internal', line):
line = re.sub(r",?\stype=internal", "", line)
self._index(line, int_ports)
return int_ports
- @property
- def tunnels(self):
if not self._numbered_ports:
self._get_port_info()
tunnels = {}
for line in self._port_lines:
if re.search('type=(?!internal)', line):
self._index(line, tunnels)
return tunnels
- @property
- def bonds(self):
bonds = {}
bond_list = []
out = exec_cmd("ovs-appctl bond/list", log_outputs=False)[0]
for line in out.split('\n'):
if line:
bond_list.append(line.split('\t'))
for bond in bond_list[1:]:
bonds[bond[0]] = {'type' : bond[1], 'slaves' : bond[3]}
return bonds
- @property
- def flows(self):
flows = []
ignore_exprs = [r"cookie", r"duration", r"n_packets",
r"n_bytes", r"idle_age"]
out = exec_cmd("ovs-ofctl dump-flows %s" % self.name,
log_outputs=False)[0]
for line in out.split('\n'):
if line:
flows.append(line.split(', '))
for flow in flows[1:]:
for entry in list(flow):
for expr in ignore_exprs:
if re.search(expr, entry):
del flow[flow.index(entry)]
break
return pprint.pformat(flows[1:])
The previous properties return a data structure (a dict), but this one returns a formatted string, is that intentional? If so would it make more sense to name the method with something more descriptive e.g. flows_desc or flows_str. Or if not then returning a data structure would be better?
This was intentional, I tried to filter out non-relevant info from the output, while still storing it line by line. I will name the method differently.
- def _get_port_info(self):
dumped_ports = exec_cmd("ovs-ofctl dump-ports-desc %s" %
self.name, log_outputs=False)[0]
for match in re.finditer(r'(\w+)\((\w*)\)',
dumped_ports):
self._numbered_ports[match.groups()[1]] = match.groups()[0]
ovs_show = exec_cmd("ovs-vsctl show",
log_outputs=False)[0]
regex = r'(Port[\w\W]*?)(?=Port|ovs_version)'
for match in re.finditer(regex, ovs_show):
line = match.groups()[0].replace('\n', ' ')
line = self._port_format(line)
self._port_lines.append(line)
wrt. the first comment, yeah I think saving this method output data into instance attributes is a bit weird, considering that almost immediately after they get used they can be considered "outdated", in such a case it might make more sense to call this method every time you call one of the "ports" properties and return the output values directly instead of going through instance attributes.
Sure, I will change the code accordingly.
- def _port_format(self, line):
res = re.sub(r":", "", line)
res = re.sub(r"(\S[^,])\s(\S)", "\\1=\\2", res)
res = re.sub(r"\s{2,}(?=\S)", ", ", res)
res = re.sub(r"\s*$", "", res)
return res
- def _index(self, line, result):
I'm not sure what this method is doing exactly, could a more descriptive name be used?
This creates a pair held in 'result', where the key is the port number and the value is 'line' (which includes the port's name). Perhaps a different name (again), like 'line_to_port_number', would be better.
Thanks,
Christos
name = re.match(r"Port=\"(\w+)\"", line).groups()[0]
try:
number = self._numbered_ports[name]
result[number] = line
except KeyError:
pass
-- 2.17.1 _______________________________________________ LNST-developers mailing list -- lnst-developers@lists.fedorahosted.org To unsubscribe send an email to lnst-developers-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedorahos...
Mon, Jul 22, 2019 at 04:45:12PM CEST, csfakian@redhat.com wrote:
Hi,
----- Original Message -----
From: "Ondrej Lichtner" olichtne@redhat.com To: csfakian@redhat.com Cc: lnst-developers@lists.fedorahosted.org Sent: Monday, July 22, 2019 3:48:05 PM Subject: Re: [PATCH-next 7/8] lnst.Devices.OvsBridgeDevice: add config getters
On Fri, Jul 19, 2019 at 01:36:50PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
- @property
- def flows(self):
flows = []
ignore_exprs = [r"cookie", r"duration", r"n_packets",
r"n_bytes", r"idle_age"]
out = exec_cmd("ovs-ofctl dump-flows %s" % self.name,
log_outputs=False)[0]
for line in out.split('\n'):
if line:
flows.append(line.split(', '))
for flow in flows[1:]:
for entry in list(flow):
for expr in ignore_exprs:
if re.search(expr, entry):
del flow[flow.index(entry)]
break
return pprint.pformat(flows[1:])
The previous properties return a data structure (a dict), but this one returns a formatted string, is that intentional? If so would it make more sense to name the method with something more descriptive e.g. flows_desc or flows_str. Or if not then returning a data structure would be better?
This was intentional, I tried to filter out non-relevant info from the output, while still storing it line by line. I will name the method differently.
When you say non-relevant I assume the code would work even without filtering. Is this really necessary or just some sort of optimization?
And I wonder if we find out that we need some of the 'non-relevant' data some time in the future.
If the code works without filtering I think it should be removed.
-Jan
----- Original Message -----
From: "Jan Tluka" jtluka@redhat.com To: "Christos Sfakianakis" csfakian@redhat.com Cc: "Ondrej Lichtner" olichtne@redhat.com, lnst-developers@lists.fedorahosted.org Sent: Tuesday, July 23, 2019 10:56:22 AM Subject: Re: [PATCH-next 7/8] lnst.Devices.OvsBridgeDevice: add config getters
Mon, Jul 22, 2019 at 04:45:12PM CEST, csfakian@redhat.com wrote:
Hi,
----- Original Message -----
From: "Ondrej Lichtner" olichtne@redhat.com To: csfakian@redhat.com Cc: lnst-developers@lists.fedorahosted.org Sent: Monday, July 22, 2019 3:48:05 PM Subject: Re: [PATCH-next 7/8] lnst.Devices.OvsBridgeDevice: add config getters
On Fri, Jul 19, 2019 at 01:36:50PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
- @property
- def flows(self):
flows = []
ignore_exprs = [r"cookie", r"duration", r"n_packets",
r"n_bytes", r"idle_age"]
out = exec_cmd("ovs-ofctl dump-flows %s" % self.name,
log_outputs=False)[0]
for line in out.split('\n'):
if line:
flows.append(line.split(', '))
for flow in flows[1:]:
for entry in list(flow):
for expr in ignore_exprs:
if re.search(expr, entry):
del flow[flow.index(entry)]
break
return pprint.pformat(flows[1:])
The previous properties return a data structure (a dict), but this one returns a formatted string, is that intentional? If so would it make more sense to name the method with something more descriptive e.g. flows_desc or flows_str. Or if not then returning a data structure would be better?
This was intentional, I tried to filter out non-relevant info from the output, while still storing it line by line. I will name the method differently.
When you say non-relevant I assume the code would work even without filtering. Is this really necessary or just some sort of optimization?
'Non-relevant' in this case means that you do not control it or set it explicitly in the recipe. Cookies, bytes, packets, idle_age or duration do not belong to the explicitly configured part of the device and are rather informative fields that accompany the 'flows' information of it.
Also, most of these fields contain data that is non-static, meaning these values may change over time, without the tester's control. I think such data should be filtered out. The original intention was to use these properties to present a description of the configuration of an OvsBridgeDevice to the tester while actually running the recipe (this could be useful in a number of ways and had already been supported anyways, but we were missing these properties).
And I wonder if we find out that we need some of the 'non-relevant' data some time in the future.
If we found out we do need some of these 5 fields, the fix should be easy. But IMO we will not be needing that info:)
If the code works without filtering I think it should be removed.
If you still preserve this opinion, I will proceed to remove these filters, please confirm.
-Jan
Tue, Jul 23, 2019 at 11:21:59AM CEST, csfakian@redhat.com wrote:
----- Original Message -----
From: "Jan Tluka" jtluka@redhat.com To: "Christos Sfakianakis" csfakian@redhat.com Cc: "Ondrej Lichtner" olichtne@redhat.com, lnst-developers@lists.fedorahosted.org Sent: Tuesday, July 23, 2019 10:56:22 AM Subject: Re: [PATCH-next 7/8] lnst.Devices.OvsBridgeDevice: add config getters
Mon, Jul 22, 2019 at 04:45:12PM CEST, csfakian@redhat.com wrote:
Hi,
----- Original Message -----
From: "Ondrej Lichtner" olichtne@redhat.com To: csfakian@redhat.com Cc: lnst-developers@lists.fedorahosted.org Sent: Monday, July 22, 2019 3:48:05 PM Subject: Re: [PATCH-next 7/8] lnst.Devices.OvsBridgeDevice: add config getters
On Fri, Jul 19, 2019 at 01:36:50PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
- @property
- def flows(self):
flows = []
ignore_exprs = [r"cookie", r"duration", r"n_packets",
r"n_bytes", r"idle_age"]
out = exec_cmd("ovs-ofctl dump-flows %s" % self.name,
log_outputs=False)[0]
for line in out.split('\n'):
if line:
flows.append(line.split(', '))
for flow in flows[1:]:
for entry in list(flow):
for expr in ignore_exprs:
if re.search(expr, entry):
del flow[flow.index(entry)]
break
return pprint.pformat(flows[1:])
The previous properties return a data structure (a dict), but this one returns a formatted string, is that intentional? If so would it make more sense to name the method with something more descriptive e.g. flows_desc or flows_str. Or if not then returning a data structure would be better?
This was intentional, I tried to filter out non-relevant info from the output, while still storing it line by line. I will name the method differently.
When you say non-relevant I assume the code would work even without filtering. Is this really necessary or just some sort of optimization?
'Non-relevant' in this case means that you do not control it or set it explicitly in the recipe. Cookies, bytes, packets, idle_age or duration do not belong to the explicitly configured part of the device and are rather informative fields that accompany the 'flows' information of it.
Also, most of these fields contain data that is non-static, meaning these values may change over time, without the tester's control. I think such data should be filtered out. The original intention was to use these properties to present a description of the configuration of an OvsBridgeDevice to the tester while actually running the recipe (this could be useful in a number of ways and had already been supported anyways, but we were missing these properties).
And I wonder if we find out that we need some of the 'non-relevant' data some time in the future.
If we found out we do need some of these 5 fields, the fix should be easy. But IMO we will not be needing that info:)
If the code works without filtering I think it should be removed.
If you still preserve this opinion, I will proceed to remove these filters, please confirm.
No, I don't. With your explanation it completely makes sense.
Thanks, Jan
From: Christos Sfakianakis csfakian@redhat.com
Rewrite recipes to be compatible with the above patches.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com --- lnst/Recipes/ENRT/BondRecipe.py | 137 +++++--- lnst/Recipes/ENRT/DoubleBondRecipe.py | 140 +++++--- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 159 ++++++---- lnst/Recipes/ENRT/IpsecEspAeadRecipe.py | 280 ++++++++-------- lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py | 297 ++++++++--------- lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py | 130 +++++--- lnst/Recipes/ENRT/PingFloodRecipe.py | 21 +- .../ENRT/ShortLivedConnectionsRecipe.py | 87 ++--- lnst/Recipes/ENRT/SimpleMacsecRecipe.py | 252 +++++++-------- lnst/Recipes/ENRT/SimplePerfRecipe.py | 42 +-- lnst/Recipes/ENRT/TeamRecipe.py | 134 +++++--- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 162 ++++++---- lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py | 230 +++++++++++--- .../VirtualBridgeVlanInGuestMirroredRecipe.py | 191 ++++++----- .../ENRT/VirtualBridgeVlanInGuestRecipe.py | 167 ++++++---- .../VirtualBridgeVlanInHostMirroredRecipe.py | 178 +++++++---- .../ENRT/VirtualBridgeVlanInHostRecipe.py | 155 ++++++--- .../ENRT/VirtualBridgeVlansOverBondRecipe.py | 298 +++++++++++++----- ...rtualOvsBridgeVlanInGuestMirroredRecipe.py | 193 +++++++----- .../ENRT/VirtualOvsBridgeVlanInGuestRecipe.py | 164 ++++++---- ...irtualOvsBridgeVlanInHostMirroredRecipe.py | 155 +++++---- .../ENRT/VirtualOvsBridgeVlanInHostRecipe.py | 144 ++++++--- .../VirtualOvsBridgeVlansOverBondRecipe.py | 277 ++++++++++------ lnst/Recipes/ENRT/VlansOverBondRecipe.py | 267 +++++++++++----- lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 269 +++++++++++----- lnst/Recipes/ENRT/VlansRecipe.py | 230 ++++++++++---- lnst/Recipes/ENRT/VxlanMulticastRecipe.py | 155 ++++++--- lnst/Recipes/ENRT/VxlanRemoteRecipe.py | 115 +++++-- 28 files changed, 3193 insertions(+), 1836 deletions(-)
diff --git a/lnst/Recipes/ENRT/BondRecipe.py b/lnst/Recipes/ENRT/BondRecipe.py index bbe2d57..bb3bc6b 100644 --- a/lnst/Recipes/ENRT/BondRecipe.py +++ b/lnst/Recipes/ENRT/BondRecipe.py @@ -1,14 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase1/ -({active_backup, round_robin}_bond.xml + bonding_test.py) -""" from lnst.Common.Parameters import Param, IntParam, StrParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import BondDevice
-class BondRecipe(BaseEnrtRecipe): +class BondRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="net1", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="net1", driver=RecipeParam("driver")) @@ -27,57 +28,91 @@ class BondRecipe(BaseEnrtRecipe):
def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2 + host1.bond0 = BondDevice(mode=self.params.bonding_mode, + miimon=self.params.miimon_value) + configuration = super().test_wide_configuration() + configuration.test_wide_devices = []
- host1.bond0 = BondDevice(mode=self.params.bonding_mode, miimon=self.params.miimon_value) - host1.eth0.down() - host1.eth1.down() - host1.bond0.slave_add(host1.eth0) - host1.bond0.slave_add(host1.eth1) - - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.bond0 - configuration.endpoint2 = host2.eth0 - - if "mtu" in self.params: - host1.bond0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu + for dev in [host1.eth0, host1.eth1]: + dev.down() + host1.bond0.slave_add(dev)
net_addr = "192.168.101" net_addr6 = "fc00:0:0:0" - host1.bond0.ip_add(ipaddress(net_addr + ".1/24")) - host1.bond0.ip_add(ipaddress(net_addr6 + "::1/64")) - host1.eth0.up() - host1.eth1.up() - host1.bond0.up() - - host2.eth0.ip_add(ipaddress(net_addr + ".2/24")) - host2.eth0.ip_add(ipaddress(net_addr6 + "::2/64")) - host2.eth0.up() - - if "adaptive_rx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host1.eth0, host1.eth1, host2.eth0]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host, dev in [(host1, host1.eth0), (host1, host1.eth1), (host2, host2.eth0)]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + for i, dev in enumerate([host1.bond0, host2.eth0]): + dev.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) + dev.ip_add(ipaddress(net_addr6 + "::" + str(i+1) + "/64")) + configuration.test_wide_devices.append(dev) + + for dev in [host1.eth0, host1.eth1, host1.bond0, host2.eth0]: + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): + def generate_test_wide_description(self, config): host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "Configured {}.{}.slaves = {}".format( + host1.hostid, host1.bond0.name, + ['.'.join([host1.hostid, slave.name]) + for slave in host1.bond0.slaves] + ), + "Configured {}.{}.mode = {}".format( + host1.hostid, host1.bond0.name, + host1.bond0.mode + ), + "Configured {}.{}.miimon = {}".format( + host1.hostid, host1.bond0.name, + host1.bond0.miimon + ) + ] + return desc + + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.bond0, self.matched.host2.eth0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.bond0, self.matched.host2.eth0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.bond0, self.matched.host2.eth0] + + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.bond0, self.matched.host2.eth0] + + @property + def coalescing_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host1.eth1, + self.matched.host2.eth0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/DoubleBondRecipe.py b/lnst/Recipes/ENRT/DoubleBondRecipe.py index 3b437af..3951f7c 100644 --- a/lnst/Recipes/ENRT/DoubleBondRecipe.py +++ b/lnst/Recipes/ENRT/DoubleBondRecipe.py @@ -1,14 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase1/ -({round_robin, active_backup}_double_bond.xml + bonding_test.py). -""" from lnst.Common.Parameters import Param, StrParam, IntParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import BondDevice
-class DoubleBondRecipe(BaseEnrtRecipe): +class DoubleBondRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="net1", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="net1", driver=RecipeParam("driver")) @@ -29,57 +30,96 @@ class DoubleBondRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- for host in (host1, host2): - host.bond0 = BondDevice(mode=self.params.bonding_mode, miimon=self.params.miimon_value) - host.eth0.down() - host.eth1.down() - host.bond0.slave_add(host.eth0) - host.bond0.slave_add(host.eth1) - - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.bond0 - configuration.endpoint2 = host2.bond0 - - if "mtu" in self.params: - host1.bond0.mtu = self.params.mtu - host2.bond0.mtu = self.params.mtu - net_addr = "192.168.101" net_addr6 = "fc00:0:0:0" for i, host in enumerate([host1, host2]): + host.bond0 = BondDevice(mode=self.params.bonding_mode, + miimon=self.params.miimon_value) + for dev in [host.eth0, host.eth1]: + dev.down() + host.bond0.slave_add(dev) host.bond0.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) - host.bond0.ip_add(ipaddress(net_addr6 + "::" + str(i+1) + "/64")) - host.eth0.up() - host.eth1.up() - host.bond0.up() - - if "adaptive_tx_coalescing" in self.params: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - if "adaptive_tx_coalescing" in self.params: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host.eth0, host.eth1]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + host.bond0.ip_add(ipaddress(net_addr6 + "::" + str(i+1) + + "/64")) + for dev in [host.eth0, host.eth1, host.bond0]: + dev.up() + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.bond0, host2.bond0] + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.slaves = {}".format( + dev.host.hostid, dev.name, + ['.'.join([dev.host.hostid, slave.name]) + for slave in dev.slaves] + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.mode = {}".format( + dev.host.hostid, dev.name, dev.mode + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.miimon = {}".format( + dev.host.hostid, dev.name, dev.miimon + ) + for dev in config.test_wide_devices + ]) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.bond0, self.matched.host2.bond0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.bond0, self.matched.host2.bond0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.bond0, self.matched.host2.bond0] + + @property + def mtu_hw_config_dev_list(self): + return self.offload_nics + + @property + def coalescing_hw_config_dev_list(self): host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host1.eth1, host2.eth0, host2.eth1] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/DoubleTeamRecipe.py b/lnst/Recipes/ENRT/DoubleTeamRecipe.py index cea3542..452936a 100644 --- a/lnst/Recipes/ENRT/DoubleTeamRecipe.py +++ b/lnst/Recipes/ENRT/DoubleTeamRecipe.py @@ -1,14 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase2/ -({active_backup,round_robin}_double_team.xml + team_test.py) -""" -from lnst.Common.Parameters import Param, IntParam, StrParam, BoolParam +from lnst.Common.Parameters import Param, StrParam, BoolParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import TeamDevice
-class DoubleTeamRecipe(BaseEnrtRecipe): +class DoubleTeamRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="tnet", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="tnet", driver=RecipeParam("driver")) @@ -29,71 +30,95 @@ class DoubleTeamRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- host1.eth0.down() - host1.eth1.down() - #The config argument needs to be used with a team device normally (e.g to specify - #the runner mode), but it is not used here due to a bug in the TeamDevice module - host1.team0 = TeamDevice() - host1.team0.slave_add(host1.eth0) - host1.team0.slave_add(host1.eth1) - - host2.eth0.down() - host2.eth1.down() - host2.team0 = TeamDevice() - host2.team0.slave_add(host2.eth0) - host2.team0.slave_add(host2.eth1) - - #EnrtConfiguration and both-side Netperf config need to be checked - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.team0 - configuration.endpoint2 = host2.team0 - - if "mtu" in self.params: - host1.team0.mtu = self.params.mtu - host2.team0.mtu = self.params.mtu - net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" - - host1.team0.ip_add(ipaddress(net_addr_1 + ".1/24")) - host1.team0.ip_add(ipaddress(net_addr6_1 + "::1/64")) - host2.team0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.team0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - - host1.eth0.up() - host1.eth1.up() - host1.team0.up() - host2.eth0.up() - host2.eth1.up() - host2.team0.up() - - if "adaptive_rx_coalescing" in self.params: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - dev.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host.eth0, host.eth1]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + for i, host in enumerate([host1, host2]): + #The config argument needs to be used with a team device + #normally (e.g to specify the runner mode), but it is not used + #here due to a bug in the TeamDevice module + host.team0 = TeamDevice() + for dev in [host.eth0, host.eth1]: + dev.down() + host.team0.slave_add(dev) + host.team0.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + + "/24")) + host.team0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + + "/64")) + for dev in [host.eth0, host.eth1, host.team0]: + dev.up() + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.team0, host2.team0] + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.slaves = {}".format( + dev.host.hostid, dev.name, + ['.'.join([dev.host.hostid, slave.name]) + for slave in dev.slaves] + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.runner_name = {}".format( + dev.host.hostid, dev.name, dev.config + ) + for dev in config.test_wide_devices + ]) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.team0, self.matched.host2.team0), + (self.matched.host2.team0, self.matched.host1.team0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.team0, self.matched.host2.team0), + (self.matched.host2.team0, self.matched.host1.team0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.team0, self.matched.host2.team0] + + @property + def mtu_hw_config_dev_list(self): + return self.offload_nics + + @property + def coalescing_hw_config_dev_list(self): host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host1.eth1, host2.eth0, host2.eth1] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py b/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py index c78b130..4d8ee42 100644 --- a/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py +++ b/lnst/Recipes/ENRT/IpsecEspAeadRecipe.py @@ -1,43 +1,27 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(ipsec_esp_aead.xml + ipsec_esp_aead.py) -""" import signal import logging +import copy from lnst.Common.IpAddress import ipaddress from lnst.Common.IpAddress import AF_INET, AF_INET6 from lnst.Common.Parameters import StrParam from lnst.Common.LnstError import LnstError from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration, EnrtSubConfiguration -from lnst.RecipeCommon.PacketAssert import PacketAssertConf, PacketAssertTestAndEvaluate +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.BaseSubConfigMixin import ( + BaseSubConfigMixin as ConfMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) +from lnst.RecipeCommon.PacketAssert import (PacketAssertConf, + PacketAssertTestAndEvaluate) from lnst.RecipeCommon.Perf.Measurements import Flow as PerfFlow from lnst.RecipeCommon.Ping import PingConf -from lnst.Recipes.ENRT.XfrmTools import configure_ipsec_esp_aead, generate_key +from lnst.Recipes.ENRT.XfrmTools import (configure_ipsec_esp_aead, + generate_key)
-class IpsecEnrtSubConfiguration(EnrtSubConfiguration): - def __init__(self): - super(IpsecEnrtSubConfiguration, self).__init__() - self._ips = () - self._ipsec_settings = () - - @property - def ipsec_settings(self): - return self._ipsec_settings - - @ipsec_settings.setter - def ipsec_settings(self, value): - self._ipsec_settings = value - - @property - def ips(self): - return self._ips - - @ips.setter - def ips(self, value): - self._ips = value - -class IpsecEspAeadRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): +class IpsecEspAeadRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe, PacketAssertTestAndEvaluate): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver"))
@@ -48,49 +32,108 @@ class IpsecEspAeadRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): spi_values = ["0x00001000", "0x00001001"] ipsec_mode = StrParam(default="transport")
- def generate_sub_configurations(self, main_config): + def test_wide_configuration(self): + host1, host2 = self.matched.host1, self.matched.host2 + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host2.eth0] + + net_addr = "192.168." + net_addr6 = "fc00:" + for i, host in enumerate([host1, host2]): + host.eth0.down() + host.eth0.ip_add(ipaddress(net_addr + str(i+99) + ".1/24")) + host.eth0.ip_add(ipaddress(net_addr6 + str(i+1) + "::1/64")) + host.eth0.up() + + self.wait_tentative_ips(configuration.test_wide_devices) + + if self.params.ping_parallel or self.params.ping_bidirect: + logging.debug("Parallelism in pings is not supported for this " + "recipe, ping_parallel/ping_bidirect will be ignored.") + + for host, dst in [(host1, host2), (host2, host1)]: + for family in [AF_INET, AF_INET6]: + host.run("ip route add %s dev %s" % + (dst.eth0.ips_filter(family=family)[0], + host.eth0.name)) + + configuration.endpoint1 = host1.eth0 + configuration.endpoint2 = host2.eth0 + + return configuration + + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]) + ] + return desc + + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + def generate_sub_configurations(self, config): ipsec_mode = self.params.ipsec_mode spi_values = self.spi_values - for offload_settings in self.params.offload_combinations: - for ipv in self.params.ip_versions: - if ipv == "ipv4": - family = AF_INET - elif ipv == "ipv6": - family = AF_INET6 - - ip1 = main_config.endpoint1.ips_filter(family=family)[0] - ip2 = main_config.endpoint2.ips_filter(family=family)[0] - - for algo, key_len, icv_len in self.algorithm: - g_key = generate_key(key_len) - sub_config = IpsecEnrtSubConfiguration() - sub_config.offload_settings = offload_settings - sub_config.ips = (ip1, ip2) - sub_config.ipsec_settings = (algo, g_key, icv_len, - ipsec_mode, spi_values) - yield sub_config - - def apply_sub_configuration(self, main_config, sub_config): - super(IpsecEspAeadRecipe, self).apply_sub_configuration(main_config, sub_config) - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns - ip1, ip2 = sub_config.ips - ipsec_sets = sub_config.ipsec_settings + for subconf in ConfMixin.generate_sub_configurations(self, config): + for offload_settings in self.params.offload_combinations: + for ipv in self.params.ip_versions: + if ipv == "ipv4": + family = AF_INET + elif ipv == "ipv6": + family = AF_INET6 + + ip1 = subconf.endpoint1.ips_filter(family=family)[0] + ip2 = subconf.endpoint2.ips_filter(family=family)[0] + + for algo, key_len, icv_len in self.algorithm: + g_key = generate_key(key_len) + new_config = copy.copy(subconf) + new_config.offload_settings = offload_settings + new_config.ips = (ip1, ip2) + new_config.ipsec_settings = (algo, g_key, icv_len, + ipsec_mode, spi_values) + yield new_config + + def apply_sub_configuration(self, config): + super().apply_sub_configuration(config) + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns + ip1, ip2 = config.ips + ipsec_sets = config.ipsec_settings configure_ipsec_esp_aead(ns1, ip1, ns2, ip2, *ipsec_sets)
- def remove_sub_configuration(self, main_config, sub_config): - super(IpsecEspAeadRecipe, self).remove_sub_configuration(main_config, sub_config) - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns + def remove_sub_configuration(self, config): + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns for ns in (ns1, ns2): ns.run("ip xfrm policy flush") ns.run("ip xfrm state flush") + super().remove_sub_configuration(config)
- def generate_ping_configurations(self, main_config, sub_config): - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns - ip1, ip2 = sub_config.ips + def generate_ping_configurations(self, config): + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns + ip1, ip2 = config.ips count = self.params.ping_count interval = self.params.ping_interval size = self.params.ping_psize - common_args = {'count' : count, 'interval' : interval, 'size' : size} + common_args = {'count' : count, 'interval' : interval, + 'size' : size} ping_conf = PingConf(client = ns1, client_bind = ip1, destination = ns2, @@ -98,45 +141,49 @@ class IpsecEspAeadRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): **common_args) yield [ping_conf]
- def generate_flow_combinations(self, main_config, sub_config): - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns - ip1, ip2 = sub_config.ips + def generate_flow_combinations(self, config): + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns + ip1, ip2 = config.ips for perf_test in self.params.perf_tests: - offload_values = sub_config.offload_settings.values() - offload_items = sub_config.offload_settings.items() - if ((perf_test == 'udp_stream' and ('gro', 'off') in offload_items) + offload_values = config.offload_settings.values() + offload_items = config.offload_settings.items() + if ((perf_test == 'udp_stream' and ('gro', 'off') in + offload_items) or - (perf_test == 'sctp_stream' and 'off' in offload_values and - ('gso', 'on') in offload_items)): + (perf_test == 'sctp_stream' and 'off' in + offload_values and ('gso', 'on') in offload_items)): continue
for size in self.params.perf_msg_sizes: flow = PerfFlow( - type = perf_test, - generator = ns1, - generator_bind = ip1, - receiver = ns2, - receiver_bind = ip2, - msg_size = size, - duration = self.params.perf_duration, - parallel_streams = self.params.perf_parallel_streams, - cpupin = self.params.perf_tool_cpu if "perf_tool_cpu" in self.params else None - ) + type = perf_test, + generator = ns1, + generator_bind = ip1, + receiver = ns2, + receiver_bind = ip2, + msg_size = size, + duration = self.params.perf_duration, + parallel_streams = self.params.perf_parallel_streams, + cpupin = self.params.perf_tool_cpu if ( + "perf_tool_cpu" in self.params) else None + ) yield [flow]
- if "perf_reverse" in self.params and self.params.perf_reverse: + if ("perf_reverse" in self.params and + self.params.perf_reverse): reverse_flow = self._create_reverse_flow(flow) yield [reverse_flow]
def ping_test(self, ping_config): m1, m2 = ping_config[0].client, ping_config[0].destination - ip1, ip2 = ping_config[0].client_bind, ping_config[0].destination_address + ip1, ip2 = (ping_config[0].client_bind, + ping_config[0].destination_address) if1_name = self.get_dev_by_ip(m1, ip1).name if2 = self.get_dev_by_ip(m2, ip2)
pa_kwargs = {} pa_kwargs["p_filter"] = "esp" - pa_kwargs["grep_for"] = ["ESP(spi=" + self.spi_values[1]] + pa_kwargs["grep_for"] = ['ESP(spi=' + self.spi_values[1]] if ping_config[0].count: pa_kwargs["p_min"] = ping_config[0].count pa_config = PacketAssertConf(m2, if2, **pa_kwargs) @@ -144,7 +191,7 @@ class IpsecEspAeadRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): dump = m1.run("tcpdump -i %s -nn -vv" % if1_name, bg=True) self.packet_assert_test_start(pa_config) self.ctl.wait(2) - ping_result = super(IpsecEspAeadRecipe, self).ping_test(ping_config) + ping_result = super().ping_test(ping_config) self.ctl.wait(2) pa_result = self.packet_assert_test_stop() dump.kill(signal=signal.SIGINT) @@ -152,7 +199,7 @@ class IpsecEspAeadRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): return (ping_result, pa_config, pa_result)
def ping_evaluate_and_report(self, ping_config, result): - super(IpsecEspAeadRecipe, self).ping_evaluate_and_report(ping_config, result[0]) + super().ping_evaluate_and_report(ping_config, result[0]) self.packet_assert_evaluate_and_report(result[1], result[2])
def get_dev_by_ip(self, netns, ip): @@ -162,57 +209,18 @@ class IpsecEspAeadRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): raise LnstError("Could not match ip %s to any device of %s." % (ip, netns.name))
- def test_wide_configuration(self): - host1, host2 = self.matched.host1, self.matched.host2 - - for host in [host1, host2]: - host.eth0.down() - - net_addr = "192.168." - net_addr6 = "fc00:" - - for i, host in enumerate([host1, host2]): - host.eth0.ip_add(ipaddress(net_addr + str(i+99) + ".1/24")) - host.eth0.ip_add(ipaddress(net_addr6 + str(i+1) + "::1/64")) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.eth0 - configuration.endpoint2 = host2.eth0 - - if self.params.ping_parallel or self.params.ping_bidirect: - logging.debug("Parallelism in pings is not supported for this recipe, " - "ping_parallel/bidirect will be ignored.") - - if "mtu" in self.params: - for host in [host1, host2]: - host.eth0.mtu = self.params.mtu - - for host in [host1, host2]: - host.eth0.up() - - for host, dst in [(host1, host2), (host2, host1)]: - for family in [AF_INET, AF_INET6]: - host.run("ip route add %s dev %s" % - (dst.eth0.ips_filter(family=family)[0], host.eth0.name)) - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + @property + def offload_nics(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- return configuration + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- def test_wide_deconfiguration(self, config): - host1, host2 = self.matched.host1, self.matched.host2 + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py b/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py index 70a52cd..4f53979 100644 --- a/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py +++ b/lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py @@ -1,44 +1,27 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(ipsec_esp_ah_comp.xml + ipsec_esp_ah_comp.py) -""" import signal import logging -from copy import copy +import copy from lnst.Common.IpAddress import ipaddress from lnst.Common.IpAddress import AF_INET, AF_INET6 from lnst.Common.Parameters import StrParam from lnst.Common.LnstError import LnstError from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration, EnrtSubConfiguration -from lnst.RecipeCommon.PacketAssert import PacketAssertConf, PacketAssertTestAndEvaluate +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.BaseSubConfigMixin import ( + BaseSubConfigMixin as ConfMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) +from lnst.RecipeCommon.PacketAssert import (PacketAssertConf, + PacketAssertTestAndEvaluate) from lnst.RecipeCommon.Perf.Measurements import Flow as PerfFlow from lnst.RecipeCommon.Ping import PingConf -from lnst.Recipes.ENRT.XfrmTools import configure_ipsec_esp_ah_comp, generate_key +from lnst.Recipes.ENRT.XfrmTools import (configure_ipsec_esp_ah_comp, + generate_key)
-class IpsecEnrtSubConfiguration(EnrtSubConfiguration): - def __init__(self): - super(IpsecEnrtSubConfiguration, self).__init__() - self._ips = () - self._ipsec_settings = () - - @property - def ipsec_settings(self): - return self._ipsec_settings - - @ipsec_settings.setter - def ipsec_settings(self, value): - self._ipsec_settings = value - - @property - def ips(self): - return self._ips - - @ips.setter - def ips(self, value): - self._ips = value - -class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): +class IpsecEspAhCompRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe, PacketAssertTestAndEvaluate): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver"))
@@ -50,51 +33,111 @@ class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): spi_values = ["0x00000001", "0x00000002", "0x00000003", "0x00000004"] ipsec_mode = StrParam(default="transport")
- def generate_sub_configurations(self, main_config): + def test_wide_configuration(self): + host1, host2 = self.matched.host1, self.matched.host2 + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host2.eth0] + + net_addr = "192.168." + net_addr6 = "fc00:" + for i, host in enumerate([host1, host2]): + host.eth0.down() + host.eth0.ip_add(ipaddress(net_addr + str(i+99) + ".1/24")) + host.eth0.ip_add(ipaddress(net_addr6 + str(i+1) + "::1/64")) + host.eth0.up() + + self.wait_tentative_ips(configuration.test_wide_devices) + + if self.params.ping_parallel or self.params.ping_bidirect: + logging.debug("Parallelism in pings is not supported for this" + "recipe, ping_parallel/ping_bidirect will be ignored.") + + for host, dst in [(host1, host2), (host2, host1)]: + for family in [AF_INET, AF_INET6]: + host.run("ip route add %s dev %s" % + (dst.eth0.ips_filter(family=family)[0], + host.eth0.name)) + + configuration.endpoint1 = host1.eth0 + configuration.endpoint2 = host2.eth0 + + return configuration + + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]) + ] + return desc + + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + def generate_sub_configurations(self, config): ipsec_mode = self.params.ipsec_mode spi_values = self.spi_values - for offload_settings in self.params.offload_combinations: - for ipv in self.params.ip_versions: - if ipv == "ipv4": - family = AF_INET - elif ipv == "ipv6": - family = AF_INET6 - - ip1 = main_config.endpoint1.ips_filter(family=family)[0] - ip2 = main_config.endpoint2.ips_filter(family=family)[0] - - for ciph_alg, ciph_len in self.ciphers: - for hash_alg, hash_len in self.hashes: - ciph_key = generate_key(ciph_len) - hash_key = generate_key(hash_len) - sub_config = IpsecEnrtSubConfiguration() - sub_config.offload_settings = offload_settings - sub_config.ips = (ip1, ip2) - sub_config.ipsec_settings = (ciph_alg, ciph_key, hash_alg, - hash_key, ipsec_mode, spi_values) - yield sub_config - - def apply_sub_configuration(self, main_config, sub_config): - super(IpsecEspAhCompRecipe, self).apply_sub_configuration(main_config, sub_config) - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns - ip1, ip2 = sub_config.ips - ipsec_sets = sub_config.ipsec_settings + for subconf in ConfMixin.generate_sub_configurations(self, config): + for offload_settings in self.params.offload_combinations: + for ipv in self.params.ip_versions: + if ipv == "ipv4": + family = AF_INET + elif ipv == "ipv6": + family = AF_INET6 + + ip1 = config.endpoint1.ips_filter(family=family)[0] + ip2 = config.endpoint2.ips_filter(family=family)[0] + + for ciph_alg, ciph_len in self.ciphers: + for hash_alg, hash_len in self.hashes: + ciph_key = generate_key(ciph_len) + hash_key = generate_key(hash_len) + new_config = copy.copy(subconf) + new_config.offload_settings = offload_settings + new_config.ips = (ip1, ip2) + new_config.ipsec_settings = (ciph_alg, + ciph_key, hash_alg, hash_key, ipsec_mode, + spi_values) + yield new_config + + def apply_sub_configuration(self, config): + super().apply_sub_configuration(config) + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns + ip1, ip2 = config.ips + ipsec_sets = config.ipsec_settings configure_ipsec_esp_ah_comp(ns1, ip1, ns2, ip2, *ipsec_sets)
- def remove_sub_configuration(self, main_config, sub_config): - super(IpsecEspAhCompRecipe, self).remove_sub_configuration(main_config, sub_config) - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns + def remove_sub_configuration(self, config): + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns for ns in (ns1, ns2): ns.run("ip xfrm policy flush") ns.run("ip xfrm state flush") + super().remove_sub_configuration(config)
- def generate_ping_configurations(self, main_config, sub_config): - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns - ip1, ip2 = sub_config.ips + def generate_ping_configurations(self, config): + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns + ip1, ip2 = config.ips count = self.params.ping_count interval = self.params.ping_interval size = self.params.ping_psize - common_args = {'count' : count, 'interval' : interval, 'size' : size} + common_args = {'count' : count, 'interval' : interval, + 'size' : size} ping_conf = PingConf(client = ns1, client_bind = ip1, destination = ns2, @@ -102,45 +145,50 @@ class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): **common_args) yield [ping_conf]
- def generate_flow_combinations(self, main_config, sub_config): - ns1, ns2 = main_config.endpoint1.netns, main_config.endpoint2.netns - ip1, ip2 = sub_config.ips + def generate_flow_combinations(self, config): + ns1, ns2 = config.endpoint1.netns, config.endpoint2.netns + ip1, ip2 = config.ips for perf_test in self.params.perf_tests: - offload_values = sub_config.offload_settings.values() - offload_items = sub_config.offload_settings.items() - if ((perf_test == 'udp_stream' and ('gro', 'off') in offload_items) + offload_values = config.offload_settings.values() + offload_items = config.offload_settings.items() + if ((perf_test == 'udp_stream' and ('gro', 'off') in + offload_items) or (perf_test == 'sctp_stream' and 'off' in offload_values and - ('gso', 'on') in offload_items)): + ('gso', 'on') in offload_items)): continue
for size in self.params.perf_msg_sizes: flow = PerfFlow( - type = perf_test, - generator = ns1, - generator_bind = ip1, - receiver = ns2, - receiver_bind = ip2, - msg_size = size, - duration = self.params.perf_duration, - parallel_streams = self.params.perf_parallel_streams, - cpupin = self.params.perf_tool_cpu if "perf_tool_cpu" in self.params else None - ) + type = perf_test, + generator = ns1, + generator_bind = ip1, + receiver = ns2, + receiver_bind = ip2, + msg_size = size, + duration = self.params.perf_duration, + parallel_streams = self.params.perf_parallel_streams, + cpupin = self.params.perf_tool_cpu if ( + "perf_tool_cpu" in self.params) else None + ) yield [flow]
- if "perf_reverse" in self.params and self.params.perf_reverse: + if ("perf_reverse" in self.params and + self.params.perf_reverse): reverse_flow = self._create_reverse_flow(flow) yield [reverse_flow]
def ping_test(self, ping_config): m1, m2 = ping_config[0].client, ping_config[0].destination - ip1, ip2 = ping_config[0].client_bind, ping_config[0].destination_address + ip1, ip2 = (ping_config[0].client_bind, + ping_config[0].destination_address) if1_name = self.get_dev_by_ip(m1, ip1).name if2 = self.get_dev_by_ip(m2, ip2)
pa_kwargs = {} pa_kwargs["p_filter"] = "ah" - pa_kwargs["grep_for"] = ["AH(spi=" + self.spi_values[2], "ESP(spi=" + self.spi_values[1]] + pa_kwargs["grep_for"] = ["AH(spi=" + self.spi_values[2], + "ESP(spi=" + self.spi_values[1]] if ping_config[0].count: pa_kwargs["p_min"] = 2 * ping_config[0].count pa_config = PacketAssertConf(m2, if2, **pa_kwargs) @@ -148,7 +196,7 @@ class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): dump = m1.run("tcpdump -i %s -nn -vv" % if1_name, bg=True) self.packet_assert_test_start(pa_config) self.ctl.wait(2) - ping_result = super(IpsecEspAhCompRecipe, self).ping_test(ping_config) + ping_result = super().ping_test(ping_config) self.ctl.wait(2) pa_result = self.packet_assert_test_stop() dump.kill(signal=signal.SIGINT) @@ -158,16 +206,18 @@ class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate):
dump2 = m1.run("tcpdump -i %s -nn -vv" % if1_name, bg=True) no_trans = self.params.ipsec_mode != 'transport' + ping_config2 = copy.copy(ping_config) + ping_config2[0].size = 1500 if no_trans: - pa_kwargs2 = copy(pa_kwargs) + pa_kwargs2 = copy.copy(pa_kwargs) pa_kwargs2["p_filter"] = '' pa_kwargs2["grep_for"] = ["IPComp"] + if ping_config2[0].count: + pa_kwargs2["p_min"] = ping_config2[0].count pa_config2 = PacketAssertConf(m2, if2, **pa_kwargs2) self.packet_assert_test_start(pa_config2) self.ctl.wait(2) - ping_config2 = copy(ping_config) - ping_config2[0].size = 1500 - ping_result2 = super(IpsecEspAhCompRecipe, self).ping_test(ping_config2) + ping_result2 = super().ping_test(ping_config2) self.ctl.wait(2) if no_trans: pa_result2 = self.packet_assert_test_stop() @@ -180,7 +230,7 @@ class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate):
def ping_evaluate_and_report(self, ping_config, result): for res in result: - super(IpsecEspAhCompRecipe, self).ping_evaluate_and_report(ping_config, res[0]) + super().ping_evaluate_and_report(ping_config, res[0]) self.packet_assert_evaluate_and_report(res[1], res[2])
def get_dev_by_ip(self, netns, ip): @@ -188,59 +238,20 @@ class IpsecEspAhCompRecipe(BaseEnrtRecipe, PacketAssertTestAndEvaluate): if ip in dev.ips: return dev raise LnstError("Could not match ip %s to any device of %s." % - (ip, netns.name)) - - def test_wide_configuration(self): - host1, host2 = self.matched.host1, self.matched.host2 - - for host in [host1, host2]: - host.eth0.down() - - net_addr = "192.168." - net_addr6 = "fc00:" - - for i, host in enumerate([host1, host2]): - host.eth0.ip_add(ipaddress(net_addr + str(i+99) + ".1/24")) - host.eth0.ip_add(ipaddress(net_addr6 + str(i+1) + "::1/64")) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.eth0 - configuration.endpoint2 = host2.eth0 - - if self.params.ping_parallel or self.params.ping_bidirect: - logging.debug("Parallelism in pings is not supported for this recipe, " - "ping_parallel/bidirect will be ignored.") - - if "mtu" in self.params: - for host in [host1, host2]: - host.eth0.mtu = self.params.mtu - - for host in [host1, host2]: - host.eth0.up() - - for host, dst in [(host1, host2), (host2, host1)]: - for family in [AF_INET, AF_INET6]: - host.run("ip route add %s dev %s" % - (dst.eth0.ips_filter(family=family)[0], host.eth0.name)) - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) + (ip, netns.name))
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + @property + def offload_nics(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- return configuration + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- def test_wide_deconfiguration(self, config): - host1, host2 = self.matched.host1, self.matched.host2 + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py b/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py index 31e2635..7d1ee7c 100644 --- a/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py +++ b/lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py @@ -1,13 +1,14 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(novirt_ovs_vxlan.xml + novirt_ovs_vxlan.py) -""" from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import OvsBridgeDevice
-class NoVirtOvsVxlanRecipe(BaseEnrtRecipe): +class NoVirtOvsVxlanRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver"))
@@ -17,60 +18,105 @@ class NoVirtOvsVxlanRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- for host in [host1, host2]: - host.eth0.down() - net_addr = "192.168.2" vxlan_net_addr = "192.168.100" vxlan_net_addr6 = "fc00:0:0:0"
flow_entries=[] - flow_entries.append("table=0,in_port=5,actions=set_field:100->tun_id,output:10") - flow_entries.append("table=0,in_port=10,tun_id=100,actions=output:5") + flow_entries.append("table=0,in_port=5,actions=set_field:100->" + "tun_id,output:10") + flow_entries.append("table=0,in_port=10,tun_id=100,actions=" + "output:5") flow_entries.append("table=0,priority=100,actions=drop")
for i, host in enumerate([host1, host2]): - host.eth0.ip_add(ipaddress(net_addr + "." + str (i+1) + "/24")) + host.eth0.down() + host.eth0.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) host.br0 = OvsBridgeDevice() - host.int0 = host.br0.internal_port_add(ofport_request='5', name="int0") - host.int0.ip_add(ipaddress(vxlan_net_addr + "." + str (i+1) + "/24")) - host.int0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str (i+1) + "/64")) - tunnel_opts = {"option:remote_ip" : net_addr + "." + str (2-i), "option:key" : "flow", - "ofport_request" : "10"} + host.int0 = host.br0.internal_port_add(ofport_request='5', + name="int0") + host.int0.ip_add(ipaddress(vxlan_net_addr + "." + str(i+1) + + "/24")) + host.int0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str(i+1) + + "/64")) + tunnel_opts = {"option:remote_ip" : net_addr + "." + str(2-i), + "option:key" : "flow", "ofport_request" : "10"} host.br0.tunnel_add("vxlan", tunnel_opts) host.br0.flows_add(flow_entries) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.int0 - configuration.endpoint2 = host2.int0 - - if "mtu" in self.params: - for host in [host1, host2]: - host.int0.mtu = self.params.mtu - - for host in [host1, host2]: host.eth0.up() host.int0.up() host.br0.up()
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host1.int0, + host2.eth0, host2.int0]
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): + def generate_test_wide_description(self, config): host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.internal_ports = {}".format( + dev.host.hostid, dev.name, dev.internal_ports + ) + for dev in [host1.br0, host2.br0] + ]), + "\n".join([ + "Configured {}.{}.tunnels = {}".format( + dev.host.hostid, dev.name, dev.tunnels + ) + for dev in [host1.br0, host2.br0] + ]), + "\n".join([ + "Configured {}.{}.flows = {}".format( + dev.host.hostid, dev.name, dev.flows + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc + + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.int0, self.matched.host2.int0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.int0, self.matched.host2.int0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.int0, self.matched.host2.int0] + + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.int0, self.matched.host2.int0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/PingFloodRecipe.py b/lnst/Recipes/ENRT/PingFloodRecipe.py index 0b0c574..bb04e16 100644 --- a/lnst/Recipes/ENRT/PingFloodRecipe.py +++ b/lnst/Recipes/ENRT/PingFloodRecipe.py @@ -1,9 +1,3 @@ -#!/bin/python3 -""" -Implements scenario similar to regression_tests/phase1/ -(ping_flood.xml + simple_ping.py). -""" - from lnst.Common.Parameters import Param, IntParam, StrParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam @@ -17,12 +11,12 @@ class PingFloodRecipe(PingTestAndEvaluate): host2 = HostReq() host2.eth0 = DeviceReq(label="net1", driver=RecipeParam("driver"))
- #TODO: use a parameter for the network src_addr = StrParam(default = "192.168.1.1/24") dst_addr = StrParam(default = "192.168.1.2/24") count = IntParam(default = 100) interval = StrParam(default = 0.2) - size = IntParam(default = None) + size = IntParam(mandatory = False) + mtu = IntParam(mandatory = False)
def test(self): host1, host2 = self.matched.host1, self.matched.host2 @@ -37,13 +31,16 @@ class PingFloodRecipe(PingTestAndEvaluate): host1.eth0.up() host2.eth0.up()
- if1 = host1.eth0 + ip1 = host1.eth0.ips[0] ip2 = host2.eth0.ips[0] cn = self.params.count iv = self.params.interval - sz = self.params.size - - pcfg=PingConf(host1, if1, host2, ip2, count = cn, interval = iv, size = sz or None) + if "size" in self.params: + sz = self.params.size + else: + sz = None
+ pcfg=PingConf(host1, ip1, host2, ip2, count = cn, interval = iv, + size = sz) result = self.ping_test([pcfg]) self.ping_evaluate_and_report(pcfg, result) diff --git a/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py b/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py index 422e68b..be0ddce 100644 --- a/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py +++ b/lnst/Recipes/ENRT/ShortLivedConnectionsRecipe.py @@ -1,13 +1,14 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(short_lived_connections.xml + short_lived_connections.py) -""" from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe from lnst.Common.Parameters import Param, IntParam, ListParam +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin)
-class ShortLivedConnectionsRecipe(BaseEnrtRecipe): +class ShortLivedConnectionsRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver"))
@@ -22,46 +23,60 @@ class ShortLivedConnectionsRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- for host in [host1, host2]: - host.eth0.down() + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host2.eth0]
net_addr = "192.168.101" - for i, host in enumerate([host1, host2], 10): + host.eth0.down() host.eth0.ip_add(ipaddress(net_addr + "." + str(i) + "/24")) + host.eth0.up()
- #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.eth0 - configuration.endpoint2 = host2.eth0 + self.wait_tentative_ips(configuration.test_wide_devices)
- if "mtu" in self.params: - for host in [host1, host2]: - host.eth0.mtu = self.params.mtu + return configuration
- for host in [host1, host2]: - host.eth0.up() + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) + def test_wide_deconfiguration(self, config): + del config.test_wide_devices
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + super().test_wide_deconfiguration(config)
- return configuration + def generate_perf_endpoints(self, config): + return [(self.matched.host1.eth0, self.matched.host2.eth0)]
- def test_wide_deconfiguration(self, config): - host1, host2 = self.matched.host1, self.matched.host2 + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- def generate_ping_configurations(self, main_config, sub_config): - return [] + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/SimpleMacsecRecipe.py b/lnst/Recipes/ENRT/SimpleMacsecRecipe.py index 47ab907..fb78ebb 100644 --- a/lnst/Recipes/ENRT/SimpleMacsecRecipe.py +++ b/lnst/Recipes/ENRT/SimpleMacsecRecipe.py @@ -1,63 +1,23 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(simple_macsec.xml + simple_macsec.py) -""" import logging +import copy from lnst.Common.IpAddress import ipaddress from lnst.Common.IpAddress import AF_INET, AF_INET6 from lnst.Common.LnstError import LnstError from lnst.Controller import HostReq, DeviceReq, RecipeParam from lnst.Devices import MacsecDevice -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration, EnrtSubConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.BaseSubConfigMixin import ( + BaseSubConfigMixin as ConfMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.RecipeCommon.Perf.Recipe import RecipeConf as PerfRecipeConf from lnst.RecipeCommon.Perf.Measurements import Flow as PerfFlow from lnst.RecipeCommon.Ping import PingConf
-class MacsecEnrtConfiguration(EnrtConfiguration): - def __init__(self): - super(MacsecEnrtConfiguration, self).__init__() - self._host1 = None - self._host2 = None - - @property - def host1(self): - return self._host1 - - @host1.setter - def host1(self, value): - self._host1 = value - - @property - def host2(self): - return self._host2 - - @host2.setter - def host2(self, value): - self._host2 = value - -class MacsecEnrtSubConfiguration(EnrtSubConfiguration): - def __init__(self): - super(MacsecEnrtSubConfiguration, self).__init__() - self._ip_vers = ('ipv4',) - self._encrypt = None - - @property - def encrypt(self): - return self._encrypt - - @encrypt.setter - def encrypt(self, value): - self._encrypt = value - - @property - def ip_vers(self): - return self._ip_vers - - @ip_vers.setter - def ip_vers(self, value): - self._ip_vers = value - -class SimpleMacsecRecipe(BaseEnrtRecipe): +class SimpleMacsecRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver"))
@@ -66,75 +26,89 @@ class SimpleMacsecRecipe(BaseEnrtRecipe):
macsec_settings = [None, 'on', 'off'] ids = ['00', '01'] - keys = ["7a16780284000775d4f0a3c0f0e092c0", "3212ef5c4cc5d0e4210b17208e88779e"] + keys = ["7a16780284000775d4f0a3c0f0e092c0", + "3212ef5c4cc5d0e4210b17208e88779e"]
def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- for host in [host1, host2]: - host.eth0.down() + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host2.eth0]
net_addr = "192.168.0" - for i, host in enumerate([host1, host2]): + host.eth0.down() host.eth0.ip_add(ipaddress(net_addr + '.' + str(i+1) + "/24"))
- #Due to limitations in the current EnrtConfiguration - #class, a single test pair is chosen - configuration = EnrtConfiguration() + self.wait_tentative_ips(configuration.test_wide_devices) + + if (self.params.ping_parallel or self.params.ping_bidirect or + self.params.perf_reverse): + logging.debug("Parallel pings or reverse perf tests are " + "not supported for this recipe, ping_parallel" + "/ping_bidirect/perf_reverse will be ignored.") + configuration.endpoint1 = host1.eth0 configuration.endpoint2 = host2.eth0 configuration.host1 = host1 configuration.host2 = host2
- if (self.params.ping_parallel or self.params.ping_bidirect or - self.params.perf_reverse): - logging.debug("Parallelism in pings or reverse perf tests are " - "not supported for this recipe, ping_parallel/bidirect " - " or perf_reverse will be ignored.") + return configuration
- if "mtu" in self.params: - for host in [host1, host2]: - host.eth0.mtu = self.params.mtu + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) + def test_wide_deconfiguration(self, config): + del config.test_wide_devices
- return configuration + super().test_wide_deconfiguration(config)
- def test_wide_deconfiguration(self, config): - host1, host2 = self.matched.host1, self.matched.host2 + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + )
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") - - def generate_sub_configurations(self, main_config): - for encryption in self.macsec_settings: - sub_config = MacsecEnrtSubConfiguration() - sub_config.encrypt = encryption - if encryption is not None: - sub_config.ip_vers = self.params.ip_versions - yield sub_config - - def apply_sub_configuration(self, main_config, sub_config): - if not sub_config.encrypt: - main_config.endpoint1.up() - main_config.endpoint2.up() + self.ctl.wait_for_condition(condition, timeout=5) + + def generate_sub_configurations(self, config): + for subconf in ConfMixin.generate_sub_configurations(self, config): + for offload_settings in self.params.offload_combinations: + for encryption in self.macsec_settings: + new_config = copy.copy(subconf) + new_config.offload_settings = offload_settings + new_config.encrypt = encryption + if encryption is not None: + new_config.ip_vers = self.params.ip_versions + yield new_config + + def apply_sub_configuration(self, config): + if not config.encrypt: + config.endpoint1.up() + config.endpoint2.up() else: net_addr = "192.168.100" net_addr6 = "fc00:0:0:0" - host1, host2 = main_config.host1, main_config.host2 - k_ids = zip(self.ids, self.keys) - hosts_and_keys = [(host1, host2, k_ids), (host2, host1, k_ids[::-1])] + host1, host2 = config.host1, config.host2 + k_ids = list(zip(self.ids, self.keys)) + hosts_and_keys = [(host1, host2, k_ids), (host2, host1, + k_ids[::-1])] for host_a, host_b, k_ids in hosts_and_keys: - host_a.msec0 = MacsecDevice(realdev=host_a.eth0, encrypt=sub_config.encrypt) + host_a.msec0 = MacsecDevice(realdev=host_a.eth0, + encrypt=config.encrypt) rx_kwargs = dict(port=1, address=host_b.eth0.hwaddr) - tx_sa_kwargs = dict(sa=0, pn=1, enable='on', id=k_ids[0][0], key=k_ids[0][1]) + tx_sa_kwargs = dict(sa=0, pn=1, enable='on', + id=k_ids[0][0], key=k_ids[0][1]) rx_sa_kwargs = rx_kwargs.copy() rx_sa_kwargs.update(tx_sa_kwargs) rx_sa_kwargs['id'] = k_ids[1][0] @@ -143,34 +117,37 @@ class SimpleMacsecRecipe(BaseEnrtRecipe): host_a.msec0.tx_sa('add', **tx_sa_kwargs) host_a.msec0.rx_sa('add', **rx_sa_kwargs) for i, host in enumerate([host1, host2]): - host.msec0.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) - host.msec0.ip_add(ipaddress(net_addr6 + "::" + str(i+1) + "/64")) + host.msec0.ip_add(ipaddress(net_addr + "." + str(i+1) + + "/24")) + host.msec0.ip_add(ipaddress(net_addr6 + "::" + str(i+1) + + "/64")) host.eth0.up() host.msec0.up()
- def remove_sub_configuration(self, main_config, sub_config): - if sub_config.encrypt: - host1, host2 = main_config.host1, main_config.host2 + def remove_sub_configuration(self, config): + if config.encrypt: + host1, host2 = config.host1, config.host2 for host in (host1, host2): host.msec0.destroy() del host.msec0 - main_config.endpoint1.down() - main_config.endpoint2.down() + config.endpoint1.down() + config.endpoint2.down()
- def generate_ping_configurations(self, main_config, sub_config): - if not sub_config.encrypt: - client_nic = main_config.endpoint1 - server_nic = main_config.endpoint2 + def generate_ping_configurations(self, config): + if not config.encrypt: + client_nic = config.endpoint1 + server_nic = config.endpoint2 ip_vers = ('ipv4',) else: - client_nic = main_config.host1.msec0 - server_nic = main_config.host2.msec0 + client_nic = config.host1.msec0 + server_nic = config.host2.msec0 ip_vers = self.params.ip_versions
count = self.params.ping_count interval = self.params.ping_interval size = self.params.ping_psize - common_args = {'count' : count, 'interval' : interval, 'size' : size} + common_args = {'count' : count, 'interval' : interval, + 'size' : size}
for ipv in ip_vers: kwargs = {} @@ -186,8 +163,10 @@ class SimpleMacsecRecipe(BaseEnrtRecipe): client_ips = client_ips[::-1] server_ips = server_ips[::-1]
- if len(client_ips) != len(server_ips) or len(client_ips) * len(server_ips) == 0: - raise LnstError("Source/destination ip lists are of different size or empty.") + if len(client_ips) != len(server_ips) or (len(client_ips) * + len(server_ips) == 0): + raise LnstError("Source/destination ip lists are of " + "different size or empty.")
for src_addr, dst_addr in zip(client_ips, server_ips): pconf = PingConf(client = client_nic.netns, @@ -198,21 +177,20 @@ class SimpleMacsecRecipe(BaseEnrtRecipe):
yield [pconf]
- def generate_perf_configurations(self, main_config, sub_config): - if sub_config.encrypt: - client_nic = main_config.host1.msec0 - server_nic = main_config.host2.msec0 + def generate_perf_configurations(self, config): + if config.encrypt: + client_nic = config.host1.msec0 + server_nic = config.host2.msec0 client_netns = client_nic.netns server_netns = server_nic.netns
flow_combinations = self.generate_flow_combinations( - main_config, sub_config + config )
for flows in flow_combinations: perf_recipe_conf=dict( - main_config=main_config, - sub_config=sub_config, + recipe_config=config, flows=flows, )
@@ -240,9 +218,9 @@ class SimpleMacsecRecipe(BaseEnrtRecipe):
yield perf_conf
- def generate_flow_combinations(self, main_config, sub_config): - client_nic = main_config.host1.msec0 - server_nic = main_config.host2.msec0 + def generate_flow_combinations(self, config): + client_nic = config.host1.msec0 + server_nic = config.host2.msec0 client_netns = client_nic.netns server_netns = server_nic.netns
@@ -257,15 +235,29 @@ class SimpleMacsecRecipe(BaseEnrtRecipe):
for perf_test in self.params.perf_tests: for size in self.params.perf_msg_sizes: + pstreams = self.params.perf_parallel_streams flow = PerfFlow( - type = perf_test, - generator = client_netns, - generator_bind = client_bind, - receiver = server_netns, - receiver_bind = server_bind, - msg_size = size, - duration = self.params.perf_duration, - parallel_streams = self.params.perf_parallel_streams, - cpupin = self.params.perf_tool_cpu if "perf_tool_cpu" in self.params else None + type = perf_test, + generator = client_netns, + generator_bind = client_bind, + receiver = server_netns, + receiver_bind = server_bind, + msg_size = size, + duration = self.params.perf_duration, + parallel_streams = pstreams, + cpupin = self.params.perf_tool_cpu if ( + "perf_tool_cpu" in self.params) else None ) yield [flow] + + @property + def offload_nics(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/SimplePerfRecipe.py b/lnst/Recipes/ENRT/SimplePerfRecipe.py index ca1e9b9..ebf2ff2 100644 --- a/lnst/Recipes/ENRT/SimplePerfRecipe.py +++ b/lnst/Recipes/ENRT/SimplePerfRecipe.py @@ -1,11 +1,7 @@ -from lnst.Common.LnstError import LnstError -from lnst.Common.Parameters import IntParam, Param, StrParam, BoolParam -from lnst.Common.IpAddress import ipaddress, AF_INET, AF_INET6 - +from lnst.Common.Parameters import Param +from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam - from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe - from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( OffloadSubConfigMixin, ) @@ -13,7 +9,6 @@ from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( CommonHWConfigMixin, )
- class SimplePerfRecipe( OffloadSubConfigMixin, CommonHWConfigMixin, BaseEnrtRecipe ): @@ -35,17 +30,13 @@ class SimplePerfRecipe( configuration = super().test_wide_configuration() configuration.test_wide_devices = []
- host1.eth0.ip_add(ipaddress("192.168.101.1/24")) - host1.eth0.ip_add(ipaddress("fc00::1/64")) - host1.eth0.up() - configuration.test_wide_devices.append(host1.eth0) + for i, host in enumerate([host1, host2]): + host.eth0.ip_add(ipaddress("192.168.101." + str(i+1) + "/24")) + host.eth0.ip_add(ipaddress("fc00::" + str(i+1) + "/64")) + host.eth0.up() + configuration.test_wide_devices.append(host.eth0)
- host2.eth0.ip_add(ipaddress("192.168.101.2/24")) - host2.eth0.ip_add(ipaddress("fc00::2/64")) - host2.eth0.up() - configuration.test_wide_devices.append(host2.eth0) - - self.wait_tentative_ips([host1.eth0, host2.eth0]) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
@@ -64,9 +55,6 @@ class SimplePerfRecipe(
super().test_wide_deconfiguration(config)
- def generate_ping_endpoints(self, config): - return [(self.matched.host1.eth0, self.matched.host2.eth0)] - def generate_perf_endpoints(self, config): return [(self.matched.host1.eth0, self.matched.host2.eth0)]
@@ -83,5 +71,17 @@ class SimplePerfRecipe( return [self.matched.host1.eth0, self.matched.host2.eth0]
@property - def hw_config_dev_list(self): + def mtu_hw_config_dev_list(self): return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def coalescing_hw_config_dev_list(self): + return self.mtu_hw_config_dev_list + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.mtu_hw_config_dev_list + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.mtu_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/TeamRecipe.py b/lnst/Recipes/ENRT/TeamRecipe.py index 93d1c05..d6e58ec 100644 --- a/lnst/Recipes/ENRT/TeamRecipe.py +++ b/lnst/Recipes/ENRT/TeamRecipe.py @@ -1,14 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase2/ -({active_backup,round_robin}_team.xml + team_test.py) -""" -from lnst.Common.Parameters import Param, IntParam, StrParam, BoolParam +from lnst.Common.Parameters import Param, StrParam, BoolParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import TeamDevice
-class TeamRecipe(BaseEnrtRecipe): +class TeamRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="tnet", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="tnet", driver=RecipeParam("driver")) @@ -28,60 +29,91 @@ class TeamRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- host1.eth0.down() - host1.eth1.down() - #The config argument needs to be used with a team device normally (e.g to specify - #the runner mode), but it is not used here due to a bug in the TeamDevice module + #The config argument needs to be used with a team device normally + #(e.g to specify the runner mode), but it is not used here due to + #a bug in the TeamDevice module host1.team0 = TeamDevice() - host1.team0.slave_add(host1.eth0) - host1.team0.slave_add(host1.eth1)
- #EnrtConfiguration and both-side Netperf config need to be checked - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.team0 - configuration.endpoint2 = host2.eth0 + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.team0, host2.eth0]
- if "mtu" in self.params: - host1.team0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu + for dev in [host1.eth0, host1.eth1]: + dev.down() + host1.team0.slave_add(dev)
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" + for i, dev in enumerate([host1.team0, host2.eth0]): + dev.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + "/24")) + dev.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + "/64"))
- host1.team0.ip_add(ipaddress(net_addr_1 + ".1/24")) - host1.team0.ip_add(ipaddress(net_addr6_1 + "::1/64")) - host2.eth0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.eth0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - - host1.eth0.up() - host1.eth1.up() - host1.team0.up() - host2.eth0.up() - - if "adaptive_rx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host1.eth0, host1.eth1, host2.eth0]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host, dev in [(host1, host1.eth0), (host1, host1.eth1), (host2, host2.eth0)]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + for dev in [host1.eth0, host1.eth1, host1.team0, host2.eth0]: + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "Configured {}.{}.slaves = {}".format( + host1.hostid, host1.team0.name, + ['.'.join([host1.hostid, slave.name]) + for slave in host1.team0.slaves] + ), + "Configured {}.{}.runner_name = {}".format( + host1.hostid, host1.team0.name, + host1.team0.config + ) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.team0, self.matched.host2.eth0), + (self.matched.host2.eth0, self.matched.host1.team0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.team0, self.matched.host2.eth0), + (self.matched.host2.eth0, self.matched.host1.team0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.team0, self.matched.host2.eth0] + + @property + def mtu_hw_config_dev_list(self): + return self.offload_nics + + @property + def coalescing_hw_config_dev_list(self): host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host1.eth1, host2.eth0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/TeamVsBondRecipe.py b/lnst/Recipes/ENRT/TeamVsBondRecipe.py index 5170071..4c8d34d 100644 --- a/lnst/Recipes/ENRT/TeamVsBondRecipe.py +++ b/lnst/Recipes/ENRT/TeamVsBondRecipe.py @@ -1,16 +1,16 @@ -""" -Implements scenario similar to regression_tests/phase2/ -({active_backup,round_robin}_team_vs_{active_backup,round_robin} -_bond.xml + team_test.py) -""" from lnst.Common.Parameters import Param, IntParam, StrParam, BoolParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import TeamDevice from lnst.Devices import BondDevice
-class TeamVsBondRecipe(BaseEnrtRecipe): +class TeamVsBondRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="tnet", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="tnet", driver=RecipeParam("driver")) @@ -33,71 +33,109 @@ class TeamVsBondRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- host1.eth0.down() - host1.eth1.down() - #The config argument needs to be used with a team device normally (e.g to specify - #the runner mode), but it is not used here due to a bug in the TeamDevice module + #The config argument needs to be used with a team device normally + #(e.g to specify the runner mode), but it is not used here due to + #a bug in the TeamDevice module host1.team0 = TeamDevice() - host1.team0.slave_add(host1.eth0) - host1.team0.slave_add(host1.eth1)
- host2.eth0.down() - host2.eth1.down() - host2.bond0 = BondDevice(mode=self.params.bonding_mode, miimon=self.params.miimon_value) - host2.bond0.slave_add(host2.eth0) - host2.bond0.slave_add(host2.eth1) + host2.bond0 = BondDevice(mode=self.params.bonding_mode, + miimon=self.params.miimon_value)
- #EnrtConfiguration and both-side Netperf config need to be checked - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.team0 - configuration.endpoint2 = host2.bond0 - - if "mtu" in self.params: - host1.team0.mtu = self.params.mtu - host2.bond0.mtu = self.params.mtu + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.team0, host2.bond0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1"
- host1.team0.ip_add(ipaddress(net_addr_1 + ".1/24")) - host1.team0.ip_add(ipaddress(net_addr6_1 + "::1/64")) - host2.bond0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.bond0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - - host1.eth0.up() - host1.eth1.up() - host1.team0.up() - host2.eth0.up() - host2.eth1.up() - host2.bond0.up() - - if "adaptive_rx_coalescing" in self.params: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - dev.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host.eth0, host.eth1]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + for i, (host, dev) in enumerate([(host1, host1.team0), (host2, + host2.bond0)]): + host.eth0.down() + host.eth1.down() + dev.slave_add(host.eth0) + dev.slave_add(host.eth1) + dev.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + "/24")) + dev.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + "/64")) + + for host, dev in [(host1, host1.team0), (host2, host2.bond0)]: + host.eth0.up() + host.eth1.up() + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.slaves = {}".format( + dev.host.hostid, dev.name, + ['.'.join([dev.host.hostid, slave.name]) + for slave in dev.slaves] + ) + for dev in config.test_wide_devices + ]), + "Configured {}.{}.runner_name = {}".format( + host1.hostid, host1.team0.name, + host1.team0.config + ), + "Configured {}.{}.mode = {}".format( + host2.hostid, host2.bond0.name, + host2.bond0.mode + ), + "Configured {}.{}.miimon = {}".format( + host2.hostid, host2.bond0.name, + host2.bond0.miimon + ) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.team0, self.matched.host2.bond0), + (self.matched.host2.bond0, self.matched.host1.team0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.team0, self.matched.host2.bond0), + (self.matched.host2.bond0, self.matched.host1.team0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.team0, self.matched.host2.bond0] + + @property + def mtu_hw_config_dev_list(self): + return self.offload_nics + + @property + def coalescing_hw_config_dev_list(self): host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host1.eth1, host2.eth0, host2.eth1] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py b/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py index 3c25bc8..4a9fefb 100644 --- a/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py +++ b/lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py @@ -1,13 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(2_virt_ovs_vxlan.xml + 2_virt_ovs_vxlan.py) -""" +from itertools import combinations from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import OvsBridgeDevice
-class VirtOvsVxlanRecipe(BaseEnrtRecipe): +class VirtOvsVxlanRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest1") @@ -31,8 +33,9 @@ class VirtOvsVxlanRecipe(BaseEnrtRecipe): guest4.eth0 = DeviceReq(label="to_guest4")
def test_wide_configuration(self): - host1, host2, guest1, guest2, guest3, guest4 = self.matched.host1, self.matched.host2,\ - self.matched.guest1, self.matched.guest2, self.matched.guest3, self.matched.guest4 + host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2, + self.matched.guest3, self.matched.guest4)
for host in [host1, host2]: host.eth0.down() @@ -46,62 +49,189 @@ class VirtOvsVxlanRecipe(BaseEnrtRecipe): vxlan_net_addr6 = "fc00:0:0:0"
flow_entries=[] - flow_entries.append("table=0,in_port=5,actions=set_field:100->tun_id,output:10") - flow_entries.append("table=0,in_port=6,actions=set_field:200->tun_id,output:10") - flow_entries.append("table=0,in_port=10,tun_id=100,actions=output:5") - flow_entries.append("table=0,in_port=10,tun_id=200,actions=output:6") + flow_entries.append("table=0,in_port=5,actions=set_field:100->" + "tun_id,output:10") + flow_entries.append("table=0,in_port=6,actions=set_field:200->" + "tun_id,output:10") + flow_entries.append("table=0,in_port=10,tun_id=100,actions=" + "output:5") + flow_entries.append("table=0,in_port=10,tun_id=200,actions=" + "output:6") flow_entries.append("table=0,priority=100,actions=drop")
+ configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host2.eth0, + guest1.eth0, guest2.eth0, guest3.eth0, guest4.eth0] + for i, host in enumerate([host1, host2]): - host.eth0.ip_add(ipaddress(net_addr + "." + str (i+1) + "/24")) + host.eth0.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) host.br0 = OvsBridgeDevice() for dev, ofport_r in [(host.tap0, '5'), (host.tap1, '6')]: - host.br0.port_add(dev, set_iface=True, ofport_request=ofport_r) - tunnel_opts = {"option:remote_ip" : net_addr + "." + str (2-i), "option:key" : "flow", - "ofport_request" : '10'} + host.br0.port_add(dev, set_iface=True, + ofport_request=ofport_r) + tunnel_opts = {"option:remote_ip" : net_addr + "." + str(2-i), + "option:key" : "flow", "ofport_request" : '10'} host.br0.tunnel_add("vxlan", tunnel_opts) host.br0.flows_add(flow_entries) + for dev in [host.eth0, host.tap0, host.tap1, host.br0]: + dev.up()
for i, guest in enumerate([guest1, guest2, guest3, guest4]): - guest.eth0.ip_add(ipaddress(vxlan_net_addr + "." + str (i+1) + "/24")) - guest.eth0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str (i+1) + "/64")) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.eth0 - configuration.endpoint2 = guest3.eth0 - - if "mtu" in self.params: - for guest in [guest1, guest2, guest3, guest4]: - guest.eth0.mtu = self.params.mtu - - for host in [host1, host2]: - host.eth0.up() - host.tap0.up() - host.tap1.up() - host.br0.up() - for guest in [guest1, guest2, guest3, guest4]: + guest.eth0.ip_add(ipaddress(vxlan_net_addr + "." + str(i+1) + + "/24")) + guest.eth0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str(i+1) + + "/64")) guest.eth0.up()
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for machine in [host1, host2, guest1, guest2, guest3, guest4]: - machine.run("service irqbalance stop") - for host in [host1, host2]: - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.ports = {}".format( + dev.host.hostid, dev.name, dev.ports + ) + for dev in [host1.br0, host2.br0] + ]), + "\n".join([ + "Configured {}.{}.tunnels = {}".format( + dev.host.hostid, dev.name, dev.tunnels + ) + for dev in [host1.br0, host2.br0] + ]), + "\n".join([ + "Configured {}.{}.flows = {}".format( + dev.host.hostid, dev.name, dev.flows + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc + def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2, guest3, guest4 = self.matched.host1, self.matched.host2,\ - self.matched.guest1, self.matched.guest2, self.matched.guest3, self.matched.guest4 + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + guest1, guest2, guest3, guest4 = (self.matched.guest1, + self.matched.guest2, self.matched.guest3, self.matched.guest4) + devs = [guest1.eth0, guest2.eth0, guest3.eth0, guest4.eth0] + return combinations(devs, 2) + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest3.eth0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.guest2.eth0, + self.matched.guest3.eth0, self.matched.guest4.eth0] + + @property + def mtu_hw_config_dev_list(self): + return self.offload_nics + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + def do_ping_tests(self, recipe_config): + for ping_config in self.generate_ping_configurations(recipe_config): + exp_fail = [] + for pconf in ping_config: + cond = self.tun_id_match(pconf.client_bind, + pconf.destination_address) + exp_fail.append(cond) + result = self.ping_test(ping_config, exp_fail) + self.ping_evaluate_and_report(ping_config, result) + + def ping_test(self, ping_config, exp_fail): + results = {} + + running_ping_array = [] + for pingconf, fail in zip(ping_config, exp_fail): + ping, client = self.ping_init(pingconf) + running_ping = client.prepare_job(ping, fail=fail) + running_ping.start(bg = True) + running_ping_array.append((pingconf, running_ping)) + + for _, pingjob in running_ping_array: + try: + pingjob.wait() + finally: + pingjob.kill() + + for pingconf, pingjob in running_ping_array: + result = pingjob.result + passed = pingjob.passed + results[pingconf] = (result, passed) + + return results + + def single_ping_evaluate_and_report(self, ping_config, result): + fmt = "From: <{0.client.hostid} ({0.client_bind})> To: " \ + "<{0.destination.hostid} ({0.destination_address})>" + description = fmt.format(ping_config) + if result[0].get("rate", 0) > 50: + message = "Ping successful --- " + description + self.add_result(result[1], message, result[0]) + else: + message = "Ping unsuccessful --- " + description + self.add_result(result[1], message, result[0]) + + def tun_id_match(self, src_addr, dst_addr): + guest1, guest2, guest3, guest4 = (self.matched.guest1, + self.matched.guest2, self.matched.guest3, self.matched.guest4) + + matching_pairs = [] + for pair in [(guest1, guest3), (guest2, guest4)]: + matching_pairs.extend([pair, pair[::-1]]) + + devs = [] + for dev in (guest1.devices + guest2.devices + guest3.devices + + guest4.devices): + if src_addr in dev.ips or dst_addr in dev.ips: + devs.append(dev) + try: + return (devs[0].host, devs[1].host) not in matching_pairs + except IndexError: + return False + + def hw_config(self, config): + host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2, + self.matched.guest3, self.matched.guest4) + + config.hw_config = {} + hw_config = config.hw_config
- #TODO better service handling through HostAPI if "dev_intr_cpu" in self.params: - for machine in [host1, host2, guest1, guest2, guest3, guest4]: - machine.run("service irqbalance start") + intr_cfg = hw_config["dev_intr_cpu_configuration"] = {} + intr_cfg["irq_devs"] = {} + intr_cfg["irqbalance_hosts"] = [] + + for host in [host1, host2, guest1, guest2, guest3, guest4]: + host.run("service irqbalance stop") + intr_cfg["irqbalance_hosts"].append(host) + + for dev in [host1.eth0, host2.eth0]: + self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) + intr_cfg["irq_devs"][dev] = self.params.dev_intr_cpu diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py index 9a2ae93..bed889f 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestMirroredRecipe.py @@ -1,16 +1,17 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(virtual_bridge_vlan_in_guest_mirrored.xml + virtual_bridge_vlan_in_guest_mirrored.py) -""" +import logging from lnst.Common.Parameters import Param from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import BridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualBridgeVlanInGuestMirroredRecipe(BaseEnrtRecipe): +class VirtualBridgeVlanInGuestMirroredRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest1") @@ -33,87 +34,127 @@ class VirtualBridgeVlanInGuestMirroredRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
def test_wide_configuration(self): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2)
- host1.eth0.down() - host1.tap0.down() - host1.br0 = BridgeDevice() - host1.br0.slave_add(host1.eth0) - host1.br0.slave_add(host1.tap0) - - host2.eth0.down() - host2.tap0.down() - host2.br0 = BridgeDevice() - host2.br0.slave_add(host2.eth0) - host2.br0.slave_add(host2.tap0) + for host in [host1, host2]: + host.br0 = BridgeDevice() + for dev in [host.eth0, host.tap0]: + dev.down() + host.br0.slave_add(dev)
guest1.eth0.down() - guest2.eth0.down()
- guest1_vlan_args0 = dict(realdev=guest1.eth0, vlan_id=10) - guest2_vlan_args0 = dict(realdev=guest2.eth0, vlan_id=10) - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - host2.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.br0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - guest2.eth0.mtu = self.params.mtu - for vlan_args in (guest1_vlan_args0, guest2_vlan_args0): - vlan_args["mtu"] = self.params.mtu - - guest1.vlan0 = VlanDevice(**guest1_vlan_args0) - guest2.vlan0 = VlanDevice(**guest2_vlan_args0) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.vlan0 - configuration.endpoint2 = guest2.vlan0 + guest1.vlan0 = VlanDevice(realdev=guest1.eth0, vlan_id=10) + guest2.vlan0 = VlanDevice(realdev=guest2.eth0, vlan_id=10) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.vlan0, guest2.vlan0, + host1.br0, host2.br0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" - host1.br0.ip_add(ipaddress(net_addr_1 + ".1/24")) host2.br0.ip_add(ipaddress(net_addr_1 + ".2/24")) - guest1.vlan0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.vlan0.ip_add(ipaddress(net_addr6_1 + "::3/64")) - guest2.vlan0.ip_add(ipaddress(net_addr_1 + ".4/24")) - guest2.vlan0.ip_add(ipaddress(net_addr6_1 + "::4/64")) - - host1.eth0.up() - host1.tap0.up() - host1.br0.up() - host2.eth0.up() - host2.tap0.up() - host2.br0.up() - guest1.eth0.up() - guest1.vlan0.up() - guest2.eth0.up() - guest2.vlan0.up() - - #TODO better service handling through HostAPI - if "perf_tool_cpu" in self.params: - raise LnstError("'perf_tool_cpu' (%d) should not be set for this test" % self.params.perf_tool_cpu)
- if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) + for i, guest in enumerate([guest1, guest2]): + guest.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+3) + + "/24")) + guest.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+3) + + "/64"))
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0]: + dev.up() + for guest in [guest1, guest2]: + guest.eth0.up() + guest.vlan0.up() + + if "perf_tool_cpu" in self.params: + logging.info("'perf_tool_cpu' param (%d) to be set to None" % + self.params.perf_tool_cpu) + self.params.perf_tool_cpu = None + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + def generate_test_wide_description(self, config): + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2) + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in [guest1.vlan0, guest2.vlan0] + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in [guest1.vlan0, guest2.vlan0] + ]), + "\n".join([ + "Configured {}.{}.slaves = {}".format( + dev.host.hostid, dev.name, + ['.'.join([dev.host.hostid, slave.name]) + for slave in dev.slaves] + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.vlan0, self.matched.guest2.vlan0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.vlan0, self.matched.guest2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.vlan0, self.matched.guest2.vlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2) + result = [] + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0]: + result.append(dev) + for guest in [guest1, guest2]: + for dev in [guest.eth0, guest.vlan0]: + result.append(dev) + return result + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py index 02e15b3..bbc5e69 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInGuestRecipe.py @@ -1,16 +1,18 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(virtual_bridge_vlan_in_guest.xml + virtual_bridge_vlan_in_guest.py) -""" -from lnst.Common.Parameters import Param +import logging +from lnst.Common.Parameters import Param, IntParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice +from lnst.Devices.VlanDevice import VlanDevice as Vlan from lnst.Devices import BridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualBridgeVlanInGuestRecipe(BaseEnrtRecipe): +class VirtualBridgeVlanInGuestRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest") @@ -28,76 +30,121 @@ class VirtualBridgeVlanInGuestRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="off", tx="off", rx="on"), dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
+ dev_intr_cpu = IntParam(default=0) + def test_wide_configuration(self): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1)
- host1.eth0.down() - host1.tap0.down() host1.br0 = BridgeDevice() - host1.br0.slave_add(host1.eth0) - host1.br0.slave_add(host1.tap0) + for dev in [host1.eth0, host1.tap0]: + dev.down() + host1.br0.slave_add(dev)
host2.eth0.down() - guest1.eth0.down()
- host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) - guest1_vlan_args0 = dict(realdev=guest1.eth0, vlan_id=10) - - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - for vlan_args in (host2_vlan_args0, guest1_vlan_args0): - vlan_args["mtu"] = self.params.mtu - - host2.vlan0 = VlanDevice(**host2_vlan_args0) - guest1.vlan0 = VlanDevice(**guest1_vlan_args0) + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10) + guest1.vlan0 = VlanDevice(realdev=guest1.eth0, vlan_id=10)
- #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.vlan0 - configuration.endpoint2 = host2.vlan0 + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.vlan0, host1.br0, + host2.vlan0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1"
host1.br0.ip_add(ipaddress(net_addr_1 + ".1/24")) - host2.vlan0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.vlan0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - guest1.vlan0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.vlan0.ip_add(ipaddress(net_addr6_1 + "::3/64")) - - host1.eth0.up() - host1.tap0.up() - host1.br0.up() - host2.eth0.up() - host2.vlan0.up() - guest1.eth0.up() - guest1.vlan0.up() - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - raise LnstError("'dev_intr_cpu' (%d) should not be set for this test" % self.params.dev_intr_cpu) + for i, machine in enumerate([host2, guest1]): + machine.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+2) + + "/24")) + machine.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+2) + + "/64"))
- if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, 0) + for dev in [host1.eth0, host1.tap0, host1.br0, host2.eth0, + host2.vlan0, guest1.eth0, guest1.vlan0]: + dev.up()
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "Configured {}.{}.slaves = {}".format( + host1.hostid, host1.br0.name, + ['.'.join([host1.hostid, slave.name]) + for slave in host1.br0.slaves] + ) + ] + return desc + def test_wide_deconfiguration(self, config): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.vlan0, self.matched.host2.vlan0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.vlan0, self.matched.host2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.vlan0, self.matched.host2.vlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + return [host1.eth0, host1.tap0, host1.br0, guest1.eth0, + host2.eth0, host2.vlan0, guest1.vlan0]
- #TODO better service handling through HostAPI + @property + def dev_interrupt_hw_config_dev_list(self): if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + result = [self.matched.host1.eth0, self.matched.host2.eth0] + if self.params.dev_intr_cpu: + logging.info("'dev_intr_cpu' param (%d) to be set to 0" % + self.params.dev_intr_cpu) + self.params.dev_intr_cpu = 0 + return result + logging.info("'dev_intr_cpu' param (%d) to be ignored" % + self.params.dev_intr_cpu) + return [] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py index 1c9ea4d..07b8c51 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostMirroredRecipe.py @@ -1,16 +1,17 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(virtual_bridge_vlan_in_host_mirrored.xml + virtual_bridge_vlan_in_host_mirrored.py) -""" +import logging from lnst.Common.Parameters import Param from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import BridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualBridgeVlanInHostMirroredRecipe(BaseEnrtRecipe): +class VirtualBridgeVlanInHostMirroredRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest1") @@ -33,83 +34,124 @@ class VirtualBridgeVlanInHostMirroredRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
def test_wide_configuration(self): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2)
- host1.eth0.down() - host1.tap0.down() - host1.br0 = BridgeDevice() - host1.br0.slave_add(host1.tap0) - - host2.eth0.down() - host2.tap0.down() - host2.br0 = BridgeDevice() - host2.br0.slave_add(host2.tap0) + for host in [host1, host2]: + host.eth0.down() + host.tap0.down() + host.br0 = BridgeDevice() + host.br0.slave_add(host.tap0)
guest1.eth0.down() - guest2.eth0.down()
- host1_vlan_args0 = dict(realdev=host1.eth0, vlan_id=10, master=host1.br0) - host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10, master=host2.br0) - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - host2.tap0.mtu = self.params.mtu - host2.br0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - guest2.eth0.mtu = self.params.mtu - for vlan_args in (host1_vlan_args0, host2_vlan_args0): - vlan_args["mtu"] = self.params.mtu - - host1.vlan0 = VlanDevice(**host1_vlan_args0) - host2.vlan0 = VlanDevice(**host2_vlan_args0) - - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.eth0 - configuration.endpoint2 = guest2.eth0 + host1.vlan0 = VlanDevice(realdev=host1.eth0, vlan_id=10, + master=host1.br0) + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10, + master=host2.br0) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.vlan0, host2.vlan0, + host1.br0, host2.br0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" - host1.br0.ip_add(ipaddress(net_addr_1 + ".1/24")) host2.br0.ip_add(ipaddress(net_addr_1 + ".2/24")) - guest1.eth0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.eth0.ip_add(ipaddress(net_addr6_1 + "::3/64")) - guest2.eth0.ip_add(ipaddress(net_addr_1 + ".4/24")) - guest2.eth0.ip_add(ipaddress(net_addr6_1 + "::4/64")) - - host1.eth0.up() - host1.tap0.up() - host1.vlan0.up() - host1.br0.up() - host2.eth0.up() - host2.tap0.up() - host2.vlan0.up() - host2.br0.up() + for i, guest in enumerate([guest1, guest2]): + guest.eth0.ip_add(ipaddress(net_addr_1 + "." + str(i+3) + + "/24")) + guest.eth0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+3) + + "/64")) + + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.vlan0, host.br0]: + dev.up() guest1.eth0.up() guest2.eth0.up()
- #TODO better service handling through HostAPI if "perf_tool_cpu" in self.params: - raise LnstError("'perf_tool_cpu' (%d) should not be set for this test" % self.params.perf_tool_cpu) + logging.info("'perf_tool_cpu' param (%d) to be set to None" % + self.params.perf_tool_cpu) + self.params.perf_tool_cpu = None
- if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in [host1.vlan0, host2.vlan0] + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in [host1.vlan0, host2.vlan0] + ]), + "\n".join([ + "Configured {}.{}.slaves = {}".format( + dev.host.hostid, dev.name, + ['.'.join([dev.host.hostid, slave.name]) + for slave in dev.slaves] + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest2.eth0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest2.eth0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.guest2.eth0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2) + result = [] + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0, host.vlan0]: + result.append(dev) + result.extend([guest1.eth0, guest2.eth0]) + return result + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py index b5d4677..a20408d 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlanInHostRecipe.py @@ -1,16 +1,16 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(virtual_bridge_vlan_in_host.xml + virtual_bridge_vlan_in_host.py) -""" -from lnst.Common.Parameters import Param +from lnst.Common.Parameters import Param, IntParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import BridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualBridgeVlanInHostRecipe(BaseEnrtRecipe): +class VirtualBridgeVlanInHostRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest") @@ -28,8 +28,11 @@ class VirtualBridgeVlanInHostRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="off", tx="off", rx="on"), dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
+ dev_intr_cpu = IntParam(default=0) + def test_wide_configuration(self): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1)
host1.eth0.down() host1.tap0.down() @@ -37,63 +40,113 @@ class VirtualBridgeVlanInHostRecipe(BaseEnrtRecipe): host1.br0.slave_add(host1.tap0)
host2.eth0.down() - guest1.eth0.down()
- host1_vlan_args0 = dict(realdev=host1.eth0, vlan_id=10, master=host1.br0) + host1_vlan_args0 = dict() host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - for vlan_args in (host1_vlan_args0, host2_vlan_args0): - vlan_args["mtu"] = self.params.mtu - - host1.vlan0 = VlanDevice(**host1_vlan_args0) + + host1.vlan0 = VlanDevice(realdev=host1.eth0, vlan_id=10, + master=host1.br0) host2.vlan0 = VlanDevice(**host2_vlan_args0)
- configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.eth0 - configuration.endpoint2 = host2.vlan0 + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.eth0, host2.vlan0, + host1.br0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1"
host1.br0.ip_add(ipaddress(net_addr_1 + ".1/24")) - host2.vlan0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.vlan0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - guest1.eth0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.eth0.ip_add(ipaddress(net_addr6_1 + "::3/64")) - - host1.eth0.up() - host1.tap0.up() - host1.vlan0.up() - host1.br0.up() - host2.eth0.up() - host2.vlan0.up() - guest1.eth0.up() - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - raise LnstError("'dev_intr_cpu' (%d) should not be set for this test" % self.params.dev_intr_cpu) + for i, dev in enumerate([host2.vlan0, guest1.eth0]): + dev.ip_add(ipaddress(net_addr_1 + "." + str(i+2) + "/24")) + dev.ip_add(ipaddress(net_addr6_1 + "::" + str(i+2) + "/64"))
- if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, 0) + for dev in [host1.eth0, host1.tap0, host1.vlan0, host1.br0, + host2.eth0, host2.vlan0, guest1.eth0]: + dev.up()
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in [host1.vlan0, host2.vlan0] + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in [host1.vlan0, host2.vlan0] + ]), + "Configured {}.{}.slaves = {}".format( + host1.hostid, host1.br0.name, + ['.'.join([host1.hostid, slave.name]) + for slave in host1.br0.slaves] + ) + ] + return desc + def test_wide_deconfiguration(self, config): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.host2.vlan0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.host2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.host2.vlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + result = [] + for dev in [host1.eth0, host1.tap0, host1.br0, host2.eth0, + guest1.eth0, host1.vlan0, host2.vlan0]: + result.append(dev) + return result
- #TODO better service handling through HostAPI + @property + def dev_interrupt_hw_config_dev_list(self): if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + result = [self.matched.host1.eth0, self.matched.host2.eth0] + if self.params.dev_intr_cpu: + logging.info("'dev_intr_cpu' param (%d) to be set to 0" % + self.params.dev_intr_cpu) + self.params.dev_intr_cpu = 0 + return result + + logging.info("'dev_intr_cpu' param (%d) to be ignored" % + self.params.dev_intr_cpu) + return [] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py b/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py index 39d1866..31ac574 100644 --- a/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py +++ b/lnst/Recipes/ENRT/VirtualBridgeVlansOverBondRecipe.py @@ -1,17 +1,18 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(virtual_bridge_2_vlans_over_active_backup_bond.xml + virtual_bridge_2_vlans_over_bond.py) -""" +import logging from lnst.Common.Parameters import Param, IntParam, StrParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import BondDevice from lnst.Devices import BridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualBridgeVlansOverBondRecipe(BaseEnrtRecipe): +class VirtualBridgeVlansOverBondRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) @@ -46,14 +47,15 @@ class VirtualBridgeVlansOverBondRecipe(BaseEnrtRecipe): miimon_value = IntParam(mandatory=True)
def test_wide_configuration(self): - host1, host2, guest1, guest2, guest3, guest4 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2, self.matched.guest3, self.matched.guest4 + host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2, + self.matched.guest3, self.matched.guest4)
for host in [host1, host2]: - host.eth0.down() - host.eth1.down() - host.tap0.down() - host.tap1.down() - host.bond0 = BondDevice(mode=self.params.bonding_mode, miimon=self.params.miimon_value) + for dev in [host.eth0, host.eth1, host.tap0, host.tap1]: + dev.down() + host.bond0 = BondDevice(mode=self.params.bonding_mode, + miimon=self.params.miimon_value) host.bond0.slave_add(host.eth0) host.bond0.slave_add(host.eth1) host.br0 = BridgeDevice() @@ -64,82 +66,208 @@ class VirtualBridgeVlansOverBondRecipe(BaseEnrtRecipe): for guest in (guest1, guest2, guest3, guest4): guest.eth0.down()
- host1_vlan_args0 = dict(realdev=host1.bond0, vlan_id=10, master=host1.br0) - host1_vlan_args1 = dict(realdev=host1.bond0, vlan_id=20, master=host1.br1) - host2_vlan_args0 = dict(realdev=host2.bond0, vlan_id=10, master=host2.br0) - host2_vlan_args1 = dict(realdev=host2.bond0, vlan_id=20, master=host2.br1) - if "mtu" in self.params: - for host in [host1, host2]: - host1.bond0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.tap1.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host1.br1.mtu = self.params.mtu - for guest in [guest1, guest2, guest3, guest4]: - guest.eth0.mtu = self.params.mtu - for vlan_args in (host1_vlan_args0, host1_vlan_args1, - host2_vlan_args0, host2_vlan_args1): - vlan_args["mtu"] = self.params.mtu - - host1.vlan0 = VlanDevice(**host1_vlan_args0) - host1.vlan1 = VlanDevice(**host1_vlan_args1) - host2.vlan0 = VlanDevice(**host2_vlan_args0) - host2.vlan1 = VlanDevice(**host2_vlan_args1) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.eth0 - configuration.endpoint2 = guest3.eth0 - - net_addr_1 = "192.168.10" - net_addr_2 = "192.168.20" - net_addr6_1 = "fc00:0:0:1" - net_addr6_2 = "fc00:0:0:2" - - for host, (guest_a, guest_b), n in [(host1, (guest1, guest2), 1), (host2, (guest3, guest4), 3)]: - host.bond0.ip_add(ipaddress("1.2.3.4")) - host.br0.ip_add(ipaddress(net_addr_1 + "." + str(n) + "/24")) - host.br1.ip_add(ipaddress(net_addr_2 + "." + str(n) + "/24")) - guest_a.eth0.ip_add(ipaddress(net_addr_1 + "." + str(n+1) + "/24")) - guest_a.eth0.ip_add(ipaddress(net_addr6_1 + "::" + str(n+1) + "/64")) - guest_b.eth0.ip_add(ipaddress(net_addr_2 + "." + str(n+1) + "/24")) - guest_b.eth0.ip_add(ipaddress(net_addr6_2 + "::" + str(n+1) + "/64")) - - for host, guest_a, guest_b in [(host1, guest1, guest2), (host2, guest3, guest4)]: - host.eth0.up() - host.eth1.up() - host.tap0.up() - host.tap1.up() - host.bond0.up() - host.vlan0.up() - host.vlan1.up() - host.br0.up() - host.br1.up() - guest_a.eth0.up() - guest_b.eth0.up() - - #TODO better service handling through HostAPI - if "perf_tool_cpu" in self.params: - raise LnstError("'perf_tool_cpu' (%d) should not be set for this test" % self.params.perf_tool_cpu) + host1.vlan0 = VlanDevice(realdev=host1.bond0, vlan_id=10, + master=host1.br0) + host1.vlan1 = VlanDevice(realdev=host1.bond0, vlan_id=20, + master=host1.br1) + host2.vlan0 = VlanDevice(realdev=host2.bond0, vlan_id=10, + master=host2.br0) + host2.vlan1 = VlanDevice(realdev=host2.bond0, vlan_id=20, + master=host2.br1) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.br0, host2.br0, + guest1.eth0, guest2.eth0, guest3.eth0, guest4.eth0]
- if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host.eth0, host.eth1]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) + net_addr = "192.168" + net_addr6 = "fc00:0:0" + for host, (guest_a, guest_b), n in [(host1, (guest1, guest2), 1), + (host2, (guest3, guest4), 3)]: + host.br0.ip_add(ipaddress(net_addr + ".10." + str(n) + "/24")) + host.br1.ip_add(ipaddress(net_addr + ".20." + str(n) + "/24")) + guest_a.eth0.ip_add(ipaddress(net_addr + ".10." + str(n+1) + + "/24")) + guest_a.eth0.ip_add(ipaddress(net_addr6 + ":1::" + str(n+1) + + "/64")) + guest_b.eth0.ip_add(ipaddress(net_addr + ".20." + str(n+1) + + "/24")) + guest_b.eth0.ip_add(ipaddress(net_addr6 + ":2::" + str(n+1) + + "/64"))
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + for host in [host1, host2]: + for dev in [host.eth0, host.eth1, host.tap0, host.tap1, + host.bond0, host.vlan0, host.vlan1, host.br0, host.br1]: + dev.up() + for guest in [guest1, guest2, guest3, guest4]: + guest.eth0.up() + + if "perf_tool_cpu" in self.params: + logging.info("'perf_tool_cpu' param (%d) to be set to None" % + self.params.perf_tool_cpu) + self.params.perf_tool_cpu = None + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.slaves = {}".format( + dev.host.hostid, dev.name, + ['.'.join([dev.host.hostid, slave.name]) + for slave in dev.slaves] + ) + for dev in [host1.bond0, host2.bond0, host1.br0, + host1.br1, host2.br0, host2.br1] + ]), + "\n".join([ + "Configured {}.{}.mode = {}".format( + dev.host.hostid, dev.name, dev.mode + ) + for dev in [host1.bond0, host2.bond0] + ]), + "\n".join([ + "Configured {}.{}.miimon = {}".format( + dev.host.hostid, dev.name, dev.miimon + ) + for dev in [host1.bond0, host2.bond0] + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in [host1.vlan0, host1.vlan1, + host2.vlan0, host2.vlan1] + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in [host1.vlan0, host1.vlan1, host2.vlan0, + host2.vlan1] + ]) + ] + return desc + def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2, guest3, guest4 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2, self.matched.guest3, self.matched.guest4 + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + guest1, guest2, guest3, guest4 = (self.matched.guest1, + self.matched.guest2, self.matched.guest3, self.matched.guest4) + return [(guest1.eth0, guest3.eth0), (guest4.eth0, guest2.eth0), + (guest1.eth0, guest4.eth0), (guest3.eth0, guest2.eth0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest3.eth0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + def do_ping_tests(self, recipe_config): + for ping_config in self.generate_ping_configurations( + recipe_config): + exp_fail = [] + for pconf in ping_config: + cond = self.vlan_id_match(pconf.client_bind, + pconf.destination_address) + exp_fail.append(cond) + result = self.ping_test(ping_config, exp_fail) + self.ping_evaluate_and_report(ping_config, result) + + def ping_test(self, ping_config, exp_fail): + results = {} + + running_ping_array = [] + for pingconf, fail in zip(ping_config, exp_fail): + ping, client = self.ping_init(pingconf) + running_ping = client.prepare_job(ping, fail=fail) + running_ping.start(bg = True) + running_ping_array.append((pingconf, running_ping)) + + for _, pingjob in running_ping_array: + try: + pingjob.wait() + finally: + pingjob.kill() + + for pingconf, pingjob in running_ping_array: + result = pingjob.result + passed = pingjob.passed + results[pingconf] = (result, passed) + + return results + + def single_ping_evaluate_and_report(self, ping_config, result): + fmt = "From: <{0.client.hostid} ({0.client_bind})> To: " \ + "<{0.destination.hostid} ({0.destination_address})>" + description = fmt.format(ping_config) + if result[0].get("rate", 0) > 50: + message = "Ping successful --- " + description + self.add_result(result[1], message, result[0]) + else: + message = "Ping unsuccessful --- " + description + self.add_result(result[1], message, result[0]) + + def vlan_id_match(self, src_addr, dst_addr): + guest1, guest2, guest3, guest4 = (self.matched.guest1, + self.matched.guest2, self.matched.guest3, self.matched.guest4) + + matching_pairs = [] + for pair in [(guest1, guest3), (guest2, guest4)]: + matching_pairs.extend([pair, pair[::-1]]) + + devs = [] + for dev in (guest1.devices + guest2.devices + guest3.devices + + guest4.devices): + if src_addr in dev.ips or dst_addr in dev.ips: + devs.append(dev) + try: + return (devs[0].host, devs[1].host) not in matching_pairs + except IndexError: + return False + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.guest2.eth0, + self.matched.guest3.eth0, self.matched.guest4.eth0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2, + self.matched.guest3, self.matched.guest4) + result = [] + for host in [host1, host2]: + for dev in [host.bond0, host.tap0, host.tap1, host.br0, + host.br1, host.vlan0, host.vlan1]: + result.append(dev) + for guest in [guest1, guest2, guest3, guest4]: + result.append(guest.eth0) + return result + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host1.eth1, + self.matched.host2.eth0, self.matched.host2.eth1]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py index a744337..655e8f9 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestMirroredRecipe.py @@ -1,23 +1,23 @@ -""" -Implements scenario similar to regression_tests/phase2/ -(virtual_ovs_bridge_vlan_in_guest_mirrored.xml + virtual_ovs_bridge_vlan_in_guest_mirrored.py -) -""" +import logging from lnst.Common.Parameters import Param from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import OvsBridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualOvsBridgeVlanInGuestMirroredRecipe(BaseEnrtRecipe): +class VirtualOvsBridgeVlanInGuestMirroredRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() - host1.eth1 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) + host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest1")
host2 = HostReq() - host2.eth1 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) + host2.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host2.tap0 = DeviceReq(label="to_guest2")
guest1 = HostReq() @@ -34,89 +34,120 @@ class VirtualOvsBridgeVlanInGuestMirroredRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
def test_wide_configuration(self): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 - - host1.eth1.down() - host1.tap0.down() - host1.br0 = OvsBridgeDevice() - for host, dev in [(host1, host1.eth1), (host1, host1.tap0)]: - host.br0.port_add(dev) - - host2.eth1.down() - host2.tap0.down() - host2.br0 = OvsBridgeDevice() - for host, dev in [(host2, host2.eth1), (host2, host2.tap0)]: - if dev.master != None: - if "ovs" not in dev.master.name: - host.br0.port_add(dev) - else: + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2) + + for host in [host1, host2]: + host.eth0.down() + host.tap0.down() + host.br0 = OvsBridgeDevice() + for dev in [host.eth0, host.tap0]: host.br0.port_add(dev)
guest1.eth0.down() - guest2.eth0.down()
- guest1_vlan_args0 = dict(realdev=guest1.eth0, vlan_id=10) - guest2_vlan_args0 = dict(realdev=guest2.eth0, vlan_id=10) - if "mtu" in self.params: - host1.eth1.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth1.mtu = self.params.mtu - host2.tap0.mtu = self.params.mtu - host2.br0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - guest2.eth0.mtu = self.params.mtu - for vlan_args in (guest1_vlan_args0, guest2_vlan_args0): - vlan_args["mtu"] = self.params.mtu - - guest1.vlan0 = VlanDevice(**guest1_vlan_args0) - guest2.vlan0 = VlanDevice(**guest2_vlan_args0) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.vlan0 - configuration.endpoint2 = guest2.vlan0 + guest1.vlan0 = VlanDevice(realdev=guest1.eth0, vlan_id=10) + guest2.vlan0 = VlanDevice(realdev=guest2.eth0, vlan_id=10) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.vlan0, guest2.vlan0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" + for i, guest in enumerate([guest1, guest2]): + guest.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+3) + + "/24")) + guest.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+3) + + "/64")) + + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0]: + dev.up() + for guest in [guest1, guest2]: + guest.eth0.up() + guest.vlan0.up()
- guest1.vlan0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.vlan0.ip_add(ipaddress(net_addr6_1 + "::3/64")) - guest2.vlan0.ip_add(ipaddress(net_addr_1 + ".4/24")) - guest2.vlan0.ip_add(ipaddress(net_addr6_1 + "::4/64")) - - host1.eth1.up() - host1.tap0.up() - host1.br0.up() - host2.eth1.up() - host2.tap0.up() - host2.br0.up() - guest1.eth0.up() - guest1.vlan0.up() - guest2.eth0.up() - guest2.vlan0.up() - - #TODO better service handling through HostAPI if "perf_tool_cpu" in self.params: - raise LnstError("'perf_tool_cpu' (%d) should not be set for this test" % self.params.perf_tool_cpu) - - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth1, self.params.dev_intr_cpu) + logging.info("'perf_tool_cpu' param (%d) to be set to None" % + self.params.perf_tool_cpu) + self.params.perf_tool_cpu = None
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth1.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.ports = {}".format( + dev.host.hostid, dev.name, dev.ports + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.vlan0, self.matched.guest2.vlan0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.vlan0, self.matched.guest2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.vlan0, self.matched.guest2.vlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2) + result = [] + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0]: + result.append(dev) + for guest in [guest1, guest2]: + result.extend([guest.eth0, guest.vlan0]) + return result + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py index d719e84..c274d46 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInGuestRecipe.py @@ -1,16 +1,16 @@ -""" -Implements scenario similar to regression_tests/phase2/ -(virtual_ovs_bridge_vlan_in_guest.xml + virtual_ovs_bridge_vlan_in_guest.py) -""" -from lnst.Common.Parameters import Param +from lnst.Common.Parameters import Param, IntParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import OvsBridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualOvsBridgeVlanInGuestRecipe(BaseEnrtRecipe): +class VirtualOvsBridgeVlanInGuestRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest") @@ -28,74 +28,120 @@ class VirtualOvsBridgeVlanInGuestRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="off", tx="off", rx="on"), dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
+ dev_intr_cpu = IntParam(default=0) + def test_wide_configuration(self): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1)
- host1.eth0.down() - host1.tap0.down() host1.br0 = OvsBridgeDevice() - host1.br0.port_add(host1.eth0) - host1.br0.port_add(host1.tap0) + for dev in [host1.eth0, host1.tap0]: + dev.down() + host1.br0.port_add(dev)
host2.eth0.down() - guest1.eth0.down()
- host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) + host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) guest1_vlan_args0 = dict(realdev=guest1.eth0, vlan_id=10) - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - for vlan_args in (host2_vlan_args0, guest1_vlan_args0): - vlan_args["mtu"] = self.params.mtu - - host2.vlan0 = VlanDevice(**host2_vlan_args0) - guest1.vlan0 = VlanDevice(**guest1_vlan_args0) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.vlan0 - configuration.endpoint2 = host2.vlan0 + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10) + guest1.vlan0 = VlanDevice(realdev=guest1.eth0, vlan_id=10) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host2.vlan0, guest1.vlan0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" + for i, machine in enumerate([host2, guest1]): + machine.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+2) + + "/24")) + machine.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+2) + + "/64"))
- host2.vlan0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.vlan0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - guest1.vlan0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.vlan0.ip_add(ipaddress(net_addr6_1 + "::3/64")) + for dev in [host1.eth0, host1.tap0, host1.br0, host2.eth0, + host2.vlan0, guest1.eth0, guest1.vlan0]: + dev.up()
- host1.eth0.up() - host1.tap0.up() - host1.br0.up() - host2.eth0.up() - host2.vlan0.up() - guest1.eth0.up() - guest1.vlan0.up() + self.wait_tentative_ips(configuration.test_wide_devices)
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - raise LnstError("'dev_intr_cpu' (%d) should not be set for this test" % self.params.dev_intr_cpu) + return configuration
- if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, 0) + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in config.test_wide_devices + ]), + "Configured {}.{}.ports = {}".format( + host1.hostid, host1.br0.name, host1.br0.ports + ) + ] + return desc
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + def test_wide_deconfiguration(self, config): + del config.test_wide_devices
- return configuration + super().test_wide_deconfiguration(config)
- def test_wide_deconfiguration(self, config): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + def generate_ping_endpoints(self, config): + return [(self.matched.host2.vlan0, self.matched.guest1.vlan0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host2.vlan0, self.matched.guest1.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host2.vlan0, self.matched.guest1.vlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + result = [] + for dev in [host1.eth0, host1.tap0, host1.br0, host2.eth0, + guest1.eth0, host2.vlan0, guest1.vlan0]: + result.append(dev) + return result
- #TODO better service handling through HostAPI + @property + def dev_interrupt_hw_config_dev_list(self): if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + result = [self.matched.host1.eth0, self.matched.host2.eth0] + if self.params.dev_intr_cpu: + logging.info("'dev_intr_cpu' param (%d) to be set to 0" % + self.params.dev_intr_cpu) + self.params.dev_intr_cpu = 0 + return result + + logging.info("'dev_intr_cpu' param (%d) to be ignored" % + self.params.dev_intr_cpu) + return [] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py index 734f5c2..2e62cbb 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostMirroredRecipe.py @@ -1,16 +1,16 @@ -""" -Implements scenario similar to regression_tests/phase2/ -(virtual_ovs_bridge_vlan_in_host_mirrored.xml + virtual_ovs_bridge_vlan_in_host_mirrored.py -) -""" +import logging from lnst.Common.Parameters import Param from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import OvsBridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualOvsBridgeVlanInHostMirroredRecipe(BaseEnrtRecipe): +class VirtualOvsBridgeVlanInHostMirroredRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest1") @@ -33,76 +33,103 @@ class VirtualOvsBridgeVlanInHostMirroredRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
def test_wide_configuration(self): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2)
- host1.eth0.down() - host1.tap0.down() - host1.br0 = OvsBridgeDevice() - host1.br0.port_add(host1.eth0) - host1.br0.port_add(host1.tap0, tag="10") - - host2.eth0.down() - host2.tap0.down() - host2.br0 = OvsBridgeDevice() - host2.br0.port_add(host2.eth0) - host2.br0.port_add(host2.tap0, tag="10") + for host in [host1, host2]: + host.br0 = OvsBridgeDevice() + host.eth0.down() + host.tap0.down() + host.br0.port_add(host.eth0) + host.br0.port_add(host.tap0, tag="10")
guest1.eth0.down() - guest2.eth0.down()
- #Due to limitations in the current EnrtConfiguration - #class, a single test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.eth0 - configuration.endpoint2 = guest2.eth0 - - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - host2.tap0.mtu = self.params.mtu - host2.br0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - guest2.eth0.mtu = self.params.mtu + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.eth0, guest2.eth0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" - - guest1.eth0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.eth0.ip_add(ipaddress(net_addr6_1 + "::3/64")) - guest2.eth0.ip_add(ipaddress(net_addr_1 + ".4/24")) - guest2.eth0.ip_add(ipaddress(net_addr6_1 + "::4/64")) - - host1.eth0.up() - host1.tap0.up() - host1.br0.up() - host2.eth0.up() - host2.tap0.up() - host2.br0.up() + for i, guest in enumerate([guest1, guest2]): + guest.eth0.ip_add(ipaddress(net_addr_1 + "." + str(i+3) + + "/24")) + guest.eth0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+3) + + "/64")) + + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0]: + dev.up() guest1.eth0.up() guest2.eth0.up()
- #TODO better service handling through HostAPI if "perf_tool_cpu" in self.params: - raise LnstError("'perf_tool_cpu' (%d) should not be set for this test" % self.params.perf_tool_cpu) + logging.info("'perf_tool_cpu' param (%d) to be set to None" % + self.params.perf_tool_cpu) + self.params.perf_tool_cpu = None
- if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2 + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.ports = {}".format( + dev.host.hostid, dev.name, dev.ports + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest2.eth0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest2.eth0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.guest2.eth0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1, guest2 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2) + result = [] + for host in [host1, host2]: + for dev in [host.eth0, host.tap0, host.br0]: + result.append(dev) + for guest in [guest1, guest2]: + result.append(guest.eth0) + return result + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py index 397bebe..1d288e7 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlanInHostRecipe.py @@ -1,16 +1,17 @@ -""" -Implements scenario similar to regression_tests/phase2/ -(virtual_ovs_bridge_vlan_in_host.xml + virtual_ovs_bridge_vlan_in_host.py) -""" -from lnst.Common.Parameters import Param +import logging +from lnst.Common.Parameters import Param, IntParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice from lnst.Devices import OvsBridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualOvsBridgeVlanInHostRecipe(BaseEnrtRecipe): +class VirtualOvsBridgeVlanInHostRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest") @@ -28,8 +29,11 @@ class VirtualOvsBridgeVlanInHostRecipe(BaseEnrtRecipe): dict(gro="on", gso="on", tso="off", tx="off", rx="on"), dict(gro="on", gso="on", tso="on", tx="on", rx="off")))
+ dev_intr_cpu = IntParam(default=0) + def test_wide_configuration(self): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1)
host1.eth0.down() host1.tap0.down() @@ -38,61 +42,97 @@ class VirtualOvsBridgeVlanInHostRecipe(BaseEnrtRecipe): host1.br0.port_add(host1.tap0, tag="10")
host2.eth0.down() - guest1.eth0.down()
- host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - - host2_vlan_args0["mtu"] = self.params.mtu - - host2.vlan0 = VlanDevice(**host2_vlan_args0) + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10)
- #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host2.vlan0 - configuration.endpoint2 = guest1.eth0 + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.eth0, host2.vlan0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" + for i, dev in enumerate([host2.vlan0, guest1.eth0]): + dev.ip_add(ipaddress(net_addr_1 + "." + str(i+2) + "/24")) + dev.ip_add(ipaddress(net_addr6_1 + "::" + str(i+2) + "/64"))
- host2.vlan0.ip_add(ipaddress(net_addr_1 + ".2/24")) - host2.vlan0.ip_add(ipaddress(net_addr6_1 + "::2/64")) - guest1.eth0.ip_add(ipaddress(net_addr_1 + ".3/24")) - guest1.eth0.ip_add(ipaddress(net_addr6_1 + "::3/64")) + for dev in [host1.eth0, host1.tap0, host1.br0, host2.eth0, + host2.vlan0, guest1.eth0]: + dev.up()
- host1.eth0.up() - host1.tap0.up() - host1.br0.up() - host2.eth0.up() - host2.vlan0.up() - guest1.eth0.up() + self.wait_tentative_ips(configuration.test_wide_devices)
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - raise LnstError("'dev_intr_cpu' (%d) should not be set for this test" % self.params.dev_intr_cpu) + return configuration
- if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, 0) + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "Configured {}.{}.vlan_id = {}".format( + host2.hostid, host2.vlan0.name, host2.vlan0.vlan_id + ), + "Configured {}.{}.realdev = {}".format( + host2.hostid, host2.vlan0.name, + '.'.join([host2.hostid, host2.vlan0.realdev.name]) + ), + "Configured {}.{}.ports = {}".format( + host1.hostid, host1.br0.name, host1.br0.ports + ) + ] + return desc + + def test_wide_deconfiguration(self, config): + del config.test_wide_devices
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + super().test_wide_deconfiguration(config)
- return configuration + def generate_ping_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.host2.vlan0)]
- def test_wide_deconfiguration(self, config): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.host2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.host2.vlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1 = (self.matched.host1, + self.matched.host2, self.matched.guest1) + result = [] + for dev in [host1.eth0, host1.tap0, host1.br0, host2.eth0, + host2.vlan0, guest1.eth0]: + result.append(dev) + return result
- #TODO better service handling through HostAPI + @property + def dev_interrupt_hw_config_dev_list(self): if "perf_tool_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + result = [self.matched.host1.eth0, self.matched.host2.eth0] + if self.params.dev_intr_cpu: + logging.info("'dev_intr_cpu' param (%d) to be set to 0" % + self.params.dev_intr_cpu) + self.params.dev_intr_cpu = 0 + return result + + logging.info("'dev_intr_cpu' param (%d) to be ignored" % + self.params.dev_intr_cpu) + return [] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0] diff --git a/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py b/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py index bba905b..20b8202 100644 --- a/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py +++ b/lnst/Recipes/ENRT/VirtualOvsBridgeVlansOverBondRecipe.py @@ -1,17 +1,16 @@ -""" -Implements scenario similar to regression_tests/phase2/ -(virtual_ovs_bridge_2_vlans_over_active_backup_bond.xml + -virtual_ovs_bridge_2_vlans_over_active_backup_bond.py -) -""" +import logging from lnst.Common.Parameters import Param, IntParam, StrParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import OvsBridgeDevice -from lnst.Common.LnstError import LnstError
-class VirtualOvsBridgeVlansOverBondRecipe(BaseEnrtRecipe): +class VirtualOvsBridgeVlansOverBondRecipe(OffloadSubConfigMixin, + CommonHWConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) @@ -46,60 +45,30 @@ class VirtualOvsBridgeVlansOverBondRecipe(BaseEnrtRecipe): miimon_value = IntParam(mandatory = True)
def test_wide_configuration(self): - host1, host2, guest1, guest2, guest3, guest4 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2, self.matched.guest3, self.matched.guest4 - - host1.eth0.down() - host1.eth1.down() - host1.tap0.down() - host1.tap1.down() - host1.br0 = OvsBridgeDevice() - for dev, tag in [(host1.tap0, "10"), (host1.tap1, "20")]: - host1.br0.port_add(dev, tag=tag) - - #miimon cannot be set due to colon in argument name --> other_config:bond-miimon-interval - #https://access.redhat.com/documentation/en-us/red_hat_openstack_platform/12/... - host1.br0.bond_add("bond_host1", (host1.eth0, host1.eth1), bond_mode=self.params.bonding_mode) - - host2.eth0.down() - host2.eth1.down() - host2.tap0.down() - host2.tap1.down() - host2.br0 = OvsBridgeDevice() - - for dev, tag in [(host2.tap0, "10"), (host2.tap1, "20")]: - host2.br0.port_add(dev, tag=tag) - - host2.br0.bond_add("bond_host2", (host2.eth0, host2.eth1), bond_mode=self.params.bonding_mode) + host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2, + self.matched.guest3, self.matched.guest4) + + for host, port_name in [(host1, "bond_port1"), + (host2, "bond_port2")]: + for dev in [host.eth0, host.eth1, host.tap0, host.tap1]: + dev.down() + host.br0 = OvsBridgeDevice() + for dev, tag in [(host.tap0, "10"), (host.tap1, "20")]: + host.br0.port_add(dev, tag=tag) + #miimon cannot be set due to colon in argument name --> + #other_config:bond-miimon-interval + host.br0.bond_add(port_name, (host.eth0, host.eth1), + bond_mode=self.params.bonding_mode)
guest1.eth0.down() - guest2.eth0.down() - guest3.eth0.down() - guest4.eth0.down()
- #Due to limitations in the current EnrtConfiguration - #class, a single test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = guest1.eth0 - configuration.endpoint2 = guest3.eth0 - - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host1.eth1.mtu = self.params.mtu - host1.tap0.mtu = self.params.mtu - host1.tap1.mtu = self.params.mtu - host1.br0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - host2.eth1.mtu = self.params.mtu - host2.tap0.mtu = self.params.mtu - host2.tap1.mtu = self.params.mtu - host2.br0.mtu = self.params.mtu - guest1.eth0.mtu = self.params.mtu - guest2.eth0.mtu = self.params.mtu - guest3.eth0.mtu = self.params.mtu - guest4.eth0.mtu = self.params.mtu + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [guest1.eth0, guest2.eth0, + guest3.eth0, guest4.eth0]
net_addr_1 = "192.168.10" net_addr6_1 = "fc00:0:0:1" @@ -107,49 +76,167 @@ class VirtualOvsBridgeVlansOverBondRecipe(BaseEnrtRecipe): net_addr6_2 = "fc00:0:0:2"
for i, guest in enumerate([guest1, guest3]): - guest.eth0.ip_add(ipaddress(net_addr_1 + "." + str (i+1) + "/24")) - guest.eth0.ip_add(ipaddress(net_addr6_1 + "::" + str (i+1) + "/64")) + guest.eth0.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + + "/24")) + guest.eth0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + + "/64"))
for i, guest in enumerate([guest2, guest4]): - guest.eth0.ip_add(ipaddress(net_addr_2 + "." + str (i+1) + "/24")) - guest.eth0.ip_add(ipaddress(net_addr6_2 + "::" + str (i+1) + "/64")) - - host1.eth0.up() - host1.eth1.up() - host1.tap0.up() - host1.tap1.up() - host1.br0.up() - host2.eth0.up() - host2.eth1.up() - host2.tap0.up() - host2.tap1.up() - host2.br0.up() - guest1.eth0.up() - guest2.eth0.up() - guest3.eth0.up() - guest4.eth0.up() - - #TODO better service handling through HostAPI - if "perf_tool_cpu" in self.params: - raise LnstError("'perf_tool_cpu' (%d) should not be set for this test" % self.params.perf_tool_cpu) + guest.eth0.ip_add(ipaddress(net_addr_2 + "." + str(i+1) + + "/24")) + guest.eth0.ip_add(ipaddress(net_addr6_2 + "::" + str(i+1) + + "/64")) + + for host in [host1, host2]: + for dev in [host.eth0, host.eth1, host.tap0, host.tap1, + host.br0]: + dev.up() + for guest in [guest1, guest2, guest3, guest4]: + guest.eth0.up()
- if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host.eth0, host.eth1]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) + if "perf_tool_cpu" in self.params: + logging.info("'perf_tool_cpu' param (%d) to be set to None" % + self.params.perf_tool_cpu) + self.params.perf_tool_cpu = None
- if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - for dev in [host.eth0, host.eth1]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): - host1, host2, guest1, guest2, guest3, guest4 = self.matched.host1, self.matched.host2, self.matched.guest1, self.matched.guest2, self.matched.guest3, self.matched.guest4 + def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.ports = {}".format( + dev.host.hostid, dev.name, dev.ports + ) + for dev in [host1.br0, host2.br0] + ]), + "\n".join([ + "Configured {}.{}.bonds = {}".format( + dev.host.hostid, dev.name, dev.bonds + ) + for dev in [host1.br0, host2.br0] + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + guest1, guest2, guest3, guest4 = (self.matched.guest1, + self.matched.guest2, self.matched.guest3, self.matched.guest4) + return [(guest1.eth0, guest3.eth0), (guest4.eth0, guest2.eth0), + (guest1.eth0, guest4.eth0), (guest3.eth0, guest2.eth0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.guest1.eth0, self.matched.guest3.eth0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.guest1.eth0, self.matched.guest2.eth0, + self.matched.guest3.eth0, self.matched.guest4.eth0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1, guest2, guest3, guest4 = (self.matched.host1, + self.matched.host2, self.matched.guest1, self.matched.guest2, + self.matched.guest3, self.matched.guest4) + result = [] + for host in [host1, host2]: + for dev in [host.eth0, host.eth1, host.tap0, host.tap1, + host.br0]: + result.append(dev) + for guest in [guest1, guest2, guest3, guest4]: + result.append(guest.eth0) + return result + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host1.eth1, + self.matched.host2.eth0, self.matched.host2.eth1] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list + + def do_ping_tests(self, recipe_config): + for ping_config in self.generate_ping_configurations( + recipe_config): + exp_fail = [] + for pconf in ping_config: + cond = self.vlan_id_match(pconf.client_bind, + pconf.destination_address) + exp_fail.append(cond) + result = self.ping_test(ping_config, exp_fail) + self.ping_evaluate_and_report(ping_config, result) + + def ping_test(self, ping_config, exp_fail): + results = {} + + running_ping_array = [] + for pingconf, fail in zip(ping_config, exp_fail): + ping, client = self.ping_init(pingconf) + running_ping = client.prepare_job(ping, fail=fail) + running_ping.start(bg = True) + running_ping_array.append((pingconf, running_ping)) + + for _, pingjob in running_ping_array: + try: + pingjob.wait() + finally: + pingjob.kill() + + for pingconf, pingjob in running_ping_array: + result = pingjob.result + passed = pingjob.passed + results[pingconf] = (result, passed) + + return results + + def single_ping_evaluate_and_report(self, ping_config, result): + fmt = "From: <{0.client.hostid} ({0.client_bind})> To: " \ + "<{0.destination.hostid} ({0.destination_address})>" + description = fmt.format(ping_config) + if result[0].get("rate", 0) > 50: + message = "Ping successful --- " + description + self.add_result(result[1], message, result[0]) + else: + message = "Ping unsuccessful --- " + description + self.add_result(result[1], message, result[0]) + + def vlan_id_match(self, src_addr, dst_addr): + guest1, guest2, guest3, guest4 = (self.matched.guest1, + self.matched.guest2, self.matched.guest3, self.matched.guest4) + + matching_pairs = [] + for pair in [(guest1, guest3), (guest2, guest4)]: + matching_pairs.extend([pair, pair[::-1]]) + + devs = [] + for dev in (guest1.devices + guest2.devices + guest3.devices + + guest4.devices): + if src_addr in dev.ips or dst_addr in dev.ips: + devs.append(dev) + try: + return (devs[0].host, devs[1].host) not in matching_pairs + except IndexError: + return False diff --git a/lnst/Recipes/ENRT/VlansOverBondRecipe.py b/lnst/Recipes/ENRT/VlansOverBondRecipe.py index b48cf4d..2ec44e6 100644 --- a/lnst/Recipes/ENRT/VlansOverBondRecipe.py +++ b/lnst/Recipes/ENRT/VlansOverBondRecipe.py @@ -1,16 +1,17 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(3_vlans_over_{round_robin, active_backup}_bond.xml + 3_vlans_over_bond.py), -but 2 Vlans are used -""" from lnst.Common.Parameters import Param, IntParam, StrParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice +from lnst.Devices.VlanDevice import VlanDevice as Vlan from lnst.Devices import BondDevice
-class VlansOverBondRecipe(BaseEnrtRecipe): +class VlansOverBondRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="net1", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="net1", driver=RecipeParam("driver")) @@ -30,78 +31,196 @@ class VlansOverBondRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- host1.bond = BondDevice(mode=self.params.bonding_mode, miimon=self.params.miimon_value) - host1.eth0.down() - host1.eth1.down() - host1.bond.slave_add(host1.eth0) - host1.bond.slave_add(host1.eth1) - - host1_vlan_args0 = dict(realdev=host1.bond, vlan_id=10) - host1_vlan_args1 = dict(realdev=host1.bond, vlan_id=20) - host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) - host2_vlan_args1 = dict(realdev=host2.eth0, vlan_id=20) - if "mtu" in self.params: - host1.bond.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - for vlan_args in (host1_vlan_args0, host1_vlan_args1, - host2_vlan_args0, host2_vlan_args1): - vlan_args["mtu"] = self.params.mtu - - host1.vlan0 = VlanDevice(**host1_vlan_args0) - host1.vlan1 = VlanDevice(**host1_vlan_args1) - host2.vlan0 = VlanDevice(**host2_vlan_args0) - host2.vlan1 = VlanDevice(**host2_vlan_args1) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.vlan0 - configuration.endpoint2 = host2.vlan0 - - net_addr_1 = "192.168.10" - net_addr_2 = "192.168.20" - net_addr6_1 = "fc00:0:0:1" - net_addr6_2 = "fc00:0:0:2" + host1.bond0 = BondDevice(mode=self.params.bonding_mode, + miimon=self.params.miimon_value) + for dev in [host1.eth0, host1.eth1]: + dev.down() + host1.bond0.slave_add(dev) + + host1.vlan0 = VlanDevice(realdev=host1.bond0, vlan_id=10) + host1.vlan1 = VlanDevice(realdev=host1.bond0, vlan_id=20) + host1.vlan2 = VlanDevice(realdev=host1.bond0, vlan_id=30) + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10) + host2.vlan1 = VlanDevice(realdev=host2.eth0, vlan_id=20) + host2.vlan2 = VlanDevice(realdev=host2.eth0, vlan_id=30) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [] + for host in [host1, host2]: + configuration.test_wide_devices.extend([host.vlan0, + host.vlan1, host.vlan2]) + configuration.test_wide_devices.append(host1.bond0) + + net_addr = "192.168" + net_addr6 = "fc00:0:0"
for i, host in enumerate([host1, host2]): - host.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + "/24")) - host.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + "/64")) - host.vlan1.ip_add(ipaddress(net_addr_2 + "." + str(i+1) + "/24")) - host.vlan1.ip_add(ipaddress(net_addr6_2 + "::" + str(i+1) + "/64")) - - host1.eth0.up() - host1.eth1.up() - host1.bond.up() - host1.vlan0.up() - host1.vlan1.up() - host2.eth0.up() - host2.vlan0.up() - host2.vlan1.up() - - if "adaptive_rx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host1.eth0, host1.eth1, host2.eth0]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host, dev in [(host1, host1.eth0), (host1, host1.eth1), (host2, host2.eth0)]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + host.vlan0.ip_add(ipaddress(net_addr + '.10' + '.' + str(i+1) + + "/24")) + host.vlan0.ip_add(ipaddress(net_addr6 + ":1::" + str(i+1) + + "/64")) + host.vlan1.ip_add(ipaddress(net_addr + '.20' + '.' + str(i+1) + + "/24")) + host.vlan1.ip_add(ipaddress(net_addr6 + ":2::" + str(i+1) + + "/64")) + host.vlan2.ip_add(ipaddress(net_addr + '.30' + '.' + str(i+1) + + "/24")) + host.vlan2.ip_add(ipaddress(net_addr6 + ":3::" + str(i+1) + + "/64")) + + for dev in [host1.eth0, host1.eth1, host1.bond0, host1.vlan0, + host1.vlan1, host1.vlan2, host2.eth0, host2.vlan0, + host2.vlan1, host2.vlan2]: + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "Configured {}.{}.slaves = {}".format( + host1.hostid, host1.bond0.name, + ['.'.join([host1.hostid, slave.name]) + for slave in host1.bond0.slaves] + ), + "Configured {}.{}.mode = {}".format( + host1.hostid, host1.bond0.name, + host1.bond0.mode + ), + "Configured {}.{}.miimon = {}".format( + host1.hostid, host1.bond0.name, + host1.bond0.miimon + ) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): host1, host2 = self.matched.host1, self.matched.host2 + result = [] + for src in [host1.vlan0, host1.vlan1, host1.vlan2]: + for dst in [host2.vlan0, host2.vlan1, host2.vlan2]: + result += [(src, dst)] + return result + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.vlan0, self.matched.host2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + result = [] + for host in [self.matched.host1, self.matched.host2]: + for dev in [host.vlan0, host.vlan1, host.vlan2]: + result.append(dev) + return result + + @property + def mtu_hw_config_dev_list(self): + host1, host2 = self.matched.host1, self.matched.host2 + return self.offload_nics + [host1.bond0, host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def coalescing_hw_config_dev_list(self): + host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host1.eth1, host2.eth0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list + + def do_ping_tests(self, recipe_config): + for ping_config in self.generate_ping_configurations( + recipe_config): + exp_fail = [] + for pconf in ping_config: + cond = self.vlan_id_same(pconf.client_bind, + pconf.destination_address) + exp_fail.append(cond) + result = self.ping_test(ping_config, exp_fail) + self.ping_evaluate_and_report(ping_config, result) + + def ping_test(self, ping_config, exp_fail): + results = {} + + running_ping_array = [] + for pingconf, fail in zip(ping_config, exp_fail): + ping, client = self.ping_init(pingconf) + running_ping = client.prepare_job(ping, fail=fail) + running_ping.start(bg = True) + running_ping_array.append((pingconf, running_ping)) + + for _, pingjob in running_ping_array: + try: + pingjob.wait() + finally: + pingjob.kill() + + for pingconf, pingjob in running_ping_array: + result = pingjob.result + passed = pingjob.passed + results[pingconf] = (result, passed) + + return results + + def single_ping_evaluate_and_report(self, ping_config, result): + fmt = "From: <{0.client.hostid} ({0.client_bind})> To: " \ + "<{0.destination.hostid} ({0.destination_address})>" + description = fmt.format(ping_config) + if result[0].get("rate", 0) > 50: + message = "Ping successful --- " + description + self.add_result(result[1], message, result[0]) + else: + message = "Ping unsuccessful --- " + description + self.add_result(result[1], message, result[0]) + + def vlan_id_same(self, src_addr, dst_addr): + host1, host2 = self.matched.host1, self.matched.host2 + devs = [] + for dev in (host1.devices + host2.devices): + if src_addr in dev.ips or dst_addr in dev.ips: + devs.append(dev) + try: + return devs[0].vlan_id != devs[1].vlan_id + except (IndexError, AttributeError): + return False diff --git a/lnst/Recipes/ENRT/VlansOverTeamRecipe.py b/lnst/Recipes/ENRT/VlansOverTeamRecipe.py index 0867677..e4b9cf3 100644 --- a/lnst/Recipes/ENRT/VlansOverTeamRecipe.py +++ b/lnst/Recipes/ENRT/VlansOverTeamRecipe.py @@ -1,15 +1,17 @@ -""" -Implements scenario similar to regression_tests/phase2/ -(3_vlans_over_{active_backup,round_robin}_team.xml + 3_vlans_over_team.py) -""" -from lnst.Common.Parameters import Param, IntParam, StrParam +from lnst.Common.Parameters import Param, StrParam from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice +from lnst.Devices.VlanDevice import VlanDevice as Vlan from lnst.Devices import TeamDevice
-class VlansOverTeamRecipe(BaseEnrtRecipe): +class VlansOverTeamRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="tnet", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="tnet", driver=RecipeParam("driver")) @@ -28,81 +30,194 @@ class VlansOverTeamRecipe(BaseEnrtRecipe): def test_wide_configuration(self): host1, host2 = self.matched.host1, self.matched.host2
- host1.eth0.down() - host1.eth1.down() - #The config argument needs to be used with a team device normally (e.g to specify - #the runner mode), but it is not used here due to a bug in the TeamDevice module + #The config argument needs to be used with a team device normally + #(e.g to specify the runner mode), but it is not used here due to + #a bug in the TeamDevice module host1.team0 = TeamDevice() - host1.team0.slave_add(host1.eth0) - host1.team0.slave_add(host1.eth1) - - host1_vlan_args0 = dict(realdev=host1.team0, vlan_id=10) - host1_vlan_args1 = dict(realdev=host1.team0, vlan_id=20) - host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) - host2_vlan_args1 = dict(realdev=host2.eth0, vlan_id=20) - if "mtu" in self.params: - host1.team0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - for vlan_args in (host1_vlan_args0, host1_vlan_args1, - host2_vlan_args0, host2_vlan_args1): - vlan_args["mtu"] = self.params.mtu - - host1.vlan0 = VlanDevice(**host1_vlan_args0) - host1.vlan1 = VlanDevice(**host1_vlan_args1) - host2.vlan0 = VlanDevice(**host2_vlan_args0) - host2.vlan1 = VlanDevice(**host2_vlan_args1) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.vlan0 - configuration.endpoint2 = host2.vlan0 - - net_addr_1 = "192.168.10" - net_addr_2 = "192.168.20" - net_addr6_1 = "fc00:0:0:1" - net_addr6_2 = "fc00:0:0:2" - - host1.team0.ip_add(ipaddress("1.2.3.4/24")) + for dev in [host1.eth0, host1.eth1]: + dev.down() + host1.team0.slave_add(dev) + + host1.vlan0 = VlanDevice(realdev=host1.team0, vlan_id=10) + host1.vlan1 = VlanDevice(realdev=host1.team0, vlan_id=20) + host1.vlan2 = VlanDevice(realdev=host1.team0, vlan_id=30) + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10) + host2.vlan1 = VlanDevice(realdev=host2.eth0, vlan_id=20) + host2.vlan2 = VlanDevice(realdev=host2.eth0, vlan_id=30) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [] + for host in [host1, host2]: + configuration.test_wide_devices.extend([host.vlan0, + host.vlan1, host.vlan2]) + configuration.test_wide_devices.append(host1.team0) + + net_addr = "192.168" + net_addr6 = "fc00:0:0" + for i, host in enumerate([host1, host2]): - host.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + "/24")) - host.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + "/64")) - host.vlan1.ip_add(ipaddress(net_addr_2 + "." + str(i+1) + "/24")) - host.vlan1.ip_add(ipaddress(net_addr6_2 + "::" + str(i+1) + "/64")) - - host1.eth0.up() - host1.eth1.up() - host1.team0.up() - host1.vlan0.up() - host1.vlan1.up() - host2.eth0.up() - host2.vlan0.up() - host2.vlan1.up() - - if "adaptive_rx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for dev in [host1.eth0, host1.eth1, host2.eth0]: - dev.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - for dev in [host1.eth0, host1.eth1, host2.eth0]: - self._pin_dev_interrupts(dev, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host, dev in [(host1, host1.eth0), (host1, host1.eth1), (host2, host2.eth0)]: - host.run("tc qdisc replace dev %s root mq" % dev.name) + host.vlan0.ip_add(ipaddress(net_addr + '.10' + '.' + str(i+1) + + "/24")) + host.vlan0.ip_add(ipaddress(net_addr6 + ":1::" + str(i+1) + + "/64")) + host.vlan1.ip_add(ipaddress(net_addr + '.20' + '.' + str(i+1) + + "/24")) + host.vlan1.ip_add(ipaddress(net_addr6 + ":2::" + str(i+1) + + "/64")) + host.vlan2.ip_add(ipaddress(net_addr + '.30' + '.' + str(i+1) + + "/24")) + host.vlan2.ip_add(ipaddress(net_addr6 + ":3::" + str(i+1) + + "/64")) + + for dev in [host1.eth0, host1.eth1, host1.team0, host1.vlan0, + host1.vlan1, host1.vlan2, host2.eth0, host2.vlan0, host2.vlan1, + host2.vlan2]: + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in config.test_wide_devices if isinstance(dev, + Vlan) + ]), + "Configured {}.{}.slaves = {}".format( + host1.hostid, host1.team0.name, + ['.'.join([host1.hostid, slave.name]) + for slave in host1.team0.slaves] + ), + "Configured {}.{}.runner_name = {}".format( + host1.hostid, host1.team0.name, + host1.team0.config + ) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + result = [] + for src in [host1.vlan0, host1.vlan1, host1.vlan2]: + for dst in [host2.vlan0, host2.vlan1, host2.vlan2]: + result += [(src, dst)] + return result + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.vlan0, self.matched.host2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + result = [] + for host in [self.matched.host1, self.matched.host2]: + for dev in [host.vlan0, host.vlan1, host.vlan2]: + result.append(dev) + return result + + @property + def mtu_hw_config_dev_list(self): host1, host2 = self.matched.host1, self.matched.host2 + return self.offload_nics + [host1.team0, host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def coalescing_hw_config_dev_list(self): + host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host1.eth1, host2.eth0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list + + def do_ping_tests(self, recipe_config): + for ping_config in self.generate_ping_configurations( + recipe_config): + exp_fail = [] + for pconf in ping_config: + cond = self.vlan_id_same(pconf.client_bind, + pconf.destination_address) + exp_fail.append(cond) + result = self.ping_test(ping_config, exp_fail) + self.ping_evaluate_and_report(ping_config, result) + + def ping_test(self, ping_config, exp_fail): + results = {} + + running_ping_array = [] + for pingconf, fail in zip(ping_config, exp_fail): + ping, client = self.ping_init(pingconf) + running_ping = client.prepare_job(ping, fail=fail) + running_ping.start(bg = True) + running_ping_array.append((pingconf, running_ping)) + + for _, pingjob in running_ping_array: + try: + pingjob.wait() + finally: + pingjob.kill() + + for pingconf, pingjob in running_ping_array: + result = pingjob.result + passed = pingjob.passed + results[pingconf] = (result, passed) + + return results + + def single_ping_evaluate_and_report(self, ping_config, result): + fmt = "From: <{0.client.hostid} ({0.client_bind})> To: " \ + "<{0.destination.hostid} ({0.destination_address})>" + description = fmt.format(ping_config) + if result[0].get("rate", 0) > 50: + message = "Ping successful --- " + description + self.add_result(result[1], message, result[0]) + else: + message = "Ping unsuccessful --- " + description + self.add_result(result[1], message, result[0]) + + def vlan_id_same(self, src_addr, dst_addr): + host1, host2 = self.matched.host1, self.matched.host2 + devs = [] + for dev in (host1.devices + host2.devices): + if src_addr in dev.ips or dst_addr in dev.ips: + devs.append(dev) + try: + return devs[0].vlan_id != devs[1].vlan_id + except (IndexError, AttributeError): + return False diff --git a/lnst/Recipes/ENRT/VlansRecipe.py b/lnst/Recipes/ENRT/VlansRecipe.py index 95a70a2..1b636db 100644 --- a/lnst/Recipes/ENRT/VlansRecipe.py +++ b/lnst/Recipes/ENRT/VlansRecipe.py @@ -1,14 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase1/ -(3_vlans.xml + 3_vlans.py), but 2 Vlans are used -""" from lnst.Common.Parameters import Param from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VlanDevice
-class VlansRecipe(BaseEnrtRecipe): +class VlansRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="net1", driver=RecipeParam("driver"))
@@ -28,69 +29,170 @@ class VlansRecipe(BaseEnrtRecipe): host1.eth0.down() host2.eth0.down()
- host1_vlan_args0 = dict(realdev=host1.eth0, vlan_id=10) - host1_vlan_args1 = dict(realdev=host1.eth0, vlan_id=20) - host2_vlan_args0 = dict(realdev=host2.eth0, vlan_id=10) - host2_vlan_args1 = dict(realdev=host2.eth0, vlan_id=20) - if "mtu" in self.params: - host1.eth0.mtu = self.params.mtu - host2.eth0.mtu = self.params.mtu - for vlan_args in (host1_vlan_args0, host1_vlan_args1, - host2_vlan_args0, host2_vlan_args1): - vlan_args["mtu"] = self.params.mtu - - host1.vlan0 = VlanDevice(**host1_vlan_args0) - host1.vlan1 = VlanDevice(**host1_vlan_args1) - host2.vlan0 = VlanDevice(**host2_vlan_args0) - host2.vlan1 = VlanDevice(**host2_vlan_args1) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.vlan0 - configuration.endpoint2 = host2.vlan0 - - net_addr_1 = "192.168.10" - net_addr_2 = "192.168.20" - net_addr6_1 = "fc00:0:0:1" - net_addr6_2 = "fc00:0:0:2" + host1.vlan0 = VlanDevice(realdev=host1.eth0, vlan_id=10) + host1.vlan1 = VlanDevice(realdev=host1.eth0, vlan_id=20) + host1.vlan2 = VlanDevice(realdev=host1.eth0, vlan_id=30) + host2.vlan0 = VlanDevice(realdev=host2.eth0, vlan_id=10) + host2.vlan1 = VlanDevice(realdev=host2.eth0, vlan_id=20) + host2.vlan2 = VlanDevice(realdev=host2.eth0, vlan_id=30) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [] + for host in [host1, host2]: + configuration.test_wide_devices.extend([host.vlan0, host.vlan1, + host.vlan2]) + + net_addr = "192.168" + net_addr6 = "fc00:0:0"
for i, host in enumerate([host1, host2]): - host.vlan0.ip_add(ipaddress(net_addr_1 + "." + str(i+1) + "/24")) - host.vlan0.ip_add(ipaddress(net_addr6_1 + "::" + str(i+1) + "/64")) - host.vlan1.ip_add(ipaddress(net_addr_2 + "." + str(i+1) + "/24")) - host.vlan1.ip_add(ipaddress(net_addr6_2 + "::" + str(i+1) + "/64")) - - host1.eth0.up() - host1.vlan0.up() - host1.vlan1.up() - host2.eth0.up() - host2.vlan0.up() - host2.vlan1.up() - - if "adaptive_rx_coalescing" in self.params: - for host in [host1, host2]: - host.eth0.adaptive_rx_coalescing = self.params.adaptive_rx_coalescing - if "adaptive_tx_coalescing" in self.params: - for host in [host1, host2]: - host.eth0.adaptive_tx_coalescing = self.params.adaptive_tx_coalescing - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + host.vlan0.ip_add(ipaddress(net_addr + '.10' + '.' + str(i+1) + + "/24")) + host.vlan0.ip_add(ipaddress(net_addr6 + ":1::" + str(i+1) + + "/64")) + host.vlan1.ip_add(ipaddress(net_addr + '.20' + '.' + str(i+1) + + "/24")) + host.vlan1.ip_add(ipaddress(net_addr6 + ":2::" + str(i+1) + + "/64")) + host.vlan2.ip_add(ipaddress(net_addr + '.30' + '.' + str(i+1) + + "/24")) + host.vlan2.ip_add(ipaddress(net_addr6 + ":3::" + str(i+1) + + "/64")) + for dev in [host.eth0, host.vlan0, host.vlan1, host.vlan2]: + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
+ def generate_test_wide_description(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vlan_id = {}".format( + dev.host.hostid, dev.name, dev.vlan_id + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in config.test_wide_devices + ]) + ] + return desc + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + host1, host2 = self.matched.host1, self.matched.host2 + result = [] + for src in [host1.vlan0, host1.vlan1, host1.vlan2]: + for dst in [host2.vlan0, host2.vlan1, host2.vlan2]: + result += [(src, dst)] + return result + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.vlan0, self.matched.host2.vlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + result = [] + for host in [self.matched.host1, self.matched.host2]: + for dev in [host.vlan0, host.vlan1, host.vlan2]: + result.append(dev) + return result + + @property + def mtu_hw_config_dev_list(self): host1, host2 = self.matched.host1, self.matched.host2 + return self.offload_nics + [host1.eth0, host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def coalescing_hw_config_dev_list(self): + host1, host2 = self.matched.host1, self.matched.host2 + return [host1.eth0, host2.eth0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.coalescing_hw_config_dev_list + + def do_ping_tests(self, recipe_config): + for ping_config in self.generate_ping_configurations( + recipe_config): + exp_fail = [] + for pconf in ping_config: + cond = self.vlan_id_same(pconf.client_bind, + pconf.destination_address) + exp_fail.append(cond) + result = self.ping_test(ping_config, exp_fail) + self.ping_evaluate_and_report(ping_config, result) + + def ping_test(self, ping_config, exp_fail): + results = {} + + running_ping_array = [] + for pingconf, fail in zip(ping_config, exp_fail): + ping, client = self.ping_init(pingconf) + running_ping = client.prepare_job(ping, fail=fail) + running_ping.start(bg = True) + running_ping_array.append((pingconf, running_ping)) + + for _, pingjob in running_ping_array: + try: + pingjob.wait() + finally: + pingjob.kill() + + for pingconf, pingjob in running_ping_array: + result = pingjob.result + passed = pingjob.passed + results[pingconf] = (result, passed) + + return results + + def single_ping_evaluate_and_report(self, ping_config, result): + fmt = "From: <{0.client.hostid} ({0.client_bind})> To: " \ + "<{0.destination.hostid} ({0.destination_address})>" + description = fmt.format(ping_config) + if result[0].get("rate", 0) > 50: + message = "Ping successful --- " + description + self.add_result(result[1], message, result[0]) + else: + message = "Ping unsuccessful --- " + description + self.add_result(result[1], message, result[0]) + + def vlan_id_same(self, src_addr, dst_addr): + host1, host2 = self.matched.host1, self.matched.host2 + devs = [] + for dev in (host1.devices + host2.devices): + if src_addr in dev.ips or dst_addr in dev.ips: + devs.append(dev) + try: + return devs[0].vlan_id != devs[1].vlan_id + except (IndexError, AttributeError): + return False diff --git a/lnst/Recipes/ENRT/VxlanMulticastRecipe.py b/lnst/Recipes/ENRT/VxlanMulticastRecipe.py index 53ada96..765a063 100644 --- a/lnst/Recipes/ENRT/VxlanMulticastRecipe.py +++ b/lnst/Recipes/ENRT/VxlanMulticastRecipe.py @@ -1,13 +1,15 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(vxlan_multicast.xml + vxlan_multicast.py) -""" +from itertools import permutations from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import BridgeDevice, VxlanDevice
-class VxlanMulticastRecipe(BaseEnrtRecipe): +class VxlanMulticastRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver")) host1.tap0 = DeviceReq(label="to_guest1") @@ -19,11 +21,11 @@ class VxlanMulticastRecipe(BaseEnrtRecipe): guest1.eth0 = DeviceReq(label="to_guest1")
def test_wide_configuration(self): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1)
- for machine in [host1, host2, guest1]: - machine.eth0.down() - host1.tap0.down() + for dev in [host1.eth0, host2.eth0, guest1.eth0, host1.tap0]: + dev.down()
net_addr = "192.168.0" vxlan_net_addr = "192.168.100" @@ -35,47 +37,104 @@ class VxlanMulticastRecipe(BaseEnrtRecipe): host1.br0.slave_add(host1.eth0) host1.br0.slave_add(host1.tap0)
- for i, (machine, dev) in enumerate([(host1, host1.br0), (guest1, guest1.eth0), - (host2, host2.eth0)]): + for machine in [host1, guest1, host2]: + machine.vxlan0 = VxlanDevice(vxlan_id=1, group=vxlan_group_ip) + + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.br0, host1.vxlan0, + guest1.eth0, guest1.vxlan0, host2.eth0, host2.vxlan0] + + for i, (machine, dev) in enumerate([(host1, host1.br0), + (guest1, guest1.eth0), (host2, host2.eth0)]): dev.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) - machine.vxlan0 = VxlanDevice(vxlan_id='1', group=vxlan_group_ip) machine.vxlan0.realdev = dev - machine.vxlan0.ip_add(ipaddress(vxlan_net_addr + "." + str (i+1) + "/24")) - machine.vxlan0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str (i+1) + "/64")) - - #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.vxlan0 - configuration.endpoint2 = host2.vxlan0 - - if "mtu" in self.params: - for machine in [host1, host2, guest1]: - machine.vxlan0.mtu = self.params.mtu - - for machine in [host1, host2, guest1]: - machine.eth0.up() - host1.tap0.up() - host1.br0.up() - for machine in [host1, host2, guest1]: - machine.vxlan0.up() - - #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + machine.vxlan0.ip_add(ipaddress(vxlan_net_addr + "." + str(i+1) + + "/24")) + machine.vxlan0.ip_add(ipaddress(vxlan_net_addr6 + "::" + + str(i+1) + "/64")) + + for dev in [host1.eth0, host2.eth0, guest1.eth0, host1.tap0, + host1.br0, host1.vxlan0, host2.vxlan0, guest1.vxlan0]: + dev.up() + + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): - host1, host2, guest1 = self.matched.host1, self.matched.host2, self.matched.guest1 + def generate_test_wide_description(self, config): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vxlan_id = {}".format( + dev.host.hostid, dev.name, dev.vxlan_id + ) + for dev in [host1.vxlan0, host2.vxlan0, guest1.vxlan0] + ]), + "\n".join([ + "Configured {}.{}.group = {}".format( + dev.host.hostid, dev.name, dev.group + ) + for dev in [host1.vxlan0, host2.vxlan0, guest1.vxlan0] + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in [host1.vxlan0, host2.vxlan0, guest1.vxlan0] + ]) + ] + return desc
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + devs = [host1.vxlan0, host2.vxlan0, guest1.vxlan0] + return permutations(devs,2) + + def generate_perf_endpoints(self, config): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + return [(self.matched.host1.vxlan0, self.matched.host2.vxlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + return [host1.vxlan0, host2.vxlan0, guest1.vxlan0] + + @property + def mtu_hw_config_dev_list(self): + host1, host2, guest1 = (self.matched.host1, self.matched.host2, + self.matched.guest1) + return [host1.vxlan0, host2.vxlan0, guest1.vxlan0] + + @property + def dev_interrupt_hw_config_dev_list(self): + host1, host2 = (self.matched.host1, self.matched.host2) + return [self.matched.host1.eth0, self.matched.host2.eth0] + + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return self.dev_interrupt_hw_config_dev_list diff --git a/lnst/Recipes/ENRT/VxlanRemoteRecipe.py b/lnst/Recipes/ENRT/VxlanRemoteRecipe.py index 99e2522..828e4c7 100644 --- a/lnst/Recipes/ENRT/VxlanRemoteRecipe.py +++ b/lnst/Recipes/ENRT/VxlanRemoteRecipe.py @@ -1,13 +1,14 @@ -""" -Implements scenario similar to regression_tests/phase3/ -(vxlan_remote.xml + vxlan_remote.py) -""" from lnst.Common.IpAddress import ipaddress from lnst.Controller import HostReq, DeviceReq, RecipeParam -from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe, EnrtConfiguration +from lnst.Recipes.ENRT.BaseEnrtRecipe import BaseEnrtRecipe +from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( + OffloadSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.CommonHWConfigMixin import ( + CommonHWConfigMixin) from lnst.Devices import VxlanDevice
-class VxlanRemoteRecipe(BaseEnrtRecipe): +class VxlanRemoteRecipe(OffloadSubConfigMixin, CommonHWConfigMixin, + BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="to_switch", driver=RecipeParam("driver"))
@@ -26,41 +27,91 @@ class VxlanRemoteRecipe(BaseEnrtRecipe):
for i, host in enumerate([host1, host2]): host.eth0.ip_add(ipaddress(net_addr + "." + str(i+1) + "/24")) - host.vxlan0 = VxlanDevice(vxlan_id='1', remote=net_addr + "." + str(2-i)) - host.vxlan0.realdev = host.eth0 - host.vxlan0.ip_add(ipaddress(vxlan_net_addr + "." + str (i+1) + "/24")) - host.vxlan0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str (i+1) + "/64")) + host.vxlan0 = VxlanDevice(vxlan_id='1', remote=net_addr + + "." + str(2-i))
- #Due to limitations in the current EnrtConfiguration - #class, a single vlan test pair is chosen - configuration = EnrtConfiguration() - configuration.endpoint1 = host1.vxlan0 - configuration.endpoint2 = host2.vxlan0 + configuration = super().test_wide_configuration() + configuration.test_wide_devices = [host1.eth0, host1.vxlan0, + host2.eth0, host2.vxlan0]
- if "mtu" in self.params: - for host in [host1, host2]: - host.vxlan0.mtu = self.params.mtu + for i, host in enumerate([host1, host2]): + host.vxlan0.realdev = host.eth0 + host.vxlan0.ip_add(ipaddress(vxlan_net_addr + "." + str(i+1) + + "/24")) + host.vxlan0.ip_add(ipaddress(vxlan_net_addr6 + "::" + str(i+1) + + "/64"))
for host in [host1, host2]: host.eth0.up() host.vxlan0.up()
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance stop") - self._pin_dev_interrupts(host.eth0, self.params.dev_intr_cpu) - - if self.params.perf_parallel_streams > 1: - for host in [host1, host2]: - host.run("tc qdisc replace dev %s root mq" % host.eth0.name) + self.wait_tentative_ips(configuration.test_wide_devices)
return configuration
- def test_wide_deconfiguration(self, config): + def generate_test_wide_description(self, config): host1, host2 = self.matched.host1, self.matched.host2 + desc = super().generate_test_wide_description(config) + desc += [ + "\n".join([ + "Configured {}.{}.ips = {}".format( + dev.host.hostid, dev.name, dev.ips + ) + for dev in config.test_wide_devices + ]), + "\n".join([ + "Configured {}.{}.vxlan_id = {}".format( + dev.host.hostid, dev.name, dev.vxlan_id + ) + for dev in [host1.vxlan0, host2.vxlan0] + ]), + "\n".join([ + "Configured {}.{}.remote = {}".format( + dev.host.hostid, dev.name, dev.remote + ) + for dev in [host1.vxlan0, host2.vxlan0] + ]), + "\n".join([ + "Configured {}.{}.realdev = {}".format( + dev.host.hostid, dev.name, + '.'.join([dev.host.hostid, dev.realdev.name]) + ) + for dev in [host1.vxlan0, host2.vxlan0] + ]) + ] + return desc + + def test_wide_deconfiguration(self, config): + del config.test_wide_devices + + super().test_wide_deconfiguration(config) + + def generate_ping_endpoints(self, config): + return [(self.matched.host1.vxlan0, self.matched.host2.vxlan0)] + + def generate_perf_endpoints(self, config): + return [(self.matched.host1.vxlan0, self.matched.host2.vxlan0)] + + def wait_tentative_ips(self, devices): + def condition(): + return all( + [not ip.is_tentative for dev in devices for ip in dev.ips] + ) + + self.ctl.wait_for_condition(condition, timeout=5) + + @property + def offload_nics(self): + return [self.matched.host1.vxlan0, self.matched.host2.vxlan0] + + @property + def mtu_hw_config_dev_list(self): + return [self.matched.host1.vxlan0, self.matched.host2.vxlan0] + + @property + def dev_interrupt_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
- #TODO better service handling through HostAPI - if "dev_intr_cpu" in self.params: - for host in [host1, host2]: - host.run("service irqbalance start") + @property + def parallel_stream_qdisc_hw_config_dev_list(self): + return [self.matched.host1.eth0, self.matched.host2.eth0]
On Fri, Jul 19, 2019 at 01:36:51PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
Rewrite recipes to be compatible with the above patches.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com
lnst/Recipes/ENRT/BondRecipe.py | 137 +++++--- lnst/Recipes/ENRT/DoubleBondRecipe.py | 140 +++++--- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 159 ++++++---- lnst/Recipes/ENRT/IpsecEspAeadRecipe.py | 280 ++++++++-------- lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py | 297 ++++++++--------- lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py | 130 +++++--- lnst/Recipes/ENRT/PingFloodRecipe.py | 21 +- .../ENRT/ShortLivedConnectionsRecipe.py | 87 ++--- lnst/Recipes/ENRT/SimpleMacsecRecipe.py | 252 +++++++-------- lnst/Recipes/ENRT/SimplePerfRecipe.py | 42 +-- lnst/Recipes/ENRT/TeamRecipe.py | 134 +++++--- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 162 ++++++---- lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py | 230 +++++++++++--- .../VirtualBridgeVlanInGuestMirroredRecipe.py | 191 ++++++----- .../ENRT/VirtualBridgeVlanInGuestRecipe.py | 167 ++++++---- .../VirtualBridgeVlanInHostMirroredRecipe.py | 178 +++++++---- .../ENRT/VirtualBridgeVlanInHostRecipe.py | 155 ++++++--- .../ENRT/VirtualBridgeVlansOverBondRecipe.py | 298 +++++++++++++----- ...rtualOvsBridgeVlanInGuestMirroredRecipe.py | 193 +++++++----- .../ENRT/VirtualOvsBridgeVlanInGuestRecipe.py | 164 ++++++---- ...irtualOvsBridgeVlanInHostMirroredRecipe.py | 155 +++++---- .../ENRT/VirtualOvsBridgeVlanInHostRecipe.py | 144 ++++++--- .../VirtualOvsBridgeVlansOverBondRecipe.py | 277 ++++++++++------ lnst/Recipes/ENRT/VlansOverBondRecipe.py | 267 +++++++++++----- lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 269 +++++++++++----- lnst/Recipes/ENRT/VlansRecipe.py | 230 ++++++++++---- lnst/Recipes/ENRT/VxlanMulticastRecipe.py | 155 ++++++--- lnst/Recipes/ENRT/VxlanRemoteRecipe.py | 115 +++++-- 28 files changed, 3193 insertions(+), 1836 deletions(-)
I only checked SimplePerfRecipe as that's the one I'm currently interested in getting production ready and I disagree with removing the ping endpoints.
I don't think we need to keep compatibility with the old simple_netperf recipe anymore and we can implement the test in a way that makes sense "now".
Maybe we should consider renaming the recipe class to just "SimpleNetworkRecipe" or something that just reflects the fact that there's only 2 eth nics that are being tested without any special configuration?
-Ondrej
----- Original Message -----
From: "Ondrej Lichtner" olichtne@redhat.com To: csfakian@redhat.com Cc: lnst-developers@lists.fedorahosted.org Sent: Monday, July 22, 2019 4:03:08 PM Subject: Re: [PATCH-next 8/8] lnst.Recipes.ENRT: rework recipes to adopt to 334e2c..7bde5c
On Fri, Jul 19, 2019 at 01:36:51PM +0200, csfakian@redhat.com wrote:
From: Christos Sfakianakis csfakian@redhat.com
Rewrite recipes to be compatible with the above patches.
Signed-off-by: Christos Sfakianakis csfakian@redhat.com
lnst/Recipes/ENRT/BondRecipe.py | 137 +++++--- lnst/Recipes/ENRT/DoubleBondRecipe.py | 140 +++++--- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 159 ++++++---- lnst/Recipes/ENRT/IpsecEspAeadRecipe.py | 280 ++++++++-------- lnst/Recipes/ENRT/IpsecEspAhCompRecipe.py | 297 ++++++++--------- lnst/Recipes/ENRT/NoVirtOvsVxlanRecipe.py | 130 +++++--- lnst/Recipes/ENRT/PingFloodRecipe.py | 21 +- .../ENRT/ShortLivedConnectionsRecipe.py | 87 ++--- lnst/Recipes/ENRT/SimpleMacsecRecipe.py | 252 +++++++-------- lnst/Recipes/ENRT/SimplePerfRecipe.py | 42 +-- lnst/Recipes/ENRT/TeamRecipe.py | 134 +++++--- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 162 ++++++---- lnst/Recipes/ENRT/VirtOvsVxlanRecipe.py | 230 +++++++++++--- .../VirtualBridgeVlanInGuestMirroredRecipe.py | 191 ++++++----- .../ENRT/VirtualBridgeVlanInGuestRecipe.py | 167 ++++++---- .../VirtualBridgeVlanInHostMirroredRecipe.py | 178 +++++++---- .../ENRT/VirtualBridgeVlanInHostRecipe.py | 155 ++++++--- .../ENRT/VirtualBridgeVlansOverBondRecipe.py | 298 +++++++++++++----- ...rtualOvsBridgeVlanInGuestMirroredRecipe.py | 193 +++++++----- .../ENRT/VirtualOvsBridgeVlanInGuestRecipe.py | 164 ++++++---- ...irtualOvsBridgeVlanInHostMirroredRecipe.py | 155 +++++---- .../ENRT/VirtualOvsBridgeVlanInHostRecipe.py | 144 ++++++--- .../VirtualOvsBridgeVlansOverBondRecipe.py | 277 ++++++++++------ lnst/Recipes/ENRT/VlansOverBondRecipe.py | 267 +++++++++++----- lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 269 +++++++++++----- lnst/Recipes/ENRT/VlansRecipe.py | 230 ++++++++++---- lnst/Recipes/ENRT/VxlanMulticastRecipe.py | 155 ++++++--- lnst/Recipes/ENRT/VxlanRemoteRecipe.py | 115 +++++-- 28 files changed, 3193 insertions(+), 1836 deletions(-)
I only checked SimplePerfRecipe as that's the one I'm currently interested in getting production ready and I disagree with removing the ping endpoints.
I don't think we need to keep compatibility with the old simple_netperf recipe anymore and we can implement the test in a way that makes sense "now".
Sounds good, I will keep the ping tests as part of this recipe.
Maybe we should consider renaming the recipe class to just "SimpleNetworkRecipe" or something that just reflects the fact that there's only 2 eth nics that are being tested without any special configuration?
Sounds reasonable to me, will do so unless Jan has a different preference for the name.
-Ondrej
lnst-developers@lists.fedorahosted.org