[RFC] IP Address Generators
by Radek Pazdera
Hello everyone!
I'm sending a proposal we might have already discussed at some point. At
the moment, it is required to write some specific IP addresses to the
recipes' configuration section to each interface.
This is usually not a problem, but theoretically, there can be collisions
with addresses you use somewhere else (for instance in the controller
network or to connect your slaves to the internet or whatever).
This could be problem particularly when we start building a large
library of these recipes. We could make the recipes even more portable
by leting the LNST controller chose which address to assign to different
interfaces.
We could do this by adding a special value to the address value field,
which should be very easy to implement:
<addresses>
<address value="auto"/>
</addresses>
In this case, the parser would resolve it to a valid ip address within
some pool that was allocated to the network segment the interface is
plugged into.
We could then maybe add an address type and more:
<addresses>
<address value="auto" type="ipv6"/>
</addresses>
Any feedback is appreciated :).
Cheers,
-Radek
11 years, 1 month
[PATCH] NetTestParse: Fix handler exceptions
by Radek Pazdera
The parser wraps exceptions that go through his handler hooks to his own
exception class called XmlProcessingError to add information about
at which line of the XML file the problem happened.
Doing this however results in discarding the original traceback, which
is replaced by different one that leads to the try/catch block. This
is not helpful at all. It actually makes debugging extremelly irritating.
This patch changes this behaviour so the parser only logs an error with
the line information and it re-raises the original exeption with the
original traceback.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Controller/NetTestParse.py | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/lnst/Controller/NetTestParse.py b/lnst/Controller/NetTestParse.py
index 39d23bd..d3e9bbc 100644
--- a/lnst/Controller/NetTestParse.py
+++ b/lnst/Controller/NetTestParse.py
@@ -39,10 +39,7 @@ class NetTestParse(RecipeParser):
first_pass = FirstPass(self)
first_pass.parse(xml_dom)
- try:
- self._trigger_event("provisioning_requirements_ready", {})
- except Exception as exc:
- raise XmlProcessingError(str(exc), xml_dom)
+ self._trigger_event("provisioning_requirements_ready", {})
second_pass = SecondPass(self)
second_pass.parse(xml_dom)
@@ -180,7 +177,8 @@ class MachineParse(RecipeParser):
try:
self._trigger_event("machine_ready", {"machine_id": self._id})
except Exception as exc:
- raise XmlProcessingError(str(exc), node)
+ logging.error(XmlProcessingError(str(exc), xml_dom))
+ raise
class ParamsParse(RecipeParser):
_params = None
@@ -337,7 +335,8 @@ class SlaveMachineParse(RecipeParser):
self._trigger_event("netdevice_ready",
{"machine_id": self._machine_id, "dev_id": phys_id})
except Exception as exc:
- raise XmlProcessingError(str(exc), node)
+ logging.error(XmlProcessingError(str(exc), xml_dom))
+ raise
class NetConfigParse(RecipeParser):
_machine_id = None
@@ -393,7 +392,8 @@ class NetConfigParse(RecipeParser):
except Exception as exc:
msg = "Unable to configure interface %s on machine %s [%s]." % \
(dev_id, self._machine_id, str(exc))
- raise XmlProcessingError(msg, node)
+ logging.error(XmlProcessingError(str(msg), xml_dom))
+ raise
def _process_phys_id_attr(self, node, dev):
netconfig = self._netconfig
--
1.7.7.6
11 years, 1 month
[lnst] NetTestParse: Fix handler exceptions
by Jiří Pírko
commit bc72238a3096320a416e7e4e86db8fe78c63aa81
Author: Radek Pazdera <rpazdera(a)redhat.com>
Date: Fri Mar 8 11:07:18 2013 +0100
NetTestParse: Fix handler exceptions
The parser wraps exceptions that go through his handler hooks to his own
exception class called XmlProcessingError to add information about
at which line of the XML file the problem happened.
Doing this however results in discarding the original traceback, which
is replaced by different one that leads to the try/catch block. This
is not helpful at all. It actually makes debugging extremelly irritating.
This patch changes this behaviour so the parser only logs an error with
the line information and it re-raises the original exeption with the
original traceback.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
lnst/Controller/NetTestParse.py | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
---
diff --git a/lnst/Controller/NetTestParse.py b/lnst/Controller/NetTestParse.py
index 257fe77..430c033 100644
--- a/lnst/Controller/NetTestParse.py
+++ b/lnst/Controller/NetTestParse.py
@@ -36,10 +36,7 @@ class NetTestParse(RecipeParser):
first_pass = FirstPass(self)
first_pass.parse(xml_dom)
- try:
- self._trigger_event("provisioning_requirements_ready", {})
- except Exception as exc:
- raise XmlProcessingError(str(exc), xml_dom)
+ self._trigger_event("provisioning_requirements_ready", {})
second_pass = SecondPass(self)
second_pass.parse(xml_dom)
@@ -175,7 +172,8 @@ class MachineParse(RecipeParser):
try:
self._trigger_event("machine_ready", {"machine_id": self._id})
except Exception as exc:
- raise XmlProcessingError(str(exc), node)
+ logging.error(XmlProcessingError(str(exc), xml_dom))
+ raise
class ParamsParse(RecipeParser):
_params = None
@@ -332,7 +330,8 @@ class SlaveMachineParse(RecipeParser):
self._trigger_event("netdevice_ready",
{"machine_id": self._machine_id, "dev_id": phys_id})
except Exception as exc:
- raise XmlProcessingError(str(exc), node)
+ logging.error(XmlProcessingError(str(exc), xml_dom))
+ raise
class NetConfigParse(RecipeParser):
_machine_id = None
@@ -388,7 +387,8 @@ class NetConfigParse(RecipeParser):
except Exception as exc:
msg = "Unable to configure interface %s on machine %s [%s]." % \
(dev_id, self._machine_id, str(exc))
- raise XmlProcessingError(msg, node)
+ logging.error(XmlProcessingError(str(msg), xml_dom))
+ raise
def _process_phys_id_attr(self, node, dev):
netconfig = self._netconfig
11 years, 1 month
[lnst] XmlTemplates: Accessing $recipe no longer allowed
by Jiří Pírko
commit 032cc40be7902533ce62d8bfc514339f5138c7e7
Author: Radek Pazdera <rpazdera(a)redhat.com>
Date: Fri Mar 8 10:14:37 2013 +0100
XmlTemplates: Accessing $recipe no longer allowed
This commit removes the support for accessing the recipe dictionary
using the $recipe template variable.
These are implementation details and we should not allow users to
access them directly.
Additionally, we will do a major refactoring of the NetTestController
class in the near future and this variable might even no longer exist
after that.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
lnst/Common/XmlProcessing.py | 1 +
lnst/Common/XmlTemplates.py | 52 +++++++++++++---------------------
lnst/Controller/NetTestController.py | 13 ++++----
lnst/Controller/NetTestParse.py | 5 ---
4 files changed, 28 insertions(+), 43 deletions(-)
---
diff --git a/lnst/Common/XmlProcessing.py b/lnst/Common/XmlProcessing.py
index 839495d..80e9aca 100644
--- a/lnst/Common/XmlProcessing.py
+++ b/lnst/Common/XmlProcessing.py
@@ -237,6 +237,7 @@ class RecipeParser(XmlParser):
def set_recipe(self, recipe):
self._recipe = recipe
+ self._template_proc.set_machines(recipe["machines"])
def set_definitions(self, defs):
self._template_proc.set_definitions(defs)
diff --git a/lnst/Common/XmlTemplates.py b/lnst/Common/XmlTemplates.py
index 8f0d6e3..218031d 100644
--- a/lnst/Common/XmlTemplates.py
+++ b/lnst/Common/XmlTemplates.py
@@ -29,7 +29,7 @@ class XmlTemplateError(Exception):
class XmlTemplates:
""" This class serves as template processor """
- _alias_re = "\{\$([a-zA-Z0-9_]+)(\[[^{}]+\])?\}"
+ _alias_re = "\{\$([a-zA-Z0-9_]+)\}"
_func_re = "\{([a-zA-Z0-9_]+)\(([^\(\)]*)\)\}"
def __init__(self, definitions=None):
@@ -38,7 +38,8 @@ class XmlTemplates:
else:
self._definitions = [{}]
- self._reserved_aliases = ["recipe"]
+ self._machines = {}
+ self._reserved_aliases = []
def set_definitions(self, defs):
""" Set alias definitions
@@ -63,6 +64,14 @@ class XmlTemplates:
return defs
+ def set_machines(self, machines):
+ """ Assign machine information
+
+ XmlTemplates use these information about the machines
+ to resolve template functions within the recipe.
+ """
+ self._machines = machines
+
def define_alias(self, name, value, skip_reserved_check=False):
""" Associate an alias name with some value
@@ -163,20 +172,8 @@ class XmlTemplates:
alias_match = re.match(self._alias_re, string)
if alias_match:
alias_name = alias_match.group(1)
- array_subscript = alias_match.group(2)
-
alias_obj = self._find_definition(alias_name)
- if array_subscript != None:
- try:
- result = str(eval("alias_obj%s" % array_subscript))
- except (KeyError, IndexError):
- err = "Wrong array subscript in '%s%s'" \
- % (alias_name, array_subscript)
- raise XmlTemplateError(err)
- else:
- result = alias_obj
-
return result
def _process_func_template(self, string):
@@ -217,17 +214,17 @@ class XmlTemplates:
def _ip_func(self, params):
self._validate_func_params("ip", params, 2, 1)
- recipe = self._get_recipe_data("ip")
+ machines = self._machines
m_id = params[0]
if_id = params[1]
ip_id = int(params[2]) if len(params) == 3 else 0
- if 'machines' not in recipe or m_id not in recipe['machines']:
+ if m_id not in machines:
msg = "First parameter of function ip() is invalid: "\
"Machine %s does not exist." % m_id
raise XmlTemplateError(msg)
- machine = recipe["machines"][m_id]
+ machine = machines[m_id]
if if_id not in machine['netconfig']:
@@ -245,15 +242,15 @@ class XmlTemplates:
def _hwaddr_func(self, params):
self._validate_func_params("hwaddr", params, 2, 0)
- recipe = self._get_recipe_data("hwaddr")
+ machines = self._machines
m_id = params[0]
if_id = params[1]
- if 'machines' not in recipe or m_id not in recipe['machines']:
+ if m_id not in machines:
msg = "First parameter of function hwaddr() is invalid: "\
"Machine %s does not exist." % m_id
raise XmlTemplateError(msg)
- machine = recipe["machines"][m_id]
+ machine = machines[m_id]
if if_id not in machine['netconfig']:
msg = "Second parameter of function hwaddr() is invalid: "\
@@ -266,15 +263,15 @@ class XmlTemplates:
def _devname_func(self, params):
self._validate_func_params("devname", params, 2, 0)
- recipe = self._get_recipe_data("devname")
+ machines = self._machines
m_id = params[0]
if_id = params[1]
- if 'machines' not in recipe or m_id not in recipe['machines']:
+ if m_id not in machines:
msg = "First parameter of function devname() is invalid: "\
"Machine %s does not exist." % m_id
raise XmlTemplateError(msg)
- machine = recipe["machines"][m_id]
+ machine = machine[m_id]
if if_id not in machine['netconfig']:
msg = "Second parameter of function devname() is invalid: "\
@@ -301,12 +298,3 @@ class XmlTemplates:
except ValueError:
err = "Non-integer parameter passed to '%s'" % name
raise XmlTemplateError(err)
-
- def _get_recipe_data(self, template_name):
- try:
- recipe = self._find_definition("recipe")
- return recipe
- except XmlTemplateError as err:
- msg = "Cannot resolve %s(): " % template_name
- msg += str(err)
- raise XmlTemplateError(msg)
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index d686475..80a132d 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -51,10 +51,11 @@ class NetTestController:
check_process_running("libvirtd"))
self._slave_pool = sp
- self._recipe = {}
- definitions = {"recipe": self._recipe}
-
- self._recipe["networks"] = {}
+ self._recipe = recipe = {}
+ recipe["networks"] = {}
+ recipe["machines"] = {}
+ recipe["provisioning"] = {}
+ recipe["switches"] = {}
mac_pool_range = config.get_option('environment', 'mac_pool_range')
self._mac_pool = MacPool(mac_pool_range[0],
@@ -62,7 +63,6 @@ class NetTestController:
ntparse = NetTestParse(recipe_path)
ntparse.set_recipe(self._recipe)
- ntparse.set_definitions(definitions)
ntparse.register_event_handler("provisioning_requirements_ready",
self._prepare_provisioning)
@@ -115,7 +115,8 @@ class NetTestController:
msg = "This setup cannot be provisioned with the current pool."
raise NetTestError(msg)
- self._recipe["machines"] = machines
+ for m_id, machine in machines.iteritems():
+ self._recipe["machines"][m_id] = machine
provisioning["map"] = {}
logging.info("Provisioning initialized")
diff --git a/lnst/Controller/NetTestParse.py b/lnst/Controller/NetTestParse.py
index 39d23bd..257fe77 100644
--- a/lnst/Controller/NetTestParse.py
+++ b/lnst/Controller/NetTestParse.py
@@ -28,9 +28,6 @@ class NetTestParse(RecipeParser):
self._rp = RecipePath(None, self._filepath)
self._include_root = self._rp.get_root()
- self._recipe = {}
- self._template_proc.set_definitions({"recipe": self._recipe})
-
def parse_recipe(self):
dom_init = XmlDomTreeInit()
rp = self._rp
@@ -58,7 +55,6 @@ class FirstPass(RecipeParser):
"""
def parse(self, node):
- self._recipe["provisioning"] = {}
self._recipe["provisioning"]["setup_requirements"] = {}
if node.nodeType == node.DOCUMENT_NODE:
@@ -132,7 +128,6 @@ class SecondPass(RecipeParser):
subparser.parse(node)
def _switches(self, node, params):
- self._recipe["switches"] = {}
scheme = {"switch": self._switch}
self._process_child_nodes(node, scheme)
11 years, 1 month
[PATCH] XmlTemplates: Accessing $recipe no longer allowed
by Radek Pazdera
This commit removes the support for accessing the recipe dictionary
using the $recipe template variable.
These are implementation details and we should not allow users to
access them directly.
Additionally, we will do a major refactoring of the NetTestController
class in the near future and this variable might even no longer exist
after that.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Common/XmlProcessing.py | 1 +
lnst/Common/XmlTemplates.py | 52 +++++++++++++---------------------
lnst/Controller/NetTestController.py | 13 ++++----
lnst/Controller/NetTestParse.py | 5 ---
4 files changed, 28 insertions(+), 43 deletions(-)
diff --git a/lnst/Common/XmlProcessing.py b/lnst/Common/XmlProcessing.py
index 839495d..80e9aca 100644
--- a/lnst/Common/XmlProcessing.py
+++ b/lnst/Common/XmlProcessing.py
@@ -237,6 +237,7 @@ class RecipeParser(XmlParser):
def set_recipe(self, recipe):
self._recipe = recipe
+ self._template_proc.set_machines(recipe["machines"])
def set_definitions(self, defs):
self._template_proc.set_definitions(defs)
diff --git a/lnst/Common/XmlTemplates.py b/lnst/Common/XmlTemplates.py
index 8f0d6e3..218031d 100644
--- a/lnst/Common/XmlTemplates.py
+++ b/lnst/Common/XmlTemplates.py
@@ -29,7 +29,7 @@ class XmlTemplateError(Exception):
class XmlTemplates:
""" This class serves as template processor """
- _alias_re = "\{\$([a-zA-Z0-9_]+)(\[[^{}]+\])?\}"
+ _alias_re = "\{\$([a-zA-Z0-9_]+)\}"
_func_re = "\{([a-zA-Z0-9_]+)\(([^\(\)]*)\)\}"
def __init__(self, definitions=None):
@@ -38,7 +38,8 @@ class XmlTemplates:
else:
self._definitions = [{}]
- self._reserved_aliases = ["recipe"]
+ self._machines = {}
+ self._reserved_aliases = []
def set_definitions(self, defs):
""" Set alias definitions
@@ -63,6 +64,14 @@ class XmlTemplates:
return defs
+ def set_machines(self, machines):
+ """ Assign machine information
+
+ XmlTemplates use these information about the machines
+ to resolve template functions within the recipe.
+ """
+ self._machines = machines
+
def define_alias(self, name, value, skip_reserved_check=False):
""" Associate an alias name with some value
@@ -163,20 +172,8 @@ class XmlTemplates:
alias_match = re.match(self._alias_re, string)
if alias_match:
alias_name = alias_match.group(1)
- array_subscript = alias_match.group(2)
-
alias_obj = self._find_definition(alias_name)
- if array_subscript != None:
- try:
- result = str(eval("alias_obj%s" % array_subscript))
- except (KeyError, IndexError):
- err = "Wrong array subscript in '%s%s'" \
- % (alias_name, array_subscript)
- raise XmlTemplateError(err)
- else:
- result = alias_obj
-
return result
def _process_func_template(self, string):
@@ -217,17 +214,17 @@ class XmlTemplates:
def _ip_func(self, params):
self._validate_func_params("ip", params, 2, 1)
- recipe = self._get_recipe_data("ip")
+ machines = self._machines
m_id = params[0]
if_id = params[1]
ip_id = int(params[2]) if len(params) == 3 else 0
- if 'machines' not in recipe or m_id not in recipe['machines']:
+ if m_id not in machines:
msg = "First parameter of function ip() is invalid: "\
"Machine %s does not exist." % m_id
raise XmlTemplateError(msg)
- machine = recipe["machines"][m_id]
+ machine = machines[m_id]
if if_id not in machine['netconfig']:
@@ -245,15 +242,15 @@ class XmlTemplates:
def _hwaddr_func(self, params):
self._validate_func_params("hwaddr", params, 2, 0)
- recipe = self._get_recipe_data("hwaddr")
+ machines = self._machines
m_id = params[0]
if_id = params[1]
- if 'machines' not in recipe or m_id not in recipe['machines']:
+ if m_id not in machines:
msg = "First parameter of function hwaddr() is invalid: "\
"Machine %s does not exist." % m_id
raise XmlTemplateError(msg)
- machine = recipe["machines"][m_id]
+ machine = machines[m_id]
if if_id not in machine['netconfig']:
msg = "Second parameter of function hwaddr() is invalid: "\
@@ -266,15 +263,15 @@ class XmlTemplates:
def _devname_func(self, params):
self._validate_func_params("devname", params, 2, 0)
- recipe = self._get_recipe_data("devname")
+ machines = self._machines
m_id = params[0]
if_id = params[1]
- if 'machines' not in recipe or m_id not in recipe['machines']:
+ if m_id not in machines:
msg = "First parameter of function devname() is invalid: "\
"Machine %s does not exist." % m_id
raise XmlTemplateError(msg)
- machine = recipe["machines"][m_id]
+ machine = machine[m_id]
if if_id not in machine['netconfig']:
msg = "Second parameter of function devname() is invalid: "\
@@ -301,12 +298,3 @@ class XmlTemplates:
except ValueError:
err = "Non-integer parameter passed to '%s'" % name
raise XmlTemplateError(err)
-
- def _get_recipe_data(self, template_name):
- try:
- recipe = self._find_definition("recipe")
- return recipe
- except XmlTemplateError as err:
- msg = "Cannot resolve %s(): " % template_name
- msg += str(err)
- raise XmlTemplateError(msg)
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index d686475..80a132d 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -51,10 +51,11 @@ class NetTestController:
check_process_running("libvirtd"))
self._slave_pool = sp
- self._recipe = {}
- definitions = {"recipe": self._recipe}
-
- self._recipe["networks"] = {}
+ self._recipe = recipe = {}
+ recipe["networks"] = {}
+ recipe["machines"] = {}
+ recipe["provisioning"] = {}
+ recipe["switches"] = {}
mac_pool_range = config.get_option('environment', 'mac_pool_range')
self._mac_pool = MacPool(mac_pool_range[0],
@@ -62,7 +63,6 @@ class NetTestController:
ntparse = NetTestParse(recipe_path)
ntparse.set_recipe(self._recipe)
- ntparse.set_definitions(definitions)
ntparse.register_event_handler("provisioning_requirements_ready",
self._prepare_provisioning)
@@ -115,7 +115,8 @@ class NetTestController:
msg = "This setup cannot be provisioned with the current pool."
raise NetTestError(msg)
- self._recipe["machines"] = machines
+ for m_id, machine in machines.iteritems():
+ self._recipe["machines"][m_id] = machine
provisioning["map"] = {}
logging.info("Provisioning initialized")
diff --git a/lnst/Controller/NetTestParse.py b/lnst/Controller/NetTestParse.py
index 39d23bd..257fe77 100644
--- a/lnst/Controller/NetTestParse.py
+++ b/lnst/Controller/NetTestParse.py
@@ -28,9 +28,6 @@ class NetTestParse(RecipeParser):
self._rp = RecipePath(None, self._filepath)
self._include_root = self._rp.get_root()
- self._recipe = {}
- self._template_proc.set_definitions({"recipe": self._recipe})
-
def parse_recipe(self):
dom_init = XmlDomTreeInit()
rp = self._rp
@@ -58,7 +55,6 @@ class FirstPass(RecipeParser):
"""
def parse(self, node):
- self._recipe["provisioning"] = {}
self._recipe["provisioning"]["setup_requirements"] = {}
if node.nodeType == node.DOCUMENT_NODE:
@@ -132,7 +128,6 @@ class SecondPass(RecipeParser):
subparser.parse(node)
def _switches(self, node, params):
- self._recipe["switches"] = {}
scheme = {"switch": self._switch}
self._process_child_nodes(node, scheme)
--
1.7.7.6
11 years, 1 month
[PATCH 1/2] ctl: MachinePool renamed to SlavePool
by Radek Pazdera
This commit renames the MachinePool class to slave pool.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Controller/MachinePool.py | 539 ----------------------------------
lnst/Controller/NetTestController.py | 33 +-
lnst/Controller/SlavePool.py | 539 ++++++++++++++++++++++++++++++++++
3 files changed, 555 insertions(+), 556 deletions(-)
delete mode 100644 lnst/Controller/MachinePool.py
create mode 100644 lnst/Controller/SlavePool.py
diff --git a/lnst/Controller/MachinePool.py b/lnst/Controller/MachinePool.py
deleted file mode 100644
index 92ddf41..0000000
--- a/lnst/Controller/MachinePool.py
+++ /dev/null
@@ -1,539 +0,0 @@
-"""
-This module contains implementaion of MachinePool class that
-can be used to maintain a cluster of test machines.
-
-These machines can be provisioned and used in test recipes.
-
-Copyright 2012 Red Hat, Inc.
-Licensed under the GNU General Public License, version 2 as
-published by the Free Software Foundation; see COPYING for details.
-"""
-
-__author__ = """
-rpazdera(a)redhat.com (Radek Pazdera)
-"""
-
-import logging
-import os
-import re
-import copy
-from xml.dom import minidom
-from lnst.Common.XmlProcessing import XmlDomTreeInit
-from lnst.Controller.NetTestParse import MachineConfigParse
-
-class MachinePool:
- """
- This class is responsible for managing test machines that
- are available at the controler and can be used for testing.
- """
-
- _map = {}
- _pool = {}
-
- _machine_matches = []
- _network_matches = []
-
- _allow_virtual = False
-
- def __init__(self, pool_dirs, allow_virtual=False):
- self._allow_virtual = allow_virtual
- for pool_dir in pool_dirs:
- self.add_dir(pool_dir)
-
- def add_dir(self, pool_dir):
- dentries = os.listdir(pool_dir)
-
- for dirent in dentries:
- self.add_file("%s/%s" % (pool_dir, dirent))
-
- def add_file(self, filepath):
- if os.path.isfile(filepath) and re.search("\.xml$", filepath, re.I):
- dom_init = XmlDomTreeInit()
- dom = dom_init.parse_file(filepath)
-
- dirname, basename = os.path.split(filepath)
-
- parser = MachineConfigParse()
- parser.set_include_root(dirname)
- parser.disable_events()
-
- machine = {"info": {}, "netdevices": {}}
- machine_id = re.sub("\.xml$", "", basename, flags=re.I)
- parser.set_machine(machine_id, machine)
-
- machineconfig = dom.getElementsByTagName("machineconfig")[0]
- machine["dom_node_ref"] = machineconfig
-
- parser.parse(machineconfig)
- if 'libvirt_domain' not in machine['info'] or self._allow_virtual:
- self._pool[machine_id] = machine
- else:
- logging.warning("libvirtd not found- Machine Pool skipping "\
- "machine %s" % machine_id)
-
- def provision_setup(self, setup_requirements):
- """
- This method will try to map a dictionary of machines'
- requirements to a pool of machines that is available to
- this instance.
-
- :param templates: Setup request (dict of required machines)
- :type templates: dict
-
- :return: XML machineconfigs of requested machines
- :rtype: dict
- """
-
- mapper = SetupMapper()
- self._map = mapper.map_setup(setup_requirements, self._pool)
-
- if self._map == None:
- return None
-
- configs = {}
- for m_id in self._map["machines"]:
- configs[m_id] = self._get_mapped_machineconfig_xml(m_id)
-
- return configs
-
- def get_provisioner_id(self, m_id):
- try:
- return self._get_machine_mapping(m_id)
- except KeyError:
- return None
-
- def get_provisioner(self, m_id):
- try:
- p_id = self._get_machine_mapping(m_id)
- return self._pool[p_id]
- except KeyError:
- return None
-
- def _get_machine_mapping(self, m_id):
- return self._map["machines"][m_id]["target"]
-
- def _get_interface_mapping(self, m_id, if_id):
- return self._map["machines"][m_id]["interfaces"][if_id]
-
- def _get_network_mapping(self, net_id):
- return self._map["networks"][net_id]
-
- def _get_mapped_machineconfig_xml(self, tm_id):
- pm_id = self._get_machine_mapping(tm_id)
-
- dom = minidom.Document()
-
- mcfg = dom.createElement("machineconfig")
-
- info = dom.createElement("info")
- supported = ["hostname", "libvirt_domain", "rpcport"]
- for attr_name, attr_val in self._pool[pm_id]["info"].iteritems():
- if attr_name in supported:
- info.setAttribute(attr_name, str(attr_val))
- mcfg.appendChild(info)
-
- netdevices = dom.createElement("netdevices")
- mcfg.appendChild(netdevices)
-
- if_map = self._map["machines"][tm_id]["interfaces"]
- for t_if, p_if in if_map.iteritems():
- dev_info = self._pool[pm_id]["netdevices"][p_if]
-
- dev_node = dom.createElement("netdevice")
-
- dev_node.setAttribute("phys_id", t_if)
- dev_node.setAttribute("type", dev_info["type"])
- dev_node.setAttribute("hwaddr", dev_info["hwaddr"])
-
- for t_net, p_net in self._map["networks"].iteritems():
- if dev_info["network"] == p_net:
- dev_node.setAttribute("network", t_net)
- break
-
- netdevices.appendChild(dev_node)
-
- return mcfg.toxml()
-
-
-class SetupMapper:
- """
- This class can be used for matching machine setups against
- a pool of interconnected machines. SetupMapper will search
- through the pool for suitable matches of the requested
- setup and return the mapping between the two.
-
- Here we explain some terminology that is used consistently
- through the whole class:
-
- nc = neighbour connection; a 3-tuple that describes connection
- to an adjacent machine with the information which interface
- is used.
-
- (neighbour_id, network_id, iface_id)
-
- nc_list = list of neighbour connections; it is a building block
- of a topology
-
- topology = dictionary of nc_lists; it is an analogy to a adjacency
- list -- represenation of a graph. It is modified so it's
- able to represent non-graph structures such as our
- topology
-
- match = a correspondence between a machine, interface or a network
- from template and from a pool. It's a 2-tuple.
-
- (template_machine, pool_machine)
- (template_machines_iface, pool_machines_iface)
- (template_network, pool_network)
- """
-
- _machine_map = None
- _iface_map = None
- _network_map = None
-
- _template_machines = None
- _pool_machines = None
-
- @staticmethod
- def _get_topology(machine_configs):
- """
- This function will generate an adjacenty list from machine
- configuration dictionary. It can handle both machines and
- templates.
-
- :param machine_configs: dictionary of machines in the topology
- :type machines_configs: dict
-
- :return: Topology - neighbour connection list (adjacency list-like
- data structure
- :rtype: dict
- """
-
- networks = {}
- for m_id, m_config in machine_configs.iteritems():
- for dev_id, dev_info in m_config["netdevices"].iteritems():
- net = dev_info["network"]
- if not net in networks:
- networks[net] = []
- networks[net].append((m_id, dev_id))
-
- topology = {}
- for m_id, m_config in machine_configs.iteritems():
- topology[m_id] = []
- for net_name, net in networks.iteritems():
- devs_in_net = []
- for dev_id, dev_info in m_config["netdevices"].iteritems():
- if dev_info["network"] == net_name:
- devs_in_net.append(dev_id)
-
- for neighbour in net:
- n_m_id = neighbour[0]
- n_dev_id = neighbour[1]
- if n_m_id != m_id:
- for dev_in_net in devs_in_net:
- nc = (n_m_id, net_name, dev_in_net)
- if not nc in topology[m_id]:
- topology[m_id].append(nc)
-
- return topology
-
- @staticmethod
- def _is_match_valid(template_id, pool_id, matches):
- for match in matches:
- if (match[0] == template_id and match[1] != pool_id) or \
- (match[0] != template_id and match[1] == pool_id):
- return False
-
- return True
-
- def _is_machine_match_valid(self, template_id, pool_id):
- """
- Method for checking validity of a proposed match between
- two machines.
-
- :param template_id: machine id in template setup
- :type template_id: string
-
- :param pool_id: machine id in pool setup
- :type pool_id: string
-
- :return: True/False indicating the validity of this match
- :rtype: Bool
- """
-
- template_machine = self._template_machines[template_id]
- pool_machine = self._pool_machines[pool_id]
-
- # check machine properties
- properties = ["hostname", "libvirt_domain"]
- for prop_name, prop_value in template_machine["info"].iteritems():
- if prop_name in properties:
- if pool_machine["info"][prop_name] != prop_value:
- return False
-
- # check number of devices
- tm_ndevs = len(template_machine["netdevices"])
- pm_ndevs = len(pool_machine["netdevices"])
- if tm_ndevs > pm_ndevs:
- return False
-
- return self._is_match_valid(template_id, pool_id,
- self._machine_map)
-
- def _is_network_match_valid(self, template_id, pool_id):
- """
- Method for checking validity of a proposed match between
- two network names.
-
- :param template_id: network id in template setup
- :type template_id: string
-
- :param pool_id: network id in pool setup
- :type pool_id: string
-
- :return: True/False indicating the validity of this match
- :rtype: Bool
- """
-
- return self._is_match_valid(template_id, pool_id,
- self._network_map)
-
- def _is_if_match_valid(self, tm_id, t_if_id, pm_id, pm_if_id):
- """
- Check if matching of one interface on another is valid.
- This functions checks the parameters of those two interfaces,
- such as type, mac address etc.
-
- :param tm_id: template machine id
- :type tm_id: string
-
- :param tm_if_id: template machine's interface id
- :type tm_if_id: string
-
- :param pm_id: pool machine id
- :type tm_id: string
-
- :param pm_if_id: pool machine's interface id
- :type pm_if_id: string
-
- :return: True/False indicating the validity of this match
- :rtype: Bool
- """
-
- t_if = self._template_machines[tm_id]["netdevices"][t_if_id]
- p_if = self._pool_machines[pm_id]["netdevices"][pm_if_id]
-
- properties = ["type", "hwaddr"]
- for prop_name, prop_value in t_if.iteritems():
- if prop_name in properties:
- if p_if[prop_name] != prop_value:
- return False
-
- return True
-
- @staticmethod
- def _get_node_with_most_neighbours(topology):
- max_machine = None
- max_len = 0
- for machine, nc_list in topology.iteritems():
- if len(nc_list) > max_len:
- max_machine = machine
- max_len = len(nc_list)
-
- return max_machine
-
- def _get_possible_matches(self, machine, pool_topology):
- possible_matches = set(pool_topology.keys())
- impossible_matches = set()
-
- for match in self._machine_map:
- if match[0] == machine and not match[1] in impossible_matches:
- # in case the machine has already been matched,
- # return the corresponding match as an only option
- return set([match[1]])
- else:
- # in case the machine has been matched to a different
- # one in pool, remove it from possible matches
- impossible_matches.add(match[1])
-
- return possible_matches - impossible_matches
-
- def _get_nc_matches(self, tm_id, tm_nc_list, pm_id, pm_nc_list):
- """
- Return all possible ways of matching list of neighbours of a template
- machine on another list of neighbours of a pool machine. This function
- also keeps in mind what matches already exist and avoids conflicts.
-
- :param tm_nc_list: short for template machine neighbour connection list
- :type tm_nc_list: list
-
- :param pm_nc_list: short for pool machine neighbour connection list
- :type pm_nc_list: list
-
- :return: List of all possible mapping updates that are result of a
- successful matching between the machine's neighbour connections.
- :rtype: list
- """
-
- mmap = self._machine_map
- nmap = self._network_map
- mapping_update = []
-
- t_neigh, t_net, t_if = tm_nc_list[0]
-
- # recursion stop condition
- if len(tm_nc_list) == 1:
- for nc in pm_nc_list:
- p_neigh, p_net, p_if = nc
- if self._is_machine_match_valid(t_neigh, p_neigh) and \
- self._is_network_match_valid(t_net, p_net) and \
- self._is_if_match_valid(tm_id, t_if, pm_id, p_if):
- mapping = self._get_mapping_update(tm_nc_list[0], nc)
- mapping_update.append(mapping)
-
- return mapping_update
-
- for nc in pm_nc_list:
- p_neigh, p_net, p_if = nc
- if self._is_machine_match_valid(t_neigh, p_neigh) and \
- self._is_network_match_valid(t_net, p_net) and \
- self._is_if_match_valid(tm_id, t_if, pm_id, p_if):
- recently_added = self._get_mapping_update(tm_nc_list[0], nc)
- self._save_nc_match(recently_added)
-
- new_pm_nc_list = copy.deepcopy(pm_nc_list)
- new_pm_nc_list.remove(nc)
-
- possible_matches = self._get_nc_matches(tm_id, tm_nc_list[1:],
- pm_id, new_pm_nc_list)
- self._revert_nc_match(recently_added)
- for possible_match in possible_matches:
- mapping = (recently_added[0] + possible_match[0],
- recently_added[1] + possible_match[1],
- recently_added[2] + possible_match[2])
- mapping_update.append(mapping)
-
- return mapping_update
-
- def _get_mapping_update(self, template_nc, pool_nc):
- i = [(template_nc[2], pool_nc[2])]
-
- m = []
- m_match = (template_nc[0], pool_nc[0])
- if not m_match in self._machine_map:
- m.append(m_match)
-
- n = []
- n_match = (template_nc[1], pool_nc[1])
- if not n_match in self._network_map:
- n.append(n_match)
-
- return (i, m, n)
-
- def _save_nc_match(self, nc_match):
- self._machine_map |= set(nc_match[1])
- self._network_map |= set(nc_match[2])
-
- def _revert_nc_match(self, nc_match):
- self._machine_map -= set(nc_match[1])
- self._network_map -= set(nc_match[2])
-
- def _format_map_dict(self, machine_map, network_map):
- map_dict = {}
-
- map_dict["machines"] = {}
- for match in machine_map:
- if_map = {}
- for if_match in match[2]:
- if_map[if_match[0]] = if_match[1]
- map_dict["machines"][match[0]] = {"target": match[1],
- "interfaces": if_map}
-
- map_dict["networks"] = {}
- for match in network_map:
- map_dict["networks"][match[0]] = match[1]
-
- return map_dict
-
- def map_setup(self, template_machines, pool_machines):
- """
- Attempt to match template topology to pool topology.
-
- :param template_topology: dictionary of machine templates to be matched
- against the pool
- :type template_topology: dict
-
- :param pool_topology: dictionary o machine structures that will be used
- as a pool of available machines
- :type pool_topology: dict
-
- :return: 2-tuple (machine_map, network_map). Machine map is a list of
- 3-tuples (template_machine_id, pool_machine_id, iface_map),
- both iface_map and network_map are list of mappings between
- matched equivalents in template and pool.
- :rtype: tuple containing machine and network mappings
- """
-
- self._machine_map = set()
- self._iface_map = {}
- self._network_map = set()
-
- self._template_machines = template_machines
- self._pool_machines = pool_machines
-
- template_topology = self._get_topology(template_machines)
- pool_topology = self._get_topology(pool_machines)
-
- if self._map_setup(template_topology, pool_topology):
- machine_map = [(tm, pm, self._iface_map[tm]) \
- for tm, pm in self._machine_map]
- network_map = list(self._network_map)
- return self._format_map_dict(machine_map, network_map)
- else:
- return None
-
- def _map_setup(self, template_topology, pool_topology):
-
- if len(template_topology) <= 0:
- return True
-
- mmap = self._machine_map
- nmap = self._network_map
-
- # by choosing to match the biggest nodes in the topology first,
- # we optimize the amount of time it takes to find out that the
- # topology cannot be matched (in most cases)
- machine = self._get_node_with_most_neighbours(template_topology)
-
- possible_matches = self._get_possible_matches(machine, pool_topology)
- for possible_match in possible_matches:
- if not self._is_match_valid(machine, possible_match, mmap):
- continue
-
- mmap.add((machine, possible_match))
-
- template_nc_list = template_topology[machine]
- pool_nc_list = pool_topology[possible_match]
-
- nc_matches = self._get_nc_matches(machine, template_nc_list,
- possible_match, pool_nc_list)
- for nc_match in nc_matches:
- self._save_nc_match(nc_match)
- self._iface_map[machine] = nc_match[0]
-
- new_pool = copy.deepcopy(pool_topology)
- del new_pool[possible_match]
-
- new_template = copy.deepcopy(template_topology)
- del new_template[machine]
-
- if not self._map_setup(new_template, new_pool):
- self._revert_nc_match(nc_match)
- del self._iface_map[machine]
- continue
- else:
- return True
-
- mmap.discard((machine, possible_match))
-
- return False
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index df22a07..d915420 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -29,7 +29,7 @@ from lnst.Common.Utils import check_process_running
from lnst.Common.NetTestCommand import NetTestCommandContext, NetTestCommand
from lnst.Common.NetTestCommand import str_command
from lnst.Controller.NetTestParse import NetTestParse
-from lnst.Controller.MachinePool import MachinePool
+from lnst.Controller.SlavePool import SlavePool
class NetTestError(Exception):
pass
@@ -47,12 +47,9 @@ class NetTestController:
self._log_root_path = Logs.get_logging_root_path()
self._recipe_path = recipe_path
- if check_process_running("libvirtd"):
- self._machine_pool = MachinePool(config.get_option('environment',
- 'pool_dirs'), allow_virtual=True)
- else:
- self._machine_pool = MachinePool(config.get_option('environment',
- 'pool_dirs'), allow_virtual=False)
+ sp = SlavePool(config.get_option('environment', 'pool_dirs'),
+ check_process_running("libvirtd"))
+ self._slave_pool = sp
self._recipe = {}
definitions = {"recipe": self._recipe}
@@ -115,18 +112,20 @@ class NetTestController:
if len(provisioning["setup_requirements"]) <= 0:
return
- mp = self._machine_pool
- alloc_machines = mp.provision_setup(provisioning["setup_requirements"])
- if alloc_machines == None:
+ sp = self._slave_pool
+ machines = sp.provision_setup(provisioning["setup_requirements"])
+ if machines == None:
msg = "This setup cannot be provisioned with the current pool."
raise NetTestError(msg)
- provisioning["allocated_machines"] = alloc_machines
+ self._recipe["machines"] = machines
+ provisioning["map"] = {}
logging.info("Provisioning initialized")
- for machine in alloc_machines.keys():
- provisioner = mp.get_provisioner_id(machine)
- logging.info(" machine %s uses %s" % (machine, provisioner))
+ for m_id in machines.keys():
+ provisioner = sp.get_provisioner_id(m_id)
+ provisioning["map"][m_id] = provisioner
+ logging.info(" machine %s uses %s" % (m_id, provisioner))
def _prepare_device(self, machine_id, dev_id):
info = self._get_machineinfo(machine_id)
@@ -291,9 +290,9 @@ class NetTestController:
# Some additional initialization is necessary in case the
# underlying machine is provisioned from the pool
- prov_id = self._machine_pool.get_provisioner_id(machine_id)
+ prov_id = self._slave_pool.get_provisioner_id(machine_id)
if prov_id:
- provisioner = self._machine_pool.get_provisioner(machine_id)
+ provisioner = self._slave_pool.get_provisioner(machine_id)
logging.info("Initializing provisioned system (%s)" % prov_id)
for device in provisioner["netdevices"].itervalues():
self._rpc_call(machine_id, 'set_device_down', device["hwaddr"])
@@ -369,7 +368,7 @@ class NetTestController:
def _prepare(self):
# All the perparations are made within the recipe parsing
- # This is achieved by handling parser events (by registering
+ # This is achieved by handling parser events
try:
self._ntparse.parse_recipe()
except Exception as exc:
diff --git a/lnst/Controller/SlavePool.py b/lnst/Controller/SlavePool.py
new file mode 100644
index 0000000..15b0de2
--- /dev/null
+++ b/lnst/Controller/SlavePool.py
@@ -0,0 +1,539 @@
+"""
+This module contains implementaion of SlavePool class that
+can be used to maintain a cluster of test machines.
+
+These machines can be provisioned and used in test recipes.
+
+Copyright 2012 Red Hat, Inc.
+Licensed under the GNU General Public License, version 2 as
+published by the Free Software Foundation; see COPYING for details.
+"""
+
+__author__ = """
+rpazdera(a)redhat.com (Radek Pazdera)
+"""
+
+import logging
+import os
+import re
+import copy
+from xml.dom import minidom
+from lnst.Common.XmlProcessing import XmlDomTreeInit
+from lnst.Controller.NetTestParse import MachineConfigParse
+
+class SlavePool:
+ """
+ This class is responsible for managing test machines that
+ are available at the controler and can be used for testing.
+ """
+
+ _map = {}
+ _pool = {}
+
+ _machine_matches = []
+ _network_matches = []
+
+ _allow_virtual = False
+
+ def __init__(self, pool_dirs, allow_virtual=False):
+ self._allow_virtual = allow_virtual
+ for pool_dir in pool_dirs:
+ self.add_dir(pool_dir)
+
+ def add_dir(self, pool_dir):
+ dentries = os.listdir(pool_dir)
+
+ for dirent in dentries:
+ self.add_file("%s/%s" % (pool_dir, dirent))
+
+ def add_file(self, filepath):
+ if os.path.isfile(filepath) and re.search("\.xml$", filepath, re.I):
+ dom_init = XmlDomTreeInit()
+ dom = dom_init.parse_file(filepath)
+
+ dirname, basename = os.path.split(filepath)
+
+ parser = MachineConfigParse()
+ parser.set_include_root(dirname)
+ parser.disable_events()
+
+ machine = {"info": {}, "netdevices": {}}
+ machine_id = re.sub("\.xml$", "", basename, flags=re.I)
+ parser.set_machine(machine_id, machine)
+
+ machineconfig = dom.getElementsByTagName("machineconfig")[0]
+ machine["dom_node_ref"] = machineconfig
+
+ parser.parse(machineconfig)
+ if 'libvirt_domain' not in machine['info'] or self._allow_virtual:
+ self._pool[machine_id] = machine
+ else:
+ logging.warning("libvirtd not found- Machine Pool skipping "\
+ "machine %s" % machine_id)
+
+ def provision_setup(self, setup_requirements):
+ """
+ This method will try to map a dictionary of machines'
+ requirements to a pool of machines that is available to
+ this instance.
+
+ :param templates: Setup request (dict of required machines)
+ :type templates: dict
+
+ :return: XML machineconfigs of requested machines
+ :rtype: dict
+ """
+
+ mapper = SetupMapper()
+ self._map = mapper.map_setup(setup_requirements, self._pool)
+
+ if self._map == None:
+ return None
+
+ configs = {}
+ for m_id in self._map["machines"]:
+ configs[m_id] = self._get_mapped_machineconfig_xml(m_id)
+
+ return configs
+
+ def get_provisioner_id(self, m_id):
+ try:
+ return self._get_machine_mapping(m_id)
+ except KeyError:
+ return None
+
+ def get_provisioner(self, m_id):
+ try:
+ p_id = self._get_machine_mapping(m_id)
+ return self._pool[p_id]
+ except KeyError:
+ return None
+
+ def _get_machine_mapping(self, m_id):
+ return self._map["machines"][m_id]["target"]
+
+ def _get_interface_mapping(self, m_id, if_id):
+ return self._map["machines"][m_id]["interfaces"][if_id]
+
+ def _get_network_mapping(self, net_id):
+ return self._map["networks"][net_id]
+
+ def _get_mapped_machineconfig_xml(self, tm_id):
+ pm_id = self._get_machine_mapping(tm_id)
+
+ dom = minidom.Document()
+
+ mcfg = dom.createElement("machineconfig")
+
+ info = dom.createElement("info")
+ supported = ["hostname", "libvirt_domain", "rpcport"]
+ for attr_name, attr_val in self._pool[pm_id]["info"].iteritems():
+ if attr_name in supported:
+ info.setAttribute(attr_name, str(attr_val))
+ mcfg.appendChild(info)
+
+ netdevices = dom.createElement("netdevices")
+ mcfg.appendChild(netdevices)
+
+ if_map = self._map["machines"][tm_id]["interfaces"]
+ for t_if, p_if in if_map.iteritems():
+ dev_info = self._pool[pm_id]["netdevices"][p_if]
+
+ dev_node = dom.createElement("netdevice")
+
+ dev_node.setAttribute("phys_id", t_if)
+ dev_node.setAttribute("type", dev_info["type"])
+ dev_node.setAttribute("hwaddr", dev_info["hwaddr"])
+
+ for t_net, p_net in self._map["networks"].iteritems():
+ if dev_info["network"] == p_net:
+ dev_node.setAttribute("network", t_net)
+ break
+
+ netdevices.appendChild(dev_node)
+
+ return mcfg.toxml()
+
+
+class SetupMapper:
+ """
+ This class can be used for matching machine setups against
+ a pool of interconnected machines. SetupMapper will search
+ through the pool for suitable matches of the requested
+ setup and return the mapping between the two.
+
+ Here we explain some terminology that is used consistently
+ through the whole class:
+
+ nc = neighbour connection; a 3-tuple that describes connection
+ to an adjacent machine with the information which interface
+ is used.
+
+ (neighbour_id, network_id, iface_id)
+
+ nc_list = list of neighbour connections; it is a building block
+ of a topology
+
+ topology = dictionary of nc_lists; it is an analogy to a adjacency
+ list -- represenation of a graph. It is modified so it's
+ able to represent non-graph structures such as our
+ topology
+
+ match = a correspondence between a machine, interface or a network
+ from template and from a pool. It's a 2-tuple.
+
+ (template_machine, pool_machine)
+ (template_machines_iface, pool_machines_iface)
+ (template_network, pool_network)
+ """
+
+ _machine_map = None
+ _iface_map = None
+ _network_map = None
+
+ _template_machines = None
+ _pool_machines = None
+
+ @staticmethod
+ def _get_topology(machine_configs):
+ """
+ This function will generate an adjacenty list from machine
+ configuration dictionary. It can handle both machines and
+ templates.
+
+ :param machine_configs: dictionary of machines in the topology
+ :type machines_configs: dict
+
+ :return: Topology - neighbour connection list (adjacency list-like
+ data structure
+ :rtype: dict
+ """
+
+ networks = {}
+ for m_id, m_config in machine_configs.iteritems():
+ for dev_id, dev_info in m_config["netdevices"].iteritems():
+ net = dev_info["network"]
+ if not net in networks:
+ networks[net] = []
+ networks[net].append((m_id, dev_id))
+
+ topology = {}
+ for m_id, m_config in machine_configs.iteritems():
+ topology[m_id] = []
+ for net_name, net in networks.iteritems():
+ devs_in_net = []
+ for dev_id, dev_info in m_config["netdevices"].iteritems():
+ if dev_info["network"] == net_name:
+ devs_in_net.append(dev_id)
+
+ for neighbour in net:
+ n_m_id = neighbour[0]
+ n_dev_id = neighbour[1]
+ if n_m_id != m_id:
+ for dev_in_net in devs_in_net:
+ nc = (n_m_id, net_name, dev_in_net)
+ if not nc in topology[m_id]:
+ topology[m_id].append(nc)
+
+ return topology
+
+ @staticmethod
+ def _is_match_valid(template_id, pool_id, matches):
+ for match in matches:
+ if (match[0] == template_id and match[1] != pool_id) or \
+ (match[0] != template_id and match[1] == pool_id):
+ return False
+
+ return True
+
+ def _is_machine_match_valid(self, template_id, pool_id):
+ """
+ Method for checking validity of a proposed match between
+ two machines.
+
+ :param template_id: machine id in template setup
+ :type template_id: string
+
+ :param pool_id: machine id in pool setup
+ :type pool_id: string
+
+ :return: True/False indicating the validity of this match
+ :rtype: Bool
+ """
+
+ template_machine = self._template_machines[template_id]
+ pool_machine = self._pool_machines[pool_id]
+
+ # check machine properties
+ properties = ["hostname", "libvirt_domain"]
+ for prop_name, prop_value in template_machine["info"].iteritems():
+ if prop_name in properties:
+ if pool_machine["info"][prop_name] != prop_value:
+ return False
+
+ # check number of devices
+ tm_ndevs = len(template_machine["netdevices"])
+ pm_ndevs = len(pool_machine["netdevices"])
+ if tm_ndevs > pm_ndevs:
+ return False
+
+ return self._is_match_valid(template_id, pool_id,
+ self._machine_map)
+
+ def _is_network_match_valid(self, template_id, pool_id):
+ """
+ Method for checking validity of a proposed match between
+ two network names.
+
+ :param template_id: network id in template setup
+ :type template_id: string
+
+ :param pool_id: network id in pool setup
+ :type pool_id: string
+
+ :return: True/False indicating the validity of this match
+ :rtype: Bool
+ """
+
+ return self._is_match_valid(template_id, pool_id,
+ self._network_map)
+
+ def _is_if_match_valid(self, tm_id, t_if_id, pm_id, pm_if_id):
+ """
+ Check if matching of one interface on another is valid.
+ This functions checks the parameters of those two interfaces,
+ such as type, mac address etc.
+
+ :param tm_id: template machine id
+ :type tm_id: string
+
+ :param tm_if_id: template machine's interface id
+ :type tm_if_id: string
+
+ :param pm_id: pool machine id
+ :type tm_id: string
+
+ :param pm_if_id: pool machine's interface id
+ :type pm_if_id: string
+
+ :return: True/False indicating the validity of this match
+ :rtype: Bool
+ """
+
+ t_if = self._template_machines[tm_id]["netdevices"][t_if_id]
+ p_if = self._pool_machines[pm_id]["netdevices"][pm_if_id]
+
+ properties = ["type", "hwaddr"]
+ for prop_name, prop_value in t_if.iteritems():
+ if prop_name in properties:
+ if p_if[prop_name] != prop_value:
+ return False
+
+ return True
+
+ @staticmethod
+ def _get_node_with_most_neighbours(topology):
+ max_machine = None
+ max_len = 0
+ for machine, nc_list in topology.iteritems():
+ if len(nc_list) > max_len:
+ max_machine = machine
+ max_len = len(nc_list)
+
+ return max_machine
+
+ def _get_possible_matches(self, machine, pool_topology):
+ possible_matches = set(pool_topology.keys())
+ impossible_matches = set()
+
+ for match in self._machine_map:
+ if match[0] == machine and not match[1] in impossible_matches:
+ # in case the machine has already been matched,
+ # return the corresponding match as an only option
+ return set([match[1]])
+ else:
+ # in case the machine has been matched to a different
+ # one in pool, remove it from possible matches
+ impossible_matches.add(match[1])
+
+ return possible_matches - impossible_matches
+
+ def _get_nc_matches(self, tm_id, tm_nc_list, pm_id, pm_nc_list):
+ """
+ Return all possible ways of matching list of neighbours of a template
+ machine on another list of neighbours of a pool machine. This function
+ also keeps in mind what matches already exist and avoids conflicts.
+
+ :param tm_nc_list: short for template machine neighbour connection list
+ :type tm_nc_list: list
+
+ :param pm_nc_list: short for pool machine neighbour connection list
+ :type pm_nc_list: list
+
+ :return: List of all possible mapping updates that are result of a
+ successful matching between the machine's neighbour connections.
+ :rtype: list
+ """
+
+ mmap = self._machine_map
+ nmap = self._network_map
+ mapping_update = []
+
+ t_neigh, t_net, t_if = tm_nc_list[0]
+
+ # recursion stop condition
+ if len(tm_nc_list) == 1:
+ for nc in pm_nc_list:
+ p_neigh, p_net, p_if = nc
+ if self._is_machine_match_valid(t_neigh, p_neigh) and \
+ self._is_network_match_valid(t_net, p_net) and \
+ self._is_if_match_valid(tm_id, t_if, pm_id, p_if):
+ mapping = self._get_mapping_update(tm_nc_list[0], nc)
+ mapping_update.append(mapping)
+
+ return mapping_update
+
+ for nc in pm_nc_list:
+ p_neigh, p_net, p_if = nc
+ if self._is_machine_match_valid(t_neigh, p_neigh) and \
+ self._is_network_match_valid(t_net, p_net) and \
+ self._is_if_match_valid(tm_id, t_if, pm_id, p_if):
+ recently_added = self._get_mapping_update(tm_nc_list[0], nc)
+ self._save_nc_match(recently_added)
+
+ new_pm_nc_list = copy.deepcopy(pm_nc_list)
+ new_pm_nc_list.remove(nc)
+
+ possible_matches = self._get_nc_matches(tm_id, tm_nc_list[1:],
+ pm_id, new_pm_nc_list)
+ self._revert_nc_match(recently_added)
+ for possible_match in possible_matches:
+ mapping = (recently_added[0] + possible_match[0],
+ recently_added[1] + possible_match[1],
+ recently_added[2] + possible_match[2])
+ mapping_update.append(mapping)
+
+ return mapping_update
+
+ def _get_mapping_update(self, template_nc, pool_nc):
+ i = [(template_nc[2], pool_nc[2])]
+
+ m = []
+ m_match = (template_nc[0], pool_nc[0])
+ if not m_match in self._machine_map:
+ m.append(m_match)
+
+ n = []
+ n_match = (template_nc[1], pool_nc[1])
+ if not n_match in self._network_map:
+ n.append(n_match)
+
+ return (i, m, n)
+
+ def _save_nc_match(self, nc_match):
+ self._machine_map |= set(nc_match[1])
+ self._network_map |= set(nc_match[2])
+
+ def _revert_nc_match(self, nc_match):
+ self._machine_map -= set(nc_match[1])
+ self._network_map -= set(nc_match[2])
+
+ def _format_map_dict(self, machine_map, network_map):
+ map_dict = {}
+
+ map_dict["machines"] = {}
+ for match in machine_map:
+ if_map = {}
+ for if_match in match[2]:
+ if_map[if_match[0]] = if_match[1]
+ map_dict["machines"][match[0]] = {"target": match[1],
+ "interfaces": if_map}
+
+ map_dict["networks"] = {}
+ for match in network_map:
+ map_dict["networks"][match[0]] = match[1]
+
+ return map_dict
+
+ def map_setup(self, template_machines, pool_machines):
+ """
+ Attempt to match template topology to pool topology.
+
+ :param template_topology: dictionary of machine templates to be matched
+ against the pool
+ :type template_topology: dict
+
+ :param pool_topology: dictionary o machine structures that will be used
+ as a pool of available machines
+ :type pool_topology: dict
+
+ :return: 2-tuple (machine_map, network_map). Machine map is a list of
+ 3-tuples (template_machine_id, pool_machine_id, iface_map),
+ both iface_map and network_map are list of mappings between
+ matched equivalents in template and pool.
+ :rtype: tuple containing machine and network mappings
+ """
+
+ self._machine_map = set()
+ self._iface_map = {}
+ self._network_map = set()
+
+ self._template_machines = template_machines
+ self._pool_machines = pool_machines
+
+ template_topology = self._get_topology(template_machines)
+ pool_topology = self._get_topology(pool_machines)
+
+ if self._map_setup(template_topology, pool_topology):
+ machine_map = [(tm, pm, self._iface_map[tm]) \
+ for tm, pm in self._machine_map]
+ network_map = list(self._network_map)
+ return self._format_map_dict(machine_map, network_map)
+ else:
+ return None
+
+ def _map_setup(self, template_topology, pool_topology):
+
+ if len(template_topology) <= 0:
+ return True
+
+ mmap = self._machine_map
+ nmap = self._network_map
+
+ # by choosing to match the biggest nodes in the topology first,
+ # we optimize the amount of time it takes to find out that the
+ # topology cannot be matched (in most cases)
+ machine = self._get_node_with_most_neighbours(template_topology)
+
+ possible_matches = self._get_possible_matches(machine, pool_topology)
+ for possible_match in possible_matches:
+ if not self._is_match_valid(machine, possible_match, mmap):
+ continue
+
+ mmap.add((machine, possible_match))
+
+ template_nc_list = template_topology[machine]
+ pool_nc_list = pool_topology[possible_match]
+
+ nc_matches = self._get_nc_matches(machine, template_nc_list,
+ possible_match, pool_nc_list)
+ for nc_match in nc_matches:
+ self._save_nc_match(nc_match)
+ self._iface_map[machine] = nc_match[0]
+
+ new_pool = copy.deepcopy(pool_topology)
+ del new_pool[possible_match]
+
+ new_template = copy.deepcopy(template_topology)
+ del new_template[machine]
+
+ if not self._map_setup(new_template, new_pool):
+ self._revert_nc_match(nc_match)
+ del self._iface_map[machine]
+ continue
+ else:
+ return True
+
+ mmap.discard((machine, possible_match))
+
+ return False
--
1.7.7.6
11 years, 1 month
[lnst] ctl: Recent XML format changes
by Jiří Pírko
commit 1a27b014e5328e3028070cfb8f8b4f704de1c8dd
Author: Radek Pazdera <rpazdera(a)redhat.com>
Date: Thu Mar 7 17:33:40 2013 +0100
ctl: Recent XML format changes
This patch implements the XML format changes we discussed recently.
What used to be <machineconfig> is now called <slavemachine>:
<slavemachine>
<params>
<param name="hostname" value="192.168.122.10"/>
<param name="libvirt_domain" value="Fedora16"/>
<param name="os" value="fedora16"/>
<param name="type" value="virtual"/>
</params>
<netdevices>
<netdevice network="test-net" phys_id="1">
<params>
<param name="type" value="eth"/>
<param name="hwaddr" value="52:54:00:e4:75:16"/>
<param name="driver" value="virtio"/>
</params>
</netdevice>
</netdevices>
</slavemachine>
<machinerequires> tag in the recipe changed to <requirements>:
<requirements>
<params>
<param name="os" value="rhel6.2"/>
</params>
<netdevices>
<netdevice network="ttnet" phys_id="t1"/>
<netdevice network="ttnet" phys_id="t2"/>
</netdevices>
</requirements>
Both machines and netdevices now have variable tag called <params>
that can be used to specify different parameters and also to match
accordingly to these parameters.
WARNING:
Changes in this patch are not yet done. Although the code is functional,
these changes will require larger refactoring of the code, especially
in the NetTestController class.
We expect some changes to the controller from Ondra in the following
days, so I decided to postpone the refactoring and do it afterwards.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
lnst/Controller/NetTestController.py | 19 ++--
lnst/Controller/NetTestParse.py | 219 ++++++++++++++++-----------------
lnst/Controller/SlavePool.py | 60 +++------
3 files changed, 137 insertions(+), 161 deletions(-)
---
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index d915420..d686475 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -64,16 +64,13 @@ class NetTestController:
ntparse.set_recipe(self._recipe)
ntparse.set_definitions(definitions)
- ntparse.register_event_handler("netdevice_ready",
- self._prepare_device)
- ntparse.register_event_handler("machine_info_ready",
+ ntparse.register_event_handler("provisioning_requirements_ready",
+ self._prepare_provisioning)
+ ntparse.register_event_handler("machine_ready",
self._prepare_slave)
ntparse.register_event_handler("interface_config_ready",
self._prepare_interface)
- ntparse.register_event_handler("provisioning_requirements_ready",
- self._prepare_provisioning)
-
modules_dirs = config.get_option('environment', 'module_dirs')
tools_dirs = config.get_option('environment', 'tool_dirs')
@@ -85,9 +82,9 @@ class NetTestController:
def _get_machineinfo(self, machine_id):
try:
- info = self._recipe["machines"][machine_id]["info"]
+ info = self._recipe["machines"][machine_id]["params"]
except KeyError:
- msg = "Machine info is required, but not yet available"
+ msg = "Machine parameters requested, but not yet available"
raise NetTestError(msg)
return info
@@ -127,6 +124,8 @@ class NetTestController:
provisioning["map"][m_id] = provisioner
logging.info(" machine %s uses %s" % (m_id, provisioner))
+ machines[m_id]["params"]["system_config"] = {}
+
def _prepare_device(self, machine_id, dev_id):
info = self._get_machineinfo(machine_id)
dev = self._recipe["machines"][machine_id]["netdevices"][dev_id]
@@ -297,6 +296,10 @@ class NetTestController:
for device in provisioner["netdevices"].itervalues():
self._rpc_call(machine_id, 'set_device_down', device["hwaddr"])
+ machine = self._recipe["machines"][machine_id]
+ for dev_id in machine["netdevices"].iterkeys():
+ self._prepare_device(machine_id, dev_id)
+
def _init_slave_rpc(self, machine_id):
info = self._get_machineinfo(machine_id)
hostname = info["hostname"]
diff --git a/lnst/Controller/NetTestParse.py b/lnst/Controller/NetTestParse.py
index 28fe9ed..39d23bd 100644
--- a/lnst/Controller/NetTestParse.py
+++ b/lnst/Controller/NetTestParse.py
@@ -44,23 +44,6 @@ class NetTestParse(RecipeParser):
except Exception as exc:
raise XmlProcessingError(str(exc), xml_dom)
- # process machine requirements if used in the recipe before
- # proceeding to the second pass
- provisioning = self._recipe["provisioning"]
-
- dom_init = XmlDomTreeInit()
- for name, req in provisioning["setup_requirements"].iteritems():
- xml = provisioning["allocated_machines"][name]
- dom = dom_init.parse_string(xml, "pool_machine_config")
-
- # replace the original machinerequires with
- # the config recived from MachinePool
- original_node = req["dom_node_ref"]
- parent = original_node.parentNode
- replacement_node = dom.getElementsByTagName("machineconfig")[0]
-
- parent.replaceChild(replacement_node, original_node)
-
second_pass = SecondPass(self)
second_pass.parse(xml_dom)
@@ -74,9 +57,6 @@ class FirstPass(RecipeParser):
detect provisioning at the moment.
"""
- _has_machineconfigs = False
- _provisioned_setup = False
-
def parse(self, node):
self._recipe["provisioning"] = {}
self._recipe["provisioning"]["setup_requirements"] = {}
@@ -102,33 +82,18 @@ class FirstPass(RecipeParser):
params = {}
params["id"] = self._get_attribute(node, "id")
- scheme = {"machineconfig": self._machineconfig,
- "machinerequires": self._machinerequires}
+ scheme = {"requirements": self._requirements}
self._process_child_nodes(node, scheme, params,
default_handler=self._ignore_tag)
- def _machineconfig(self, node, params):
- if self._provisioned_setup:
- msg = "Cannot mix provisioned and non-provisioned machines"
- raise XmlProcessingError(msg, node)
-
- self._has_machineconfigs = True
-
- def _machinerequires(self, node, params):
- if self._has_machineconfigs:
- msg = "Cannot mix provisioned and non-provisioned machines"
- raise XmlProcessingError(msg, node)
-
- self._provisioned_setup = True
-
+ def _requirements(self, node, params):
machine_req = self._recipe["provisioning"]["setup_requirements"]
m_id = params["id"]
template = {}
- template["info"] = {}
template["netdevices"] = {}
machine_req[m_id] = template
- subparser = MachineRequiresParse(self)
+ subparser = RequirementsParse(self)
subparser.set_template(template)
subparser.parse(node)
@@ -158,7 +123,6 @@ class SecondPass(RecipeParser):
self._process_child_nodes(node, scheme)
def _machines(self, node, params):
- self._recipe["machines"] = {}
scheme = {"machine": self._machine}
self._process_child_nodes(node, scheme)
@@ -198,71 +162,91 @@ class MachineParse(RecipeParser):
def parse(self, node):
self._id = self._get_attribute(node, "id")
- self._machine = {}
- self._recipe[self._target][self._id] = self._machine
- self._machine["info"] = {}
- self._machine["netdevices"] = {}
+ recipe = self._recipe
+ self._machine = recipe["machines"][self._id]
self._machine["netconfig"] = {}
- scheme = {"machineconfig": self._machineconfig,
+ scheme = {"requirements": self._requirements,
"netconfig": self._netconfig }
self._process_child_nodes(node, scheme)
- def _machineconfig(self, node, params):
- subparser = MachineConfigParse(self)
- subparser.set_machine(self._id, self._machine)
- subparser.parse(node)
-
def _netconfig(self, node, params):
subparser = NetConfigParse(self)
subparser.set_machine(self._id, self._machine)
subparser.parse(node)
-class MachineRequiresParse(RecipeParser):
+ def _requirements(self, node, params):
+ try:
+ self._trigger_event("machine_ready", {"machine_id": self._id})
+ except Exception as exc:
+ raise XmlProcessingError(str(exc), node)
+
+class ParamsParse(RecipeParser):
+ _params = None
+
+ def set_params_dict(self, target):
+ self._params = target
+
+ def parse(self, node):
+ scheme = {"param": self._param}
+ self._process_child_nodes(node, scheme)
+
+ def _param(self, node, params):
+ name = self._get_attribute(node, "name")
+
+ if self._has_attribute(node, "value"):
+ value = self._get_attribute(node, "value")
+ else:
+ value = self._get_text_content(node)
+
+ self._params[name] = value
+
+class RequirementsParse(RecipeParser):
_requirements = None
def set_template(self, tmp_dict):
self._requirements = tmp_dict
def parse(self, node):
- self._requirements["dom_node_ref"] = node
+ self._requirements["params"] = {}
+ self._requirements["netdevices"] = {}
- scheme = {"info": self._info,
+ scheme = {"params": self._params,
"netdevices": self._netdevices}
- self._process_child_nodes(node, scheme)
-
- def _info(self, node, params):
- template = self._requirements
- info = template["info"]
-
- if self._has_attribute(node, "hostname"):
- info["hostname"] = self._get_attribute(node, "hostname")
+ params = {"target": self._requirements["params"]}
+ self._process_child_nodes(node, scheme, params)
- if self._has_attribute(node, "libvirt_domain"):
- info["libvirt_domain"] = self._get_attribute(node,
- "libvirt_domain")
+ def _params(self, node, params):
+ subparser = ParamsParse(self)
+ subparser.set_params_dict(params["target"])
+ subparser.parse(node)
def _netdevices(self, node, params):
scheme = {"netdevice": self._netdevice}
self._process_child_nodes(node, scheme)
def _netdevice(self, node, params):
- template = self._requirements
+ reqs = self._requirements
phys_id = self._get_attribute(node, "phys_id")
- dev = template["netdevices"][phys_id] = {}
+ dev = reqs["netdevices"][phys_id] = {}
dev["network"] = self._get_attribute(node, "network")
- if self._has_attribute(node, "type"):
+ dev["params"] = {}
+
+ scheme = {"params": self._params}
+ params = {"target": dev["params"]}
+ self._process_child_nodes(node, scheme)
+
+ if "type" in dev["params"]:
dev["type"] = self._get_attribute(node, "type")
- if self._has_attribute(node, "hwaddr"):
- hwaddr = self._get_attribute(node, "hwaddr")
- dev["hwaddr"] = normalize_hwaddr(hwaddr)
+ if "hwaddr" in dev["params"]:
+ dev["hwaddr"] = normalize_hwaddr(dev["params"]["hwaddr"])
-class MachineConfigParse(RecipeParser):
+class SlaveMachineParse(RecipeParser):
_machine_id = None
_machine = None
@@ -271,36 +255,22 @@ class MachineConfigParse(RecipeParser):
self._machine = machine
def parse(self, node):
- scheme = {"info": self._info,
+ scheme = {"params": self._params,
"netdevices": self._netdevices}
- self._process_child_nodes(node, scheme)
-
- def _info(self, node, params):
- machine = self._machine
- info = machine["info"]
-
- info["hostname"] = self._get_attribute(node, "hostname")
-
- if self._has_attribute(node, "libvirt_domain"):
- info["libvirt_domain"] = self._get_attribute(node,
- "libvirt_domain")
-
- if self._has_attribute(node, "rpcport"):
- info["rpcport"] = self._get_attribute(node, "rpcport", int)
-
- if self._has_attribute(node, "skip_cleanup"):
- info["skip_cleanup"] = self._get_attribute(node,
- "skip_cleanup", bool_it)
- else:
- info["skip_cleanup"] = False
+ params = {"target": self._machine["params"]}
+ self._process_child_nodes(node, scheme, params)
- info["system_config"] = {}
+ self._machine["params"]["skip_cleanup"] = False
+ mandatory_params = ["hostname"]
+ for mandatory in mandatory_params:
+ if mandatory not in self._machine["params"]:
+ msg = "Missing required parameter '%s'" % mandatory
+ raise XmlProcessingError(msg, node)
- try:
- self._trigger_event("machine_info_ready",
- {"machine_id": self._machine_id})
- except Exception as exc:
- raise XmlProcessingError(str(exc), node)
+ def _params(self, node, params):
+ subparser = ParamsParse(self)
+ subparser.set_params_dict(params["target"])
+ subparser.parse(node)
def _netdevices(self, node, params):
scheme = {"netdevice": self._netdevice,
@@ -321,23 +291,47 @@ class MachineConfigParse(RecipeParser):
dev = machine["netdevices"][phys_id] = {}
dev["create"] = params["create"]
- dev["type"] = self._get_attribute(node, "type")
dev["network"] = self._get_attribute(node, "network")
+ dev["params"] = {}
+
+ # parse device parameters
+ scheme = {"params": self._params}
+ params = {"target": dev["params"]}
+ self._process_child_nodes(node, scheme, params)
+
+ if "type" in dev["params"]:
+ dev["type"] = dev["params"]["type"]
+ else:
+ msg = "Missing required parameter 'type'"
+ raise XmlProcessingError(msg, node)
- # hwaddr attribute is optional for dynamic devices,
+ # hwaddr parameter is optional for dynamic devices,
# but it is required by non-dynamic devices
- if dev["create"] == None or self._has_attribute(node, "hwaddr"):
- hwaddr = self._get_attribute(node, "hwaddr")
- dev["hwaddr"] = normalize_hwaddr(hwaddr)
+ if dev["create"] and "hwaddr" in dev["params"]:
+ dev["hwaddr"] = normalize_hwaddr(dev["params"]["hwaddr"])
+ else:
+ if "hwaddr" in dev["params"]:
+ dev["hwaddr"] = normalize_hwaddr(dev["params"]["hwaddr"])
+ else:
+ msg = "Missing required parameter 'hwaddr'"
+ raise XmlProcessingError(msg, node)
- # name attribute is only valid when the device is not dynamic
- if dev["create"] == None and self._has_attribute(node, "name"):
- dev["name"] = self._get_attribute(node, "name")
+ # name parameter is only valid when the device is not dynamic
+ if "name" in dev["params"]:
+ if dev["create"]:
+ msg = "'name' parameter is not valid with dynamic devices"
+ raise XmlProcessingError(msg, node)
+ else:
+ dev["name"] = dev["params"]["name"]
- # bridge attribute is valid only when the device is dynamic
- if dev["create"] == "libvirt" and \
- self._has_attribute(node, "libvirt_bridge"):
- dev["libvirt_bridge"] = self._get_attribute(node, "libvirt_bridge")
+ # bridge parameter is valid only when the device is dynamic
+ if "libvirt_bridge" in dev["params"]:
+ if dev["create"] == "libvirt":
+ dev["libvirt_bridge"] = dev["params"]["libvirt_bridge"]
+ else:
+ msg = "'libvirt_bridge' parameter is not valid with" \
+ "dynamic devices"
+ raise XmlProcessingError(msg, node)
try:
self._trigger_event("netdevice_ready",
@@ -345,7 +339,6 @@ class MachineConfigParse(RecipeParser):
except Exception as exc:
raise XmlProcessingError(str(exc), node)
-
class NetConfigParse(RecipeParser):
_machine_id = None
_machine = None
diff --git a/lnst/Controller/SlavePool.py b/lnst/Controller/SlavePool.py
index 15b0de2..e430bbe 100644
--- a/lnst/Controller/SlavePool.py
+++ b/lnst/Controller/SlavePool.py
@@ -19,7 +19,7 @@ import re
import copy
from xml.dom import minidom
from lnst.Common.XmlProcessing import XmlDomTreeInit
-from lnst.Controller.NetTestParse import MachineConfigParse
+from lnst.Controller.NetTestParse import SlaveMachineParse
class SlavePool:
"""
@@ -53,19 +53,19 @@ class SlavePool:
dirname, basename = os.path.split(filepath)
- parser = MachineConfigParse()
+ parser = SlaveMachineParse()
parser.set_include_root(dirname)
parser.disable_events()
- machine = {"info": {}, "netdevices": {}}
+ machine = {"params": {}, "netdevices": {}}
machine_id = re.sub("\.xml$", "", basename, flags=re.I)
parser.set_machine(machine_id, machine)
- machineconfig = dom.getElementsByTagName("machineconfig")[0]
- machine["dom_node_ref"] = machineconfig
+ slavemachine = dom.getElementsByTagName("slavemachine")[0]
- parser.parse(machineconfig)
- if 'libvirt_domain' not in machine['info'] or self._allow_virtual:
+ parser.parse(slavemachine)
+ if 'libvirt_domain' not in machine['params'] \
+ or self._allow_virtual:
self._pool[machine_id] = machine
else:
logging.warning("libvirtd not found- Machine Pool skipping "\
@@ -87,12 +87,13 @@ class SlavePool:
mapper = SetupMapper()
self._map = mapper.map_setup(setup_requirements, self._pool)
+
if self._map == None:
return None
configs = {}
for m_id in self._map["machines"]:
- configs[m_id] = self._get_mapped_machineconfig_xml(m_id)
+ configs[m_id] = self._get_mapped_slave(m_id)
return configs
@@ -118,42 +119,23 @@ class SlavePool:
def _get_network_mapping(self, net_id):
return self._map["networks"][net_id]
- def _get_mapped_machineconfig_xml(self, tm_id):
+ def _get_mapped_slave(self, tm_id):
pm_id = self._get_machine_mapping(tm_id)
- dom = minidom.Document()
-
- mcfg = dom.createElement("machineconfig")
-
- info = dom.createElement("info")
- supported = ["hostname", "libvirt_domain", "rpcport"]
- for attr_name, attr_val in self._pool[pm_id]["info"].iteritems():
- if attr_name in supported:
- info.setAttribute(attr_name, str(attr_val))
- mcfg.appendChild(info)
-
- netdevices = dom.createElement("netdevices")
- mcfg.appendChild(netdevices)
+ machine = copy.deepcopy(self._pool[pm_id])
+ new_netdevices = {}
if_map = self._map["machines"][tm_id]["interfaces"]
for t_if, p_if in if_map.iteritems():
- dev_info = self._pool[pm_id]["netdevices"][p_if]
-
- dev_node = dom.createElement("netdevice")
-
- dev_node.setAttribute("phys_id", t_if)
- dev_node.setAttribute("type", dev_info["type"])
- dev_node.setAttribute("hwaddr", dev_info["hwaddr"])
+ new_netdevices[t_if] = machine["netdevices"][p_if]
for t_net, p_net in self._map["networks"].iteritems():
- if dev_info["network"] == p_net:
- dev_node.setAttribute("network", t_net)
+ if new_netdevices[t_if]["network"] == p_net:
+ new_netdevices[t_if]["network"] = t_net
break
- netdevices.appendChild(dev_node)
-
- return mcfg.toxml()
-
+ machine["netdevices"] = new_netdevices
+ return machine
class SetupMapper:
"""
@@ -265,11 +247,9 @@ class SetupMapper:
pool_machine = self._pool_machines[pool_id]
# check machine properties
- properties = ["hostname", "libvirt_domain"]
- for prop_name, prop_value in template_machine["info"].iteritems():
- if prop_name in properties:
- if pool_machine["info"][prop_name] != prop_value:
- return False
+ for prop_name, prop_value in template_machine["params"].iteritems():
+ if pool_machine["params"][prop_name] != prop_value:
+ return False
# check number of devices
tm_ndevs = len(template_machine["netdevices"])
11 years, 1 month
[lnst] ctl: MachinePool renamed to SlavePool
by Jiří Pírko
commit 0d52de661bd4c5396b83e69d00bdcac1098d6a65
Author: Radek Pazdera <rpazdera(a)redhat.com>
Date: Thu Mar 7 17:33:39 2013 +0100
ctl: MachinePool renamed to SlavePool
This commit renames the MachinePool class to slave pool.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
lnst/Controller/NetTestController.py | 33 ++++++++++-----------
lnst/Controller/{MachinePool.py => SlavePool.py} | 4 +-
2 files changed, 18 insertions(+), 19 deletions(-)
---
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index df22a07..d915420 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -29,7 +29,7 @@ from lnst.Common.Utils import check_process_running
from lnst.Common.NetTestCommand import NetTestCommandContext, NetTestCommand
from lnst.Common.NetTestCommand import str_command
from lnst.Controller.NetTestParse import NetTestParse
-from lnst.Controller.MachinePool import MachinePool
+from lnst.Controller.SlavePool import SlavePool
class NetTestError(Exception):
pass
@@ -47,12 +47,9 @@ class NetTestController:
self._log_root_path = Logs.get_logging_root_path()
self._recipe_path = recipe_path
- if check_process_running("libvirtd"):
- self._machine_pool = MachinePool(config.get_option('environment',
- 'pool_dirs'), allow_virtual=True)
- else:
- self._machine_pool = MachinePool(config.get_option('environment',
- 'pool_dirs'), allow_virtual=False)
+ sp = SlavePool(config.get_option('environment', 'pool_dirs'),
+ check_process_running("libvirtd"))
+ self._slave_pool = sp
self._recipe = {}
definitions = {"recipe": self._recipe}
@@ -115,18 +112,20 @@ class NetTestController:
if len(provisioning["setup_requirements"]) <= 0:
return
- mp = self._machine_pool
- alloc_machines = mp.provision_setup(provisioning["setup_requirements"])
- if alloc_machines == None:
+ sp = self._slave_pool
+ machines = sp.provision_setup(provisioning["setup_requirements"])
+ if machines == None:
msg = "This setup cannot be provisioned with the current pool."
raise NetTestError(msg)
- provisioning["allocated_machines"] = alloc_machines
+ self._recipe["machines"] = machines
+ provisioning["map"] = {}
logging.info("Provisioning initialized")
- for machine in alloc_machines.keys():
- provisioner = mp.get_provisioner_id(machine)
- logging.info(" machine %s uses %s" % (machine, provisioner))
+ for m_id in machines.keys():
+ provisioner = sp.get_provisioner_id(m_id)
+ provisioning["map"][m_id] = provisioner
+ logging.info(" machine %s uses %s" % (m_id, provisioner))
def _prepare_device(self, machine_id, dev_id):
info = self._get_machineinfo(machine_id)
@@ -291,9 +290,9 @@ class NetTestController:
# Some additional initialization is necessary in case the
# underlying machine is provisioned from the pool
- prov_id = self._machine_pool.get_provisioner_id(machine_id)
+ prov_id = self._slave_pool.get_provisioner_id(machine_id)
if prov_id:
- provisioner = self._machine_pool.get_provisioner(machine_id)
+ provisioner = self._slave_pool.get_provisioner(machine_id)
logging.info("Initializing provisioned system (%s)" % prov_id)
for device in provisioner["netdevices"].itervalues():
self._rpc_call(machine_id, 'set_device_down', device["hwaddr"])
@@ -369,7 +368,7 @@ class NetTestController:
def _prepare(self):
# All the perparations are made within the recipe parsing
- # This is achieved by handling parser events (by registering
+ # This is achieved by handling parser events
try:
self._ntparse.parse_recipe()
except Exception as exc:
diff --git a/lnst/Controller/MachinePool.py b/lnst/Controller/SlavePool.py
similarity index 99%
rename from lnst/Controller/MachinePool.py
rename to lnst/Controller/SlavePool.py
index 92ddf41..15b0de2 100644
--- a/lnst/Controller/MachinePool.py
+++ b/lnst/Controller/SlavePool.py
@@ -1,5 +1,5 @@
"""
-This module contains implementaion of MachinePool class that
+This module contains implementaion of SlavePool class that
can be used to maintain a cluster of test machines.
These machines can be provisioned and used in test recipes.
@@ -21,7 +21,7 @@ from xml.dom import minidom
from lnst.Common.XmlProcessing import XmlDomTreeInit
from lnst.Controller.NetTestParse import MachineConfigParse
-class MachinePool:
+class SlavePool:
"""
This class is responsible for managing test machines that
are available at the controler and can be used for testing.
11 years, 1 month