[PATCH 00/19] Massive Controller Refactoring + cmd seq facelift
by Radek Pazdera
Hello everyone,
I finaly completed the refactoring of XML parsing at the controller
and finalized the patch set. I'll try to briefly summarize here what
has changed.
The goal of this series is to decouple parsing of the XMLs from processing
their contents. Up to now, the execution of lnst-ctl was controlled by the
parser, which proved to be very limiting and also quite convoluted.
This series introduces an intermediary data structure, a sort-of abstract
syntax tree, that is generated by the XML parser. The controller then
doesn't interact with the parser directly; it works with this new
structure instead.
Using this approach, I was able to move all the semantic checks to the
controller while keeping the structural checks in the parser. This will
allow us to use XML schemes for checking the structure in the future.
Apart from that, it makes the parser code much more readable (in my
opinion). Each node of the abstract syntax tree also contains references
back to the original XML file that are used for much more accurate
error reporting.
The series also contains a facelift of the former <command_sequence>
tag (now <task>), which is very closely related to the refactoring
process. The changes are the most prominent with commands. Each
command used to be represented by the <command> tag with a type
attribute. This was changed, so each command type now has it's own
tag instead.
For example:
<command type="test" value="IcmpPing" machine_id="tm">
...
</command>
changed to
<test module="IcmpPing" machine="tm">
...
</test>.
The commands are usualy shorter and easier to distinguish from each
another now.
To help with converting the existing recipes to the up-to-date format,
I modified the 'misc/recipe_conv.py' script, which can be used to
automatically conver all your recipes.
I tested everything I could with my current setup. Please let me know
if you run into problems :-).
Cheers,
-Radek
Radek Pazdera (19):
XmlProcessing: Split into two separate modules
XmlTemplates: Return XmlTemplateString
SlaveMachineParse: Decouple parsing and semantics
utils: Extend bool_it to support yes/no
slave: Support the new interface of Config cmd
Common: Adding XmlTemplateString support to parser
results: Support the new command sequence format
ctl: Bugfix for pass_result="false"
XmlTemplates: Template functions are now objects
ctl: Slightly changing the interface of SlavePool
ctl: Slight change of the interface of Interface
ctl: Massive refactoring of the RecipeParser
commands: Changing the internal structures
ctl: Massive refactoring of NetTestController
misc: Updating the recipe conversion script
recipes: Updating example recipes
recipes: Updating multicast recipes
recipes: Updating the team recipes
recipes: Updating smoke tests
lnst/Common/NetTestCommand.py | 93 ++--
lnst/Common/Utils.py | 4 +-
lnst/Common/XmlParser.py | 271 ++++++++++
lnst/Common/XmlProcessing.py | 399 +++++---------
lnst/Common/XmlTemplates.py | 295 ++++++-----
lnst/Controller/Machine.py | 11 +-
lnst/Controller/NetTestController.py | 400 +++++++++-----
lnst/Controller/NetTestResultSerializer.py | 32 +-
lnst/Controller/RecipeParse.py | 574 ++++++++-------------
lnst/Controller/SlaveMachineParse.py | 77 ++-
lnst/Controller/SlavePool.py | 95 +++-
lnst/Slave/NetTestSlave.py | 15 +-
misc/recipe_conv.py | 42 +-
recipes/examples/quick_guides/ping_check.xml | 8 +-
recipes/multicast/cmd_sequences/block_source.xml | 37 +-
recipes/multicast/cmd_sequences/if.xml | 35 +-
recipes/multicast/cmd_sequences/loop.xml | 35 +-
recipes/multicast/cmd_sequences/max_groups.xml | 14 +-
recipes/multicast/cmd_sequences/membership.xml | 25 +-
recipes/multicast/cmd_sequences/simple.xml | 18 +-
.../multicast/cmd_sequences/source_membership.xml | 39 +-
recipes/multicast/cmd_sequences/ttl.xml | 46 +-
recipes/multicast/multicast-all.xml | 32 +-
recipes/smoke/generate-recipes.py | 6 +-
recipes/smoke/lib/conf-bond.xml | 2 +-
recipes/smoke/lib/conf-eth.xml | 2 +-
recipes/smoke/lib/conf-team.xml | 4 +-
recipes/smoke/lib/conf-vlan.xml | 2 +-
recipes/smoke/lib/recipe-temp.xml | 2 +-
recipes/smoke/lib/seq-bg.xml | 13 -
recipes/smoke/lib/seq-exec.xml | 7 -
recipes/smoke/lib/seq-ping.xml | 10 -
recipes/smoke/lib/seq-sysconfig.xml | 10 -
recipes/smoke/lib/task-bg.xml | 12 +
recipes/smoke/lib/task-config.xml | 10 +
recipes/smoke/lib/task-exec.xml | 6 +
recipes/smoke/lib/task-ping.xml | 10 +
recipes/team/interfaces-bond_lacp.xml | 2 +-
recipes/team/interfaces-simple.xml | 2 +-
recipes/team/interfaces-simple_br.xml | 2 +-
recipes/team/interfaces-team_ab.xml | 2 +-
recipes/team/interfaces-team_ab_lw_001.xml | 2 +-
recipes/team/interfaces-team_ab_lw_002.xml | 2 +-
recipes/team/interfaces-team_ab_lw_003.xml | 2 +-
recipes/team/interfaces-team_ab_lw_004.xml | 2 +-
recipes/team/interfaces-team_ab_lw_005.xml | 2 +-
recipes/team/interfaces-team_ab_lw_006.xml | 2 +-
recipes/team/interfaces-team_ab_lw_007.xml | 2 +-
recipes/team/interfaces-team_ab_lw_008.xml | 2 +-
recipes/team/interfaces-team_bcast.xml | 2 +-
recipes/team/interfaces-team_lacp.xml | 2 +-
recipes/team/interfaces-team_lacp_act.xml | 2 +-
recipes/team/interfaces-team_lacp_passive.xml | 2 +-
recipes/team/interfaces-team_lacp_slowrate.xml | 2 +-
recipes/team/interfaces-team_lb.xml | 2 +-
recipes/team/interfaces-team_lb_act.xml | 8 +-
recipes/team/interfaces-team_rr.xml | 2 +-
recipes/team/interfaces-team_rr_queue_mapping.xml | 2 +-
recipes/team/machineconfig-dhcp-37-128.xml | 8 +-
recipes/team/machineconfig-peanut.xml | 8 +-
recipes/team/machineconfig-test1.xml | 6 +-
recipes/team/machineconfig-test2.xml | 6 +-
recipes/team/recipex_ab_001.xml | 4 +-
recipes/team/recipex_ab_002.xml | 18 +-
recipes/team/recipex_ab_003.xml | 4 +-
recipes/team/recipex_ab_004.xml | 4 +-
recipes/team/recipex_bcast_001.xml | 4 +-
recipes/team/recipex_bcast_002.xml | 18 +-
recipes/team/recipex_bcast_003.xml | 4 +-
recipes/team/recipex_bcast_004.xml | 4 +-
recipes/team/recipex_lacp_001.xml | 4 +-
recipes/team/recipex_lacp_002.xml | 18 +-
recipes/team/recipex_lacp_003.xml | 4 +-
recipes/team/recipex_lacp_004.xml | 4 +-
recipes/team/recipex_lacp_act_001.xml | 4 +-
recipes/team/recipex_lacp_act_002.xml | 18 +-
recipes/team/recipex_lacp_act_003.xml | 4 +-
recipes/team/recipex_lacp_act_004.xml | 4 +-
recipes/team/recipex_lb_001.xml | 4 +-
recipes/team/recipex_lb_002.xml | 18 +-
recipes/team/recipex_lb_003.xml | 4 +-
recipes/team/recipex_lb_004.xml | 4 +-
recipes/team/recipex_lb_act_001.xml | 4 +-
recipes/team/recipex_lb_act_002.xml | 18 +-
recipes/team/recipex_lb_act_003.xml | 4 +-
recipes/team/recipex_lb_act_004.xml | 4 +-
recipes/team/recipex_rr_001.xml | 4 +-
recipes/team/recipex_rr_002.xml | 18 +-
recipes/team/recipex_rr_003.xml | 4 +-
recipes/team/recipex_rr_004.xml | 4 +-
recipes/team/recipey_ab_lw_001.xml | 4 +-
recipes/team/recipey_ab_lw_002.xml | 4 +-
recipes/team/recipey_ab_lw_003.xml | 4 +-
recipes/team/recipey_ab_lw_004.xml | 4 +-
recipes/team/recipey_ab_lw_005.xml | 4 +-
recipes/team/recipey_ab_lw_006.xml | 4 +-
recipes/team/recipey_ab_lw_007.xml | 4 +-
recipes/team/recipey_ab_lw_008.xml | 4 +-
recipes/team/recipey_lacp_act_flows.xml | 4 +-
recipes/team/recipey_lacp_hashes.xml | 4 +-
recipes/team/recipey_lacp_passive.xml | 4 +-
recipes/team/recipey_lacp_slowrate.xml | 4 +-
recipes/team/recipey_lb_act_flows.xml | 4 +-
recipes/team/recipey_lb_hashes.xml | 4 +-
recipes/team/recipey_rr_queue_mapping.xml | 4 +-
recipes/team/sequence_iperf.xml | 26 +-
recipes/team/sequence_ping_simple.xml | 14 +-
recipes/team/sequence_pktgen.xml | 16 +-
recipes/team/sequence_pktgen_flows.xml | 14 +-
recipes/team/sequence_pktgen_hashes.xml | 10 +-
recipes/team/sequence_pktgen_lacp.xml | 16 +-
recipes/team/sequence_queue_mapping.xml | 38 +-
112 files changed, 1736 insertions(+), 1437 deletions(-)
create mode 100644 lnst/Common/XmlParser.py
delete mode 100644 recipes/smoke/lib/seq-bg.xml
delete mode 100644 recipes/smoke/lib/seq-exec.xml
delete mode 100644 recipes/smoke/lib/seq-ping.xml
delete mode 100644 recipes/smoke/lib/seq-sysconfig.xml
create mode 100644 recipes/smoke/lib/task-bg.xml
create mode 100644 recipes/smoke/lib/task-config.xml
create mode 100644 recipes/smoke/lib/task-exec.xml
create mode 100644 recipes/smoke/lib/task-ping.xml
--
1.8.3.1
10 years, 8 months
- 1
- 19
[lnst] slave: Fixing class init bug in NetConfig
by Jiří Pírko
commit a180a8682b23f1d54109420edccdf7e7027d6f05
Author: Radek Pazdera <rpazdera(a)redhat.com>
Date: Fri Aug 2 16:55:54 2013 +0200
slave: Fixing class init bug in NetConfig
Device type classes was not initialized properly which resulted in
problems with configuration (the required modules were not properly
loaded). This patch fixes the problem.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
lnst/Slave/NetConfig.py | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
---
diff --git a/lnst/Slave/NetConfig.py b/lnst/Slave/NetConfig.py
index d159321..cbb8487 100644
--- a/lnst/Slave/NetConfig.py
+++ b/lnst/Slave/NetConfig.py
@@ -60,6 +60,9 @@ class NetConfig:
return types
def add_interface_config(self, if_id, config):
+ dev_type = config["type"]
+ class_initialized = dev_type in self._get_used_types()
+
self._config[if_id] = config
self._devnames.rescan_netdevs()
@@ -68,8 +71,7 @@ class NetConfig:
dev_config = NetConfigDevice(config, self._config)
self._dev_configs[if_id] = dev_config
- dev_type = config["type"]
- if not dev_type in self._get_used_types():
+ if not class_initialized:
logging.info("Initializing '%s' device class", dev_type)
dev_config.type_init()
10 years, 8 months
- 1
- 0
[PATCH] slave: Fixing class init bug in NetConfig
by Radek Pazdera
Device type classes was not initialized properly which resulted in
problems with configuration (the required modules were not properly
loaded). This patch fixes the problem.
Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com>
---
lnst/Slave/NetConfig.py | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/lnst/Slave/NetConfig.py b/lnst/Slave/NetConfig.py
index d159321..cbb8487 100644
--- a/lnst/Slave/NetConfig.py
+++ b/lnst/Slave/NetConfig.py
@@ -60,6 +60,9 @@ class NetConfig:
return types
def add_interface_config(self, if_id, config):
+ dev_type = config["type"]
+ class_initialized = dev_type in self._get_used_types()
+
self._config[if_id] = config
self._devnames.rescan_netdevs()
@@ -68,8 +71,7 @@ class NetConfig:
dev_config = NetConfigDevice(config, self._config)
self._dev_configs[if_id] = dev_config
- dev_type = config["type"]
- if not dev_type in self._get_used_types():
+ if not class_initialized:
logging.info("Initializing '%s' device class", dev_type)
dev_config.type_init()
--
1.8.3.1
10 years, 8 months
- 1
- 0