[PATCH] NonzeroFlowEvaluator: add support for "metrics_to_evaluate"

From: Ondrej Lichtner <olichtne(a)redhat.com&gt;

Similar to the BaselineFlowEvaluator, this adds support for selecting
which metrics should be evaluated for the "non zero state". This can be
useful in case there is a known metrics/test that always reports a 0.

Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com&gt;
---
 .../Perf/Evaluators/NonzeroFlowEvaluator.py   | 34 +++++++++++++------
 1 file changed, 23 insertions(+), 11 deletions(-)

diff --git a/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
b/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
index 897d1e22..16384c55 100644
--- a/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
+++ b/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
@@ -10,6 +10,15 @@
 
 
 class NonzeroFlowEvaluator(BaseResultEvaluator):
+    def __init__(self, metrics_to_evaluate: List[str] = None):
+        if metrics_to_evaluate is not None:
+            self._metrics_to_evaluate = metrics_to_evaluate
+        else:
+            self._metrics_to_evaluate = [
+                "generator_results",
+                "receiver_results",
+            ]
+
     def evaluate_results(
         self,
         recipe: BaseRecipe,
@@ -22,16 +31,19 @@ def evaluate_results(
                 "Nonzero evaluation of flow:",
                 "{}".format(flow_results.flow),
             ]
-            if flow_results.generator_results.average > 0:
-                result_text.append("Generator reported non-zero throughput")
-            else:
-                result = False
-                result_text.append("Generator reported zero throughput")
-
-            if flow_results.receiver_results.average > 0:
-                result_text.append("Receiver reported non-zero throughput")
-            else:
-                result = False
-                result_text.append("Receiver reported zero throughput")
+            for metric_name in self._metrics_to_evaluate:
+                metric = getattr(flow_results, metric_name, None)
+                if metric:
+                    if metric.average > 0:
+                        result_text.append(
+                            "{} reported non-zero throughput".format(
+                                metric_name
+                            )
+                        )
+                    else:
+                        result = False
+                        result_text.append(
+                            "{} reported zero throughput".format(metric_name)
+                        )
 
             recipe.add_result(result, "\n".join(result_text))
-- 
2.31.1

[Ondrej Lichtner]

internally discussed as ok.

pushed.

-Ondrej

On Wed, Apr 28, 2021 at 04:50:43PM +0200, olichtne(a)redhat.com wrote:
>  From: Ondrej Lichtner <olichtne(a)redhat.com&gt;
>  
>  Similar to the BaselineFlowEvaluator, this adds support for selecting
>  which metrics should be evaluated for the "non zero state". This can be
>  useful in case there is a known metrics/test that always reports a 0.
>  
>  Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com&gt;
>  ---
>   .../Perf/Evaluators/NonzeroFlowEvaluator.py   | 34 +++++++++++++------
>   1 file changed, 23 insertions(+), 11 deletions(-)
>  
>  diff --git a/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
> b/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
>  index 897d1e22..16384c55 100644
>  --- a/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
>  +++ b/lnst/RecipeCommon/Perf/Evaluators/NonzeroFlowEvaluator.py
>  @@ -10,6 +10,15 @@
>   
>   
>   class NonzeroFlowEvaluator(BaseResultEvaluator):
>  +    def __init__(self, metrics_to_evaluate: List[str] = None):
>  +        if metrics_to_evaluate is not None:
>  +            self._metrics_to_evaluate = metrics_to_evaluate
>  +        else:
>  +            self._metrics_to_evaluate = [
>  +                "generator_results",
>  +                "receiver_results",
>  +            ]
>  +
>       def evaluate_results(
>           self,
>           recipe: BaseRecipe,
>  @@ -22,16 +31,19 @@ def evaluate_results(
>                   "Nonzero evaluation of flow:",
>                   "{}".format(flow_results.flow),
>               ]
>  -            if flow_results.generator_results.average > 0:
>  -                result_text.append("Generator reported non-zero throughput")
>  -            else:
>  -                result = False
>  -                result_text.append("Generator reported zero throughput")
>  -
>  -            if flow_results.receiver_results.average > 0:
>  -                result_text.append("Receiver reported non-zero throughput")
>  -            else:
>  -                result = False
>  -                result_text.append("Receiver reported zero throughput")
>  +            for metric_name in self._metrics_to_evaluate:
>  +                metric = getattr(flow_results, metric_name, None)
>  +                if metric:
>  +                    if metric.average > 0:
>  +                        result_text.append(
>  +                            "{} reported non-zero throughput".format(
>  +                                metric_name
>  +                            )
>  +                        )
>  +                    else:
>  +                        result = False
>  +                        result_text.append(
>  +                            "{} reported zero throughput".format(metric_name)
>  +                        )
>   
>               recipe.add_result(result, "\n".join(result_text))
>  -- 
>  2.31.1
>