[PATCH 0/2] ShortLivedConnections fixes

From: Perry Gagne

Found a few issues when enabling catalog support for Neper style
results.

Perry Gagne (2):
  NeperFlowMeasurement.py: Add request/response size to server params
  Neper.py: Fixed handling of stderr.

.../Perf/Measurements/NeperFlowMeasurement.py      | 14 ++++++++++++++
 lnst/Tests/Neper.py                                |  5 +++--
 2 files changed, 17 insertions(+), 2 deletions(-)

[PATCH 1/2] NeperFlowMeasurement.py: Add request/response size to server params

From: Perry Gagne

Turns out if neper doesn't have the response size definied on the server side, it
defaults to 1, instead of whatever the client param is.

Signed-off-by: Perry Gagne 
---
 .../Perf/Measurements/NeperFlowMeasurement.py      | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py b/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py
index 84a00c16..236c2c9b 100644
--- a/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py
+++ b/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py
@@ -20,11 +20,17 @@ class NeperFlowMeasurement(BaseFlowMeasurement):
         self._flows = flows
         self._running_measurements = []
         self._finished_measurements = []
+        self._host_versions = {}

@property
     def flows(self) -> List[Flow]:
         return self._flows

+    @property
+    def version(self):
+        return {"measurement_version": self._MEASUREMENT_VERSION,
+                "hosts_neper_versions": self._host_versions}
+
     def start(self):
         if len(self._running_measurements) > 0:
             raise MeasurementError("Measurement already running!")
@@ -78,6 +84,10 @@ class NeperFlowMeasurement(BaseFlowMeasurement):
         elif flow.cpupin is not None:
             raise RecipeError("Negative perf cpupin value provided.")

+        if flow.msg_size:
+            server_params["request_size"] = flow.msg_size
+            server_params["response_size"] = flow.msg_size
+
         return host.prepare_job(NeperServer(**server_params),
                                 job_level=ResultLevel.NORMAL)

@@ -123,10 +133,14 @@ class NeperFlowMeasurement(BaseFlowMeasurement):
             generator_stats = self._parse_job_samples(test_flow.client_job)
             flow_results.generator_results = generator_stats[0]
             flow_results.generator_cpu_stats = generator_stats[1]
+            self._host_versions[test_flow.flow.generator] = \
+                test_flow.client_job.result["data"]["VERSION"]

receiver_stats = self._parse_job_samples(test_flow.server_job)
             flow_results.receiver_results = receiver_stats[0]
             flow_results.receiver_cpu_stats = receiver_stats[1]
+            self._host_versions[test_flow.flow.receiver] = \
+                test_flow.server_job.result["data"]["VERSION"]

results.append(flow_results)

[PATCH 1/2] NeperFlowMeasurement.py: Add request/response size to server params

[Jan Tluka]

Mon, Mar 22, 2021 at 07:42:08PM CET, [e-mail] wrote:
> From: Perry Gagne
>
> Turns out if neper doesn't have the response size definied on the server side, it
> defaults to 1, instead of whatever the client param is.

Hello Perry.

There's additional code about versions. I'd prefer this to be in
separate commit or at least explained in the commit description.
For now I'm ok with that but please think about this in the future
commits.

Could you just explain what this change is about? What's the purpose of
this?

Thanks,
Jan
> Signed-off-by: Perry Gagne
>
> .../Perf/Measurements/NeperFlowMeasurement.py      | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py b/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py
> index 84a00c16..236c2c9b 100644
> --- a/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py
> +++ b/lnst/RecipeCommon/Perf/Measurements/NeperFlowMeasurement.py
> @@ -20,11 +20,17 @@ class NeperFlowMeasurement(BaseFlowMeasurement):
>         self._flows = flows
>         self._running_measurements = []
>         self._finished_measurements = []
>    self._host_versions = {}
>
> @property
>   def flows(self) -> List[Flow]:
>       return self._flows
>
> @property
>
> def version(self):
>    return {"measurement_version": self._MEASUREMENT_VERSION,
>            "hosts_neper_versions": self._host_versions}
>
> def start(self):
>       if len(self._running_measurements) > 0:
>           raise MeasurementError("Measurement already running!")
>
> @@ -78,6 +84,10 @@ class NeperFlowMeasurement(BaseFlowMeasurement):
>         elif flow.cpupin is not None:
>             raise RecipeError("Negative perf cpupin value provided.")
>    if flow.msg_size:
>        server_params["request_size"] = flow.msg_size
>        server_params["response_size"] = flow.msg_size
>   return host.prepare_job(NeperServer(**server_params),
>                           job_level=ResultLevel.NORMAL)
>
> @@ -123,10 +133,14 @@ class NeperFlowMeasurement(BaseFlowMeasurement):
>             generator_stats = self._parse_job_samples(test_flow.client_job)
>             flow_results.generator_results = generator_stats[0]
>             flow_results.generator_cpu_stats = generator_stats[1]
>        self._host_versions[test_flow.flow.generator] = \
>            test_flow.client_job.result["data"]["VERSION"]
>
>       receiver_stats = self._parse_job_samples(test_flow.server_job)
>       flow_results.receiver_results = receiver_stats[0]
>       flow_results.receiver_cpu_stats = receiver_stats[1]
>        self._host_versions[test_flow.flow.receiver] = \
>            test_flow.server_job.result["data"]["VERSION"]
>
>       results.append(flow_results)
>
> -- 
> 2.30.2
> _______________________________________________
> LNST-developers mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/[e-mail]...
> Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

[PATCH 1/2] NeperFlowMeasurement.py: Add request/response size to server params

[Ondrej Lichtner]

On Tue, Mar 23, 2021 at 08:54:41AM +0100, Jan Tluka wrote:
> Mon, Mar 22, 2021 at 07:42:08PM CET, [e-mail] wrote:
> > From: Perry Gagne
> >
> > Turns out if neper doesn't have the response size definied on the server side, it
> > defaults to 1, instead of whatever the client param is.
>
> Hello Perry.
>
> There's additional code about versions. I'd prefer this to be in
> separate commit or at least explained in the commit description.
> For now I'm ok with that but please think about this in the future
> commits.
>
> Could you just explain what this change is about? What's the purpose of
> this?
>
> Thanks,
> Jan

agreed, this should either be two commits, or the commit message should
talk about both changes.

-Ondrej

[PATCH 2/2] Neper.py: Fixed handling of stderr.

From: Perry Gagne

stderr in result dict wasn't getting populated when the job failed, due to it
returning before it could be set.

Signed-off-by: Perry Gagne 
---
 lnst/Tests/Neper.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lnst/Tests/Neper.py b/lnst/Tests/Neper.py
index 69d96adf..3c2b6445 100644
--- a/lnst/Tests/Neper.py
+++ b/lnst/Tests/Neper.py
@@ -64,6 +64,9 @@ class NeperBase(BaseTestModule):
                                  universal_newlines=True, shell=True,
                                  close_fds=True, cwd=NEPER_PATH)

+            self._res_data["stderr"] = res.stderr
+            self._res_data["data"] = self._parse_result(res)
+
             if res.stderr != "":
                 self._res_data["msg"] = f"errors reported by {self.params.workload}"
                 logging.error(self._res_data["msg"])
@@ -75,8 +78,6 @@ class NeperBase(BaseTestModule):
                 logging.error(self._res_data["msg"])
                 return False

-            self._res_data["data"] = self._parse_result(res)
-            self._res_data["stderr"] = res.stderr
             self._res_data["samples"] = [r for r in csv.DictReader(sf)]

return True