Thu, Nov 15, 2018 at 12:04:22PM CET, olichtne(a)redhat.com wrote:
On Thu, Nov 15, 2018 at 11:38:46AM +0100, Jan Tluka wrote:
> Thu, Nov 15, 2018 at 10:30:50AM CET, olichtne(a)redhat.com wrote:
> >On Thu, Nov 15, 2018 at 10:23:42AM +0100, Jan Tluka wrote:
> >> Wed, Nov 14, 2018 at 04:04:46PM CET, olichtne(a)redhat.com wrote:
> >> >+class Recipe(BaseRecipe):
> >> >+ def perf_test(self, recipe_conf):
> >> >+ results = RecipeResults(recipe_conf)
> >> >+
> >> >+ for i in range(recipe_conf.iterations):
> >> >+ run_results = []
> >> >+ for measurement in recipe_conf.measurements:
> >> >+ measurement.start()
> >> >+ for measurement in reversed(recipe_conf.measurements):
> >>
> >> I don't understand why it needs to be reversed here.
> >>
> >> If I start the measurements as: m1 m2 m3 m4
> >> I'd expect to finish them in the same order.
> >>
> >> Please explain.
> >
> >Start and finish isn't perfectly synchronized, specifying an order (m1 m2
m3)
> >makes the tester (at least in my case) feel that they'll start at the
> >same time or at worst in the specified order, in that case I'd expect
> >that m1 starts first and will include measurement for the full duration
> >of m2. And that m2 will include measurement for the full duration of m3
> >and so on. To do that you need to start them in the specified order and
> >then finish them in the reverse order.
> >
> >Does that answer the question?
> >
> >-Ondrej
> >
>
> Not really :-)
>
> So if we consider the "worst case":
>
> reversed finish call
> |
> m1.start() ----------| ----- m1.finish()
> m2.start() -------| -- m2.finish()
> m3.start() ----| m3.finish()
>
> So m1 lasts much longer than m3. Ideally we want them to last the same
> time, no?
Well, it actually depends on how you look at it, you can have multiple
types of measurements:
* measure from now, until I say so
* measure from now, for this long (e.g. for 60 seconds)
* probably others i haven't though of but these are the common ones that
we use
the second one still works the same as before:
m1.start(10seconds) -----> this will finish 10 seconds from it's start
m2.start(60seconds) -----> this will finish 60 seconds from it's start
so if m1 started at 00:00 and m2 started at 00:05 (m1 start takes a long
time for example)
then m1 finishes at 00:10-00:15 (depending on when the 10s actually
started counting)
and m2 finishes at 01:05
then we call:
m2.finish()
m1.finish()
and this just waits for each measurement to report that it's finished,
in a specific order yes, but the order doesn't influence when the
measurements finish. They finish based on the configuration of how long
to measure for. Finish just makes sure they don't take too long and
timeout (because of an implementation bug for example).
Ok, I finally understood how this works. I got confused because I
thought that each of the measurement is meant to be individual network stream
test, where I'd probably don't want to end the iperf started as the last
one to be finished as the first one (just to let each of them run the
same amount of time).
I looked at BaseENRTRecipe and I see that the first measurement (m1) is
a CPU measurement (that is not limited by timeout, runs indefinitely) and
the second (m2) collects data of the network performance tool.
So it makes sense to have m2 finished before m1.
Still it's somehow expected here that the first measurement is the
"driving" (and time unlimited) one. And that confused me.
>
>In the first case, the measurement measures "forever", and is stopped by
>an action of the controller - when the tester says to stop measuring.
>This means there's a different event that we wait for. And in that case
>it makes sense that when you specify:
>m1.start()
> m2.start()
>
>that the time period of m1 measurement, includes the time period m2 is
>running in. YES you ideally want to run everything for the same period
>of time, but since you really CAN'T synchronize that precisely, in my
>head it makes sense to have the intervals nicely organized:
>m1.start()
> m2.start()
> m2.finish()
>m1.finish()
>results in:
><---------------------------------------> #m1 measurement time interval
> <-------------------------------> #m2 measurement time interval
>
>whereas:
>m1.start()
> m2.start()
> m1.finish()
> m2.finish()
>results in:
><-------------------------> #m1 measurement time interval
> <-------------------------------> #m2 measurement time interval
>
>Also... the start time and the finish time don't have to be identical.
>And the start time and finish time of different measurements can also be
>different.
>
>And of course, our actual use case is a mix of the 2 types:
>m1 is a "forever" measurement sampling /proc/stat
>m2 is a time restricted measurement of a network flow (for 60 seconds)
>so what you want to end up with is, have a complete measurement of the
>network flow, and this dictates what time interval is relevant for any
>other measurement - every other measurement time interval should be a
>superset of the network flow measurement time interval.
>
>-Ondrej