On June 2, 2014, 4:58 p.m., Tomáš Bžatek wrote:
> After long time trying to understand relations between test classes it finally makes
sense to me. This set is simple and I have no objections against it.
>
> It fails on my machine however, I'm trying to figure out what's wrong at the
moment (likely a race condition somewhere in the provider).
Tomáš Bžatek wrote:
So yeah, failed tests usually mean that changes have been made in burst and
indication manager didn't find any difference as a new user has been created and
deleted right afterwards, all this before indication manager notices. Adding 1 sec. sleeps
at the right places makes the tests succeed.
Tomáš Bžatek wrote:
Also this is probably perfectly fine:
..expected stream: '50ac,50ad'
..got: '49ac,ad,ac,49ad'
I wouldn't count on specific indication order delivery as things are async from
the nature. Unless there's a settle timeout in the "test_round_50" case.
Ad "made in burst": I understand there can be a limitation in how
"fast" changes provider can still see. We can agree on a reasonable sleep time
T to use in normal test cases: e.g. between triggering two groups of opposite actions
like '50ac,50ad', we will always insert sleep(T). But I'd say we should keep
it explicit and even still maintain the "holy grail" test case where T=0.
(Thinking about it, once quantum computers become common, we might even want to pass test
with T *less* than zero ;))
Ad "49ac,ad,ac,49ad": lmi.test.ind.IndicationTestProbe uses the time *as
embedded in the indication* to sort the events (`self.ind` being what the handler gets as
first arg):
def __cmp__(self, other):
a = self.ind.exported_objects()[0]['IndicationTime'].value
b = other.ind.exported_objects()[0]['IndicationTime'].value
return cmp(a, b)
Of course, this happens before the events are "squashed". So if it you get
`49ac,ad,ac,49ad` instead of `50ac,50ad`, it's not because they arrived in this order,
but either because the time resolution is not sufficient or because the times were really
reported incorrectly.
Note that everything about the indications (and typically also the actions like `useradd
joe1234` we did to trigger them) is stored and available during the test. So if that
helps, it should be possible to modify tests so that they report precise times (or even
some kind of mammoth reports).
Another note: Current implementation of event triggering mechanism is using Popen and
separate call to useradd/userdel for each action. Idon't have numbers but apparently
it's not the fastest possible way to create users. For example on my f20 VM, it was
matter of few actions per second ... definitely not something that should be OK to
tolerate "under the radar" ... and yet I have seen '49c...' problem at
least once.
- Alois
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://reviewboard-openlmi.rhcloud.com/r/1759/#review2685
-----------------------------------------------------------
On May 21, 2014, 5:22 p.m., Alois Mahdal wrote:
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://reviewboard-openlmi.rhcloud.com/r/1759/
-----------------------------------------------------------
(Updated May 21, 2014, 5:22 p.m.)
Review request for OpenLMI Developers.
Repository: openlmi-providers
Description
-------
Basic set of tests with focus on this flow:
1. add set of subscriptions
2. add set of handlers
3. trigger sequence of "interesting" events
4. collect indications
5. and make assertions as needed
Currently the assertions are mostly that correct classes have been
delivered, reporting the events in correct order.
~~
This is followup to request #1741.
Since then:
* Rewrote old and added missing docstrings
* `TestIndicationEventStream` now inherits from
`IndicationStreamTestCase` instead using it as mixin
* fixed typos
* removed unnecessary CLASS_NAME and NEEDS_INDICATIONS
* made `handlers` option explicit instead of using
`default_driver_args`
* fixed `xtest*` typos
Diffs
-----
src/account/test/TestIndicationEventStream.py PRE-CREATION
Diff:
http://reviewboard-openlmi.rhcloud.com/r/1759/diff/
Testing
-------
Thanks,
Alois Mahdal