Greetings,
Now I am trying to integrate mediakit_sanity test into AutoQA,
mediakit sanity test includes
tests of media size, checksum, repoclosure, conflicts. I find it is
trouble to deal with the
test results, since it is impossible to specify which test is FAILED,
we can only give a general
result to all tests self.detail.result=FAILED even some tests passed.
So do you think it it OK that I split the test into small tests?
apparently, there will be many
test in the tests directory. Or do you have any suggestion to solve
this? Thanks.
Hongqing
Hi,
you can report multiple results in a single test run. For example upgradepath test checks
all proposed updates and reports PASS/FAIL for every one of them individually. The code is
then a little more complicated. Basically you create a separate TestDetail instance for
every "item" you want to have reported (in your case one for media size, one for
checksum, ...):
http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/upgradepath/...
In the end you reports all those TestDetail results manually:
http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/upgradepath/...
But I wonder, is it all really necessary? You can use Summary Details section to provide
something like this:
Media size: PASS
Checksum: PASS
Repoclosure: PASS
Conflicts: FAIL
and then set the overall result to PASS only if all the four subsections are PASS. This
way if we encounter failed mediakit_sanity, it's easy to open the log and immediately
see which parts failed.
Kamil