Evaluating MozTrap as a TCMS for Fedora QA

[Amita Sharma]

On 08/14/2014 02:44 PM, Adam Williamson wrote:
> On 2014-08-14 09:42, Amita Sharma wrote:
>
>> you missed one last question :: If this decision is made, have we
>> thought about the migration/porting of existing wiki test case/plans
>> to tcms?
>
> Oops, indeed, sorry (missed it in the nesting).
>
> That's one of the slightly trickier areas. Moztrap is bound to its 
> 'one step, one expected response' model, where we have the %prep stage 
> and we have separate stages for 'steps' and 'results' so they can have 
> any kind of mapping, it doesn't have to be 1:1.
>
> My initial thought was that we dump the steps from %prep into the test 
> description wholesale (this is what upstream basically recommends - 
> they say any preliminary details about test setup and so on should be 
> in the descripton). For the step/result thing, what I figured was that 
> we should just put in whatever we have and leave the 'corresponding' 
> step or result empty for any imbalance (so if there are 5 steps and 3 
> expected results, you'd have 2 empty expected result sections, if 
> there are 2 steps and 6 expected results, you'd have 4 empty step 
> sections), then fix it up by hand - I don't think we can get any 
> cleverer with the conversion. Even where the quantity of steps and 
> results matches they won't necessarily correspond perfectly 1-to-1 in 
> the way moztrap expects.
>
> So basically I think this issue means we can't achieve a fully 
> automated conversion, we'd need a degree of manual intervention / 
> review for every test case. I don't think that's completely 
> unmanageable, though, I believe our current test case corpus is in the 
> hundreds not the thousands and a lot of them could do with review 
> anyway (and we can always do a staged conversion - for instance only 
> move validation testing to moztrap at first, then we'd only need to 
> convert the validation test cases). It's something we could blow 
> through as a hackfest (remote) type thing, if we chose.
>
> So far as the tech goes, it's fairly trivial - moztrap supports a very 
> lightly structured text format for imports, so you just have to write 
> something which yoinks test case pages from mediawiki and does a 
> fairly straightforward bit of text munging on them. Well, if we try to 
> retain formatting in some way we can make the text munging more 
> complex, it depends how much we care.
>
> http://moztrap.readthedocs.org/en/latest/userguide/ui/import.html#bulk-test-case-entry-formats 
> is the documentation for bulk entry; it gives the basic idea, there 
> are some details that aren't covered (like if we use markdown syntax 
> in the Gherkin-style import format, will it show up in the resulting 
> test case? - that kinda thing).
>
> We could of course contribute code to make moztrap not assume that all 
> test cases have one result per step, but that's probably more work 
> than just adjusting our test cases to that format.
I agree with the idea doing it with some automation + some manual 
intervention as a hackfest. So +1
I am not sure about contributing code for this :/ ( -1)

Thanks,
Ami