Proposed must-have Features for Nitrate compared with Wiki

James Laska jlaska at redhat.com
Wed Mar 9 19:43:36 UTC 2011


On Wed, 2011-03-09 at 18:06 +0800, He Rui wrote:
> Greetings,
> 
> Now let's continue the discussion[1] about TCMS management. After the
> feature comparison between wiki and nitrate, the features which are
> supported in current wiki, but not in nitrate have been identified[2].
> Among these, I priorized some of them as Must-have features, which were
> summarized as below[3]:
> 
> 1. History rollback -
> User can view history versions and undo changes.

Good, I do use this more often than I realize.

> 2. History comparison -
> Different versions can compare with each other.
> 
> 3. Description part in test case -
> Currently there's no 'Description' part for test case format on Nitrate,
> should we add this part or use 'Setup' part instead?

No, I don't think 'setup' is the right place since we also use 'setup'
in our wiki test template.  Does nitrate support a summary or any other
field to describe the intent of the test in human readable terms?

> 4. Grouping cases (by media) -
> Need a way to separate the cases to different groups in one test run.
> How can we achieve this in Nitrate?

Probably different test runs of the same plan.  One run would be for the
DVD ISO, one for the netinst.iso etc...  Perhaps this will need further
exploration.

> 5. Documents in test result page(Run) - 
> There's only a little space called 'Note' for documentation on Nitrate
> and it doesn't support syntax. should we ask for more spaces with syntax
> supported for test runs like test days?

Can you explain in more detail?  Are you proposing we move the test day
wiki content into nitrate, so it's a one-stop-shop for test information
for that event?

> 6. Moving test results -
> Moving previous results from a test run to another. AFAIK, there's no
> way to copy some of one test run results to another.

Good catch.  I think with nitrate, you can selectively include test
cases when creating a new test run, right?  Meaning, we wouldn't need to
include *all* tests in the plan when creating a new test run?

This brings up another question ... permissions.  With the wiki, anyone
has permission to add or remove tests from a test run.  How will that be
handled in nitrate?

> 7. Result type -
> Nitrate doesn't have 'warn' but has 'error' result status. Modify it to
> 'warn'?

What is the significance of 'error' in nitrate?  Does that mean there
was a test case error, not necessarily a software error?

> 8. Multiple contributions for each case -
> Support multiple test results for one case run.

Agreed and good catch!  I think it's very important to allow multiple
testers to provide feedback on a single test given that the environment
under test (hardware) can vary for tests.

> 9. Authorities for pages -
> Pages on wiki with different namespaces can have different permissions,
> so hopefully pages on Nitrate can also have diff permissions.

I was confused by this a little earlier as well.  I don't understand the
concept of pages within nitrate.  Is this to replace the use of our wiki
test plan document and validation pages (e.g.
http://fedoraproject.org/wiki/QA:Installation_validation_testing)?

> 10. Supporting anonymous user read-write access -
> Anonymous have read-write access for specified test runs.

+1

> 11. Page protection -
> Protect certain plans/cases

Oh, I think I understand now.  Is the analogy here that a valid FAS
login would be required, like we currently have with our wiki test cases
and test plans?

> 12. License the content -
> License the contents the same with wiki

Or at least listed in the list of "GOOD" licenses -
https://fedoraproject.org/wiki/Licensing#Good_Licenses_3

> 13. Upstream project community -
> Monitoring it and actively discussing topics in nitrate-devel@, as well
> as file/share bugs/requirements in bugzilla

+1 

> 14. Test day page(run) creation - 
> In Nitrate each test run are built based on its test plan. Therefore,
> test day plan and its cases are needed before creating a test day run.
> Should we change our test day process(adding test plan and its cases) or
> should we change Nitrate to support creating test runs without test
> plan? 

Perhaps something we need to experiment/research with a pilot instance? 

> 15. Test cases priority -
> Nitrate now has P1, P2, P3..., modify them to Alpha, Beta, and Final?

Alpha/Beta/Final are appropriate for our two test plans; desktop and
installation.  I don't know if they apply for all test plans folks might
contribute.  But it seems like an easy future adjustment, so for our
immediate needs, P3=Alpha, P2=Beta, P1=Final etc...

In general, for all nitrate metadata fields (priority, status,
resolution etc...), whomever administers the tool in Fedora should have
control over re-defining the values to something appropriate for Fedora.
Perhaps that might be a more generic way to phrase the requirement?

> 16. Each case with different platforms -
> Better support two platform results(i386 and x86_64) submitting for each
> case run 

Sure, a test run can only be for a single architecture, right?  So we'll
have two test runs for a compose.

     1. RC1 - Installation (i386)
     2. RC1 - Installation (x86_64)

> Welcome to discuss the above features and provide feedback/suggestion.
> If there's any important feature missed, or this reminds you the
> advantages of other tcms tools etc, feel free to talk.

I know you mention anonymous login support above, can you also list FAS
authentication?  Apologies if I missed it already.

Thank you for continuing to move forward with this initiative.  I'll
keep thinking of important features and reply here if I have anything
worth sharing :)

Thanks,
James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20110309/efadfdbf/attachment-0001.bin 


More information about the test mailing list