Proposed must-have Features for Nitrate compared with Wiki

[James Laska]

On Fri, 2011-03-11 at 17:06 +0800, He Rui wrote:
> On Thu, 2011-03-10 at 10:19 -0500, James Laska wrote:
> > On Thu, 2011-03-10 at 18:01 +0800, He Rui wrote:
> > > On Wed, 2011-03-09 at 14:43 -0500, James Laska wrote:
> <snip>
> > > I meant in nitrate, there's only a 'Notes' field to support a simple
> > > summary of each test run. For a test day which has many contents[1] such
> > > as "What to test", "Prerequisite for Test Day", "How to test" etc, it is
> > > hard to put all of these into the 'Notes' field. This field doesn't
> > > support TinyMCE or any other syntax editing.   
> > > 
> > > [1] https://fedoraproject.org/wiki/QA/Test_Days/Template
> > 
> > Oh I see.  My initial thought is this might be more of a nice-to-have
> > feature, and possibly out of scope for the initial effort.  Either way,
> > I would say not a MUSTHAVE requirement.  To me, anything content related
> > should be left to the wiki ... the wiki does an outstanding job in this
> > regard.  Anything related to test cases and test results, should be
> > managed by nitrate (or something similar).
> 
> I see. Today I talked with TCMS members, they said they could support
> TinyMCE syntax editing to the 'Notes' area. Therefore we can either
> achieve this by a link to wiki or edit it in 'Notes' field. 

Okay

> <snip>
> > > Since it's internally use only, the permissions only have three: 1.
> > > anonymous - read. 2. tester - read and write any case/run/plan. 3. Admin
> > > - manage the tcms system. 
> 
> Correction. Testers have not only read and write any case/run/plan
> permission but also site/management administrations. Admin has
> Authentication permission besides what testers have.   
> 
> > Should we add this to the musthave requirements list.  Support for
> > anonymous read/write *and* FAS integration?
> 
> Sure, I've written them down.

Thanks

> > > > > 7. Result type -
> > > > > Nitrate doesn't have 'warn' but has 'error' result status. Modify it to
> > > > > 'warn'?
> > > > 
> > > > What is the significance of 'error' in nitrate?  Does that mean there
> > > > was a test case error, not necessarily a software error?
> > > 
> > > It presents an environment error, the user guide suggesting the use of
> > > result status is: 
> > > 
> > > Idle - Default value. The Test Case has not been examined.
> > > Passed - Test Case met all the expected results.
> > > Failed - Test Case did not meet all the expected results, or produced an
> > > unhandled exception.
> > > Running - Test Case is in progress.
> > > Paused - This status is used to denote a problem with the test case
> > > itself that prevents the test from being completed.
> > > Blocked - Test Case has a dependency that has failed.
> > > Error - Test environment has problems that prevent Test Case execution. 
> > > 
> > > But I guess such 'error' status will be seldom used.
> > 
> > Yeah, probably.  I can see the case for warn, but warn really just means
> > the test passed, but bugs were discovered during execution of the test
> > but did not impact the outcome of the test.
> 
> Yeah, then let's add another 'warn' status?

In my opinion, that can be a nice-to-have feature, but not a
requirement.  Unless I'm wrong, we use WARN to note that the test
passed, but other bugs were discovered while executing the test.  That
same information can be supplied in nitrate by marking the test PASS,
and linking to additional bugs filed.  Seems like we can convey the same
information without adding a new STATE.

> <snip>
> > Oh, how does nitrate designate the site admin and test run admins?  
> 
> So far there are two user groups: admin and tester. Both of them have
> the permission for site admin and test run admins. 

Okay, that works.  It sounds like admin and tester have the same 

> > Are
> > they single persons, or is it a user group (aka FAS group)?  I think
> > we'll want to ability to designate multiple (but few) site admins
> > (SHOULDHAVE), and need the ability to have multiple admins for test runs
> > (MUSTHAVE).  If it only supports one user being listed as the
> > administrator, we'll want to review what types of tasks that restricts
> > (add tests to the run, new runs etc...).
> 
> I was told that they are working on multiple site admins and multiple
> admins for different groups by trying to integrate with another system
> which is not open source. The TCMS members informed the FAS system are
> investigating it. 

Okay.

> <snip>
> > > > I know you mention anonymous login support above, can you also list FAS
> > > > authentication?  Apologies if I missed it already.
> > > 
> > > Do you mean integration with FAS? 
> > 
> > Yes please.  Apologies if that was already listed, I may have missed it.
> 
> It's listed. 

I see it now, thanks!

-James
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
Url : http://lists.fedoraproject.org/pipermail/test/attachments/20110314/c44d1b1e/attachment.bin