Proposed must-have Features for Nitrate compared with Wiki

[He Rui]

On Mon, 2011-03-14 at 16:23 -0400, James Laska wrote:
> On Fri, 2011-03-11 at 17:06 +0800, He Rui wrote:
> > On Thu, 2011-03-10 at 10:19 -0500, James Laska wrote:
> > > On Thu, 2011-03-10 at 18:01 +0800, He Rui wrote:
> > > > On Wed, 2011-03-09 at 14:43 -0500, James Laska wrote:
> > <snip>
> > > > I meant in nitrate, there's only a 'Notes' field to support a simple
> > > > summary of each test run. For a test day which has many contents[1] such
> > > > as "What to test", "Prerequisite for Test Day", "How to test" etc, it is
> > > > hard to put all of these into the 'Notes' field. This field doesn't
> > > > support TinyMCE or any other syntax editing.   
> > > > 
> > > > [1] https://fedoraproject.org/wiki/QA/Test_Days/Template
> > > 
> > > Oh I see.  My initial thought is this might be more of a nice-to-have
> > > feature, and possibly out of scope for the initial effort.  Either way,
> > > I would say not a MUSTHAVE requirement.  To me, anything content related
> > > should be left to the wiki ... the wiki does an outstanding job in this
> > > regard.  Anything related to test cases and test results, should be
> > > managed by nitrate (or something similar).
> > 
> > I see. Today I talked with TCMS members, they said they could support
> > TinyMCE syntax editing to the 'Notes' area. Therefore we can either
> > achieve this by a link to wiki or edit it in 'Notes' field. 
> 
> Okay
> 
> > <snip>
> > > > Since it's internally use only, the permissions only have three: 1.
> > > > anonymous - read. 2. tester - read and write any case/run/plan. 3. Admin
> > > > - manage the tcms system. 
> > 
> > Correction. Testers have not only read and write any case/run/plan
> > permission but also site/management administrations. Admin has
> > Authentication permission besides what testers have.   
> > 
> > > Should we add this to the musthave requirements list.  Support for
> > > anonymous read/write *and* FAS integration?
> > 
> > Sure, I've written them down.
> 
> Thanks
> 
> > > > > > 7. Result type -
> > > > > > Nitrate doesn't have 'warn' but has 'error' result status. Modify it to
> > > > > > 'warn'?
> > > > > 
> > > > > What is the significance of 'error' in nitrate?  Does that mean there
> > > > > was a test case error, not necessarily a software error?
> > > > 
> > > > It presents an environment error, the user guide suggesting the use of
> > > > result status is: 
> > > > 
> > > > Idle - Default value. The Test Case has not been examined.
> > > > Passed - Test Case met all the expected results.
> > > > Failed - Test Case did not meet all the expected results, or produced an
> > > > unhandled exception.
> > > > Running - Test Case is in progress.
> > > > Paused - This status is used to denote a problem with the test case
> > > > itself that prevents the test from being completed.
> > > > Blocked - Test Case has a dependency that has failed.
> > > > Error - Test environment has problems that prevent Test Case execution. 
> > > > 
> > > > But I guess such 'error' status will be seldom used.
> > > 
> > > Yeah, probably.  I can see the case for warn, but warn really just means
> > > the test passed, but bugs were discovered during execution of the test
> > > but did not impact the outcome of the test.
> > 
> > Yeah, then let's add another 'warn' status?
> 
> In my opinion, that can be a nice-to-have feature, but not a
> requirement.  Unless I'm wrong, we use WARN to note that the test
> passed, but other bugs were discovered while executing the test.  That
> same information can be supplied in nitrate by marking the test PASS,
> and linking to additional bugs filed.  Seems like we can convey the same
> information without adding a new STATE.
> 
Okay, that makes sense. Will see if there's any problem when using it in
a pilot instance. 

> > <snip>
> > > Oh, how does nitrate designate the site admin and test run admins?  
> > 
> > So far there are two user groups: admin and tester. Both of them have
> > the permission for site admin and test run admins. 
> 
> Okay, that works.  It sounds like admin and tester have the same 
> 
> > > Are
> > > they single persons, or is it a user group (aka FAS group)?  I think
> > > we'll want to ability to designate multiple (but few) site admins
> > > (SHOULDHAVE), and need the ability to have multiple admins for test runs
> > > (MUSTHAVE).  If it only supports one user being listed as the
> > > administrator, we'll want to review what types of tasks that restricts
> > > (add tests to the run, new runs etc...).
> > 
> > I was told that they are working on multiple site admins and multiple
> > admins for different groups by trying to integrate with another system
> > which is not open source. The TCMS members informed the FAS system are
> > investigating it. 
> 
> Okay.
> 
> > <snip>
> > > > > I know you mention anonymous login support above, can you also list FAS
> > > > > authentication?  Apologies if I missed it already.
> > > > 
> > > > Do you mean integration with FAS? 
> > > 
> > > Yes please.  Apologies if that was already listed, I may have missed it.
> > 
> > It's listed. 
> 
> I see it now, thanks!

Based on the discussion, I've updated the requirement page[1] with
solution field added. When these requirements are filed to bugzilla, I
will add bug no. to each one for tracking. 

Thanks,
Hurry

[1] https://fedoraproject.org/wiki/Tcms_feature_requirements

-- 
Contacts

Hurry
FAS Name: Rhe 
Timezone: UTC+8
TEL: 86-010-62608141
IRC nick: rhe #fedora-qa #fedora-zh