[Test-Announce] Call for reviewing TCMS use cases and comparison!

[He Rui]

Please ignore the last mail.

Hi James,

Wow, firstly, many thanks for your careful review of all the items.:)

On Wed, 2011-01-19 at 16:08 -0500, James Laska wrote:
> On Tue, 2011-01-18 at 19:36 +0800, He Rui wrote:
> > Greetings!
> > 
> > Wiki workflow/use cases are reorganized and grouped by general and main
> > test events(runs) use cases. The general cases cover the basic uses of
> > wiki, and the events cases covers the detailed certain steps for
> > organizing the events.
> > 
> > Please review it:
> > 
> > https://fedoraproject.org/wiki/Rhe/tcms_use_cases
> > 
> >     
> > Meanwhile, Wiki and Nitrate comparison is also listed and grouped by
> > above use cases. I've listed as many features as I can think out for
> > comparison. Please have a look and see if any features are missed. You
> > can review it by the use case you're familiar with if reviewing all of
> > them is tough, but be aware that features compared in former use cases
> > are not listed in later cases again to avoid overlaps:
> > 
> > https://fedoraproject.org/wiki/Rhe/tcms_Comparison
> > 
> >  
> > Ticket#152[1] is currently set up for tracking this event. Feel free to
> > add comments or discuss it here. 
> > 
> > Note: the feedback deadline is Jan 21, the end of this week. After that
> > date, I'll move forward to the next phase. 
> > 
> > Waiting for your feedback!
> 
> Woah, I like the fancy table layout used for comparing use cases between
> the wiki and nitrate!  Thank you for taking the time to identify our
> current test workflows and offer a comparison between nitrate and the
> wiki.  You've got a really comprehensive list of use cases that will
> certainly help us evaluate nitrate.
> 
> While looking through the comparison wiki, I've noted a few
> comments/questions below.  Hope this helps!
> 
> I'm unclear what the use case 'Creating A Common Page' is, can you
> elaborate?  Is that when we create a category page to group all test
> results for a milestone (e.g.
> https://fedoraproject.org/wiki/Category:Fedora_14_Pre-Alpha_Test_Results)?

No, it means creating a general new page on wiki, like a documentation
page etc, and test cases/plans/categories/results pages are particular
cases based on creating a normal page like this. It lists what a user
may do when creating a new page. 

> Under 'creating a common page', the row titled 'Submitting format'.
> Would that be better named as 'Supports data entry using a form'?

Ah great, I've changed it.
> 
> Under 'creating a common page', there is a row titled 'Subpage using'.
> I don't think I understand that use case, can you explain?

Okay, for example on wiki, Test_Days is under the path of QA like:
https://fedoraproject.org/wiki/QA/Test_Days

So I tried to compare if nitrate can also organize sub-page under its
main page. 

> Under section 'Creating A Test Case', there are some red cells for the
> wiki for (these are also under 'Creating a Test Plan').  What does the
> 'red' mean.  Does it mean that they are not possible, or that the
> solution provided is not optimal?

Oh the first two below are not optimal, the last one is not possible on
wiki. 

>       * Renaming a case - move to another page

I think moving to another page is not a good way to rename a case, there
will be two link addresses for this case.   

>       * Case draft status - manually add draft category/note

It's better to have a case status called draft, or proposed like nitrate
does.

>       * Case review status - approved in ticket or somewhere else

Nitrate will send an email including the proposed case id to reviewer
for approval. Then the reviewer can set 'confirmed', or 'need update'
status. For wiki, I remember we used to track the cases status in
tickets and after approved in tickets or reviewed in test@, the 'draft'
note is removed, but no sign indicates the case has been reviewed and
approved.

> Also under section 'Creating A Test Case', the row titled 'Test case
> re-use (write once, link anywhere)' talks about nitrate support for
> cloning tests.  Does this create a new copy of the original, or does it
> re-use the existing test?  If any changes are made to one, are the
> propagated to the other clone?

Nitrate creates new copies, so if any change is made to one, the other
clone won't change.

> 
> Under 'Creating A Test Result Page Template', the row titled 'Group
> cases (by media)', I imagine nitrate supports other methods of
> grouping/organizing tests in a test run?  Oh wait ... this is grouping,
> not sorting.  How would this be done with nitrate, would a test run be
> created for each media type?  Or would we sort tests of different media
> types differently?
>

AFAIK, except the above two methods, adding tags to tests and filtering
them by tags is another choice. But it requires testers to filter by
themselves. So.. I need think about them and compare these methods.

> Under 'Creating A Test Result Page', the row titled 'Redirecting
> links' ... is this when a test run has expired, or been replaced by a
> newer test run?  Like if we get RC2, we'll go and adjust RC1 so people
> know it's no longer active?  Not sure what to call this feature, retire
> a test run?
>

Yeah, it lets people know which run is current and which has retired.
how about calling it 'directing to current test run'?

> Under 'Posting Test Results', the row titled 'Opening current links' ...
> I'm trying to think of a more generic name for that feature.  Maybe
> 'Quick access to active test runs' instead?
>
Yes! That's what I meant!

> Under 'Generating A Test Summary', the row titled 'Result summary/report
> generation'.  Haha, you list that as *RED* ... very accurate! :)
> 
> Under 'Administering', the row titled 'License', is this the license for
> nitrate and mediawiki, or the license for the content in the cases,
> plans and results?
>

This is the license for nitrate and mediawiki, and I think the licensing
of nitrate has been identified/resolved, so I'll list it as Green.

> Under 'Release Validation Test Event', the row titled 'Roadmap for each
> run' ... can you explain, or link to, what is meant by this?

Oh I mean each test run on nitrate has a progress bar to show how much
percentage of the tests have been executed. I've renamed this feature as
'Progress bar for each run'.
 
> Thanks again!
> James

Many thanks for reviewing these! Just tell me if there's any more problem. :D


Cheers,
Hurry
-- 
Contacts

Hurry
FAS Name: Rhe 
Timezone: UTC+8
TEL: 86-010-62608141
IRC nick: rhe #fedora-qa #fedora-zh

-- 
Contacts

Hurry
FAS Name: Rhe 
Timezone: UTC+8
TEL: 86-010-62608141
IRC nick: rhe #fedora-qa #fedora-zh