Test Maps

[Mike Ruckman]

On Fri, 04 Apr 2014 17:53:53 -0700
Adam Williamson <awilliam at redhat.com> wrote:

> I do like the idea in general, but I'm not sure it's *quite* the most 
> logical order of attack. At least in my conception of The Problem,
> this is more of a second-order thing.
> 
> What I guess I'd say is The Fundamental Problem here is that we don't 
> have a great way of presenting our test cases and results. Our 
> 'wiki-as-a-TCMS' approach really involves representing *three* things
> as mediawiki elements, when you break it down:
> 
> i) test cases
> ii) test plans
> iii) test results
> 
> I've said in the past that I think the wiki-as-TCMS approach is 
> surprisingly good for being an obvious hack, but thinking about it
> more, I really only want to stand by that opinion in the case of i).
> The wiki only makes a barely-passable method of presenting test plans
> - especially the more complex they get, as ours have - and frankly a 
> pretty *bad* way of entering and representing test results. We're
> really reaching the point where we need to do at least ii) and iii)
> in a rather more sophisticated way.
> 
> we now have a couple of efforts that are coming at ii) and iii): Test 
> Maps and testcase_stats - 
> http://testdays.qa.fedoraproject.org/testcase_stats/ . They're both 
> pretty neat little hacks, I reckon, and they're both things we ought
> to have. But I think in an ordered conception of things, we really
> ought to be thinking about coming up with a more robust *framework*
> for test plans and test results, which would allow us to build things
> like test maps and testcase_stats as fairly thin 'presentation
> layers' on top of the robust underlying framework.
> 
> Do you (actual code-writey types) think I'm thinking along the right 
> lines there, or getting too platform-y or ambitious?

I didn't really go into the implementation - and I suppose I should
have. Test maps doesn't query the wiki every time you load it.
I wrote some crufty scraping code to pull testcase HTML one time,
and stuck it all in a database - I haven't hit the wiki for testcases
for a week or two (which is why there's noted broken links for images).
Currently the testmaps uses a Neo4J [0] graph database. Tim suggested
that I look into it - and it seems to fit pretty well with our needs. A
graph database is good at handling relationships between things, and
doing quick lookups based on those relationships. 

Mapping the relationships between requirements and testcases was
something I wanted to be able to do well, without the need for hefty sql
queries. My thought was with the UI to allow the data between test
cases and requirements to be built as we go. My pie-in-the-sky goal is
to completely move our tests and results tracking off the wiki.

So my goal here would be to address i, ii, and iii that Adam pointed
out. This is a longer term project and goal - but in the mean time we
could use this demo with the suggestions Kamil had. 

If we were to just use the UI now for a couple basic testmaps for now -
I would probably remove the "requirements" section from the UI since
that wouldn't be the goal for this demo effort. But eventually we'd
need to build out this data between hardware/software and test cases.

Sorry I wasn't clearer in the beginning as to what I was hoping to
eventually achieve. The UI is merely a whiteboard-y tip of the
iceberg :)

// Mike

[0] - http://www.neo4j.org/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/test/attachments/20140407/b68d74bd/attachment.sig>