# AutoQA ResultsDB Discussion
# Date: 2010-03-26
# Time: 14:00 UTC (10:00 EST, 15:00 CET)
# Participants - wwoods, kparal, jskladan, jlaska
= Links =
* <
https://fedoraproject.org/wiki/AutoQA_resultsdb_schema>
* Scheme #1 - includes TCMS concept for notification of results
* Scheme #2 - schema without TCMS
* Scheme #3 - schema defining general Metadata table, not specific
tables for specific test classes
* <
https://fedoraproject.org/wiki/AutoQA_resultsdb_use_cases>
= Terminology =
* Test_program - The code, which performs the testing
* Test - Description/metadata (Name, Owner, Purpose, Tested
Package ...) ~ Table Test
* Testrun - Result of running the actual Test_program ~ Table testrun
* Test class - Package tests, Installation test, Repo test ...
= Agenda =
* Test
* What information do we need to store? (Name, Version,
Description ...)
* Identifying Test from Test_program
* While running Test_program it's vital to know, which Test does it
perform, so the result of Testrun could be correctly connected to the
Test
* What is the best way, to identify if from inside the Test_program -
is Test.name + Test.version sufficient?
-> Can the wiki be used as a "cheap" test case management system for
now (maps test_case -> test_plan)?
* Test classes
* What test classes do we have?
* What information do we need to store for each test class?
-> The different dashboard/views are responsible for listing the
key/value pairs it will include
* Storing specific values for different Test classes-
* What is the best way to do this?
* Specific table for each test class
* Generic table for key/value pairs
* Mediawiki-like tags (which IMHO equals generic key-value pair)
* How to give users a way, to extend test classes with minimal
(ideally none) db-admin interference? Do we want that?
* IMHO could be easily done via key-value pairs
* do we want to define required keys for each test class?
-> For complying with some test class the test should provide some
basic few common keys
-> Don't force requirements on people, if they don't comply, it will
just not show up in front-ends etc (but they can write their own
front-end/listener)
* Tagging
* What is the purpose of tagging
* How do we want to use the tags
* can jlaska give more detail on the discussion with Dave Lawrence?
* performance issues (milions of lines in table to be searched) -
caching?
* Testplans
* Hard-coded Test_program with multiple 'phases' (like RATS)
* How to represent testplans in database? (Idea1 vs Idea3)
* one Test for each 'phase' so we can easily report progress
* additional information in Test about "how many phases it has" so
we can report progress
-> Decided to use the wiki for an interim test plan system. The wiki
will provide information about the test cases in a test plan. Perhaps a
specific format needed?
* resultdb interface
= Ideas =
* Fedora Message Bus - It will be neccessary to communicate with
external subjects (sending notifications to Bodhi, etc). It can be even
used for communication with front-end.
* TCMS - We are reluctant to implement this stuff inside ResultDB, we
should leave it up to Nitrate/similar project. (right?)
* Number of phases in test case - It can be provided by wiki (referenced
by test's URL) and parsed from that, so it doesn't have to be stored in
ResultDB (and we get rid of the TCMS part from ResultDB schema)
= Open Questions =
* How long to store old results? Archive them after end-of-release?
* How can we access the test result details (aka log files, stderr
etc...)? Autotest doesn't yet provide an easy way to find the job id.
The job id is needed to locate the log directory for results
* Once message bus is setup, determine what notifications are needed
(for our test results views/dashboards, for other projects such as
Fedora Community, bodhi, koji etc...)
= Action items =
* [jskladan+kparal] Create updated DB schema according to the discussion
* [wwoods?] Study MediaWiki RPC mechanism
* [kparal] Define default (common) key-values for basic test classes
(installation tests, repo tests, package tests)
* [jskladan] Communicate with Autotest developers how to get Job ID in
autotest client
* [jlaska] Try to set up a qpid instance and send some test data through
it (from provider to listener)
(jkeating (Oxf13) is a good person to talk to about this!)UI
= Next Meeting =
* Check-in on mailing list next week
* Schedule follow-up as needed