Taskotron Data Interfaces

[Tim Flink]

One of the design goals that I've talked about before is the idea of
using restful json at the interfaces between components of taskotron.
I'd like to simplify this a bit more to say that any interfaces between
components should be a variation on a dictionary.

When I say dictionary, I'm referring to a generic implementation that
could take one of several forms (json, yaml, python dictionaries are
the first that come to mind)

Some liberties will likely need to be taken with this concept when
interfacing with external systems, the important part is that any
single component from the trigger to the runner and reporting systems
should be isolate-able by using a simple mock object/system fed by
dictionaries.

There have been a couple of conversations that have been at least
partially related to data interfaces as of late and I thought that it
would be helpful to at least make sure we're all on the same page. I've
written up how I understand the interfaces below - please comment, ask
questions etc.

Tim


Trigger
=======

The trigger is a piece of code which is responsible for starting the
process of task execution. In our initial implementation, it will
listen for fedmsgs and based on the content of those messages, schedule
jobs with buildbot. The interface in this case is:

{
    'item': <name of item - envr, update id, koji tag, etc.>,
    'item_type': <koji_build, koji_tag, bodhi_update>,
    'taskname': <task to be scheduled>,
    'arch': <arch needed for execution, x86_64, i386 etc.>
}


Buildbot
========

When jobs are scheduled by the trigger using the above data, buildbot
will queue the task and delegate execution to an appropriate client
machine based on the input data. The git repo corresponding to the
taskname will be cloned on the client but buildbot will add some input
data. The resultant interface will be:

{
    'item': <name of item>,
    'item_type': <type of item>,
    'build_id': <uuid of buildbot build, used for logs and results>,
    'yamlfile': <taskname>.yml
}


Runner
======

The runner is responsible for taking the specified yaml file and
executing the directives contained within. In the case of CLI operation
where there is no build_id, a sensible default will be used instead to
indicate a filler value (-1000 for example).

When running a task, the runner will copy most of its own input as the
task's input:

{
    'item': <name of item>,
    'item_type': <type of item>,
    'build_id': <uuid of buildbot build, used for logs and results>
}

While the runner is responsible for running a task, each task is
responsible for its own reporting mechanism. The output from the runner
itself is in the form of execution information only and does not
necessarily parallel the result of the executed task.

{
    'execution_status': <OK or NOT_OK>,
    'status': <status message, possibly including any exeptions>
}


Task
====

I won't go too far into detail for this as it is being discussed in
phabricator:
 - https://phab.qadevel.cloud.fedoraproject.org/T84

I think that I've captured all that we've been talking about but I'm
sure that Josef or Kamil will correct me if I've missed something :)

The primary idea is that tasks will generate a list of TAP results
which can be farther processed into ResultsDB submissions or Bodhi
comments (for now). While TAP isn't scrictly a dictionary, it is a
reasonably well defined output format and when TAP13 is used with
embedded YAML, we end up with something that doesn't diverge too much
from a dictionary.

Example TAP output (stolen from D26):

ok - $CHECKNAME for Koji build xchat-0.5-1.fc20
  ---
  details:
    output: |-
      xchat.x86_64: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog
      xchat.x86_64: W: non-conffile-in-etc 
                       /etc/gconf/schem/apps_xchat_url_handler.schemas
      xchat.x86_64: W: no-manual-page-for-binary xchat
  item: xchat-0.5-1.fc20
  outcome: PASSED
  summary: 5 ERRORS, 10 WARNINGS
  type: koji_build
  ...
  not ok - $CHECKNAME for Bodhi update foobar-1.2-3.fc20  # FAIL 
  ---
  item: foobar-1.2-3.fc20
  outcome: FAILED
  summary: dependency error
  type: bodhi_update
  ...
  ok - $CHECKNAME for Yum repository f20-updates
  ---
  item: f20-updates
  outcome: INFO
  summary: 2 stale updates
  type: yum_repository
  ...

Example ResultsDB post (stolen from T84):

{
    "outcome": 'NEEDS_INSPECTION',
    "testcase_name": 'upgradepath',
    "summary": 'not ok upgradepath for build foo-1.2-3.fc20'
    "result_data": {
                     'item': 'foo-1.2-3.fc20',
                     'type': 'koji_build',
                     },
    "job_id": <resultsdb id>,
    "log_url": <path to log files>
}
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/qa-devel/attachments/20140318/ee33b404/attachment.sig>