Some liberties will likely need to be taken with this concept when
interfacing with external systems, the important part is that any
single component from the trigger to the runner and reporting systems
should be isolate-able by using a simple mock object/system fed by
dictionaries.
This is a good goal.
Trigger
=======
The trigger is a piece of code which is responsible for starting the
process of task execution.
In this whole email you say 'task'. Do you mean 'check' in our usual
terminology, or something else? Especially when describing the data structures.
In our initial implementation, it will
listen for fedmsgs and based on the content of those messages, schedule
jobs with buildbot. The interface in this case is:
{
'item': <name of item - envr, update id, koji tag, etc.>,
'item_type': <koji_build, koji_tag, bodhi_update>,
'taskname': <task to be scheduled>,
'arch': <arch needed for execution, x86_64, i386 etc.>
}
So this is data sent from Trigger to Buildbot, right?
Buildbot
========
When jobs are scheduled by the trigger using the above data, buildbot
will queue the task and delegate execution to an appropriate client
machine based on the input data. The git repo corresponding to the
taskname will be cloned on the client
We will need to do some optimization here, otherwise the git servers won't be happy.
but buildbot will add some input
data. The resultant interface will be:
{
'item': <name of item>,
'item_type': <type of item>,
'build_id': <uuid of buildbot build, used for logs and results>,
'yamlfile': <taskname>.yml
}
The yaml file is in the git repo, so I assume all the git repos are going to be cloned on
the server as well, not just clients, right?
It makes sense, because the Trigger also needs to know what checks are available,
therefore it must be available on the server.
Runner
======
The runner is responsible for taking the specified yaml file and
executing the directives contained within. In the case of CLI operation
where there is no build_id, a sensible default will be used instead to
indicate a filler value (-1000 for example).
What is -1000? Maybe just use None? Or don't include the field at all.
When running a task, the runner will copy most of its own input as the
task's input:
{
'item': <name of item>,
'item_type': <type of item>,
'build_id': <uuid of buildbot build, used for logs and results>
}
While the runner is responsible for running a task, each task is
responsible for its own reporting mechanism. The output from the runner
itself is in the form of execution information only and does not
necessarily parallel the result of the executed task.
{
'execution_status': <OK or NOT_OK>,
Is this determined purely from the check exit code, or somehow else?
'status': <status message, possibly including any
exeptions>
What goes here?
}
Task
====
I won't go too far into detail for this as it is being discussed in
phabricator:
-
https://phab.qadevel.cloud.fedoraproject.org/T84
I think that I've captured all that we've been talking about but I'm
sure that Josef or Kamil will correct me if I've missed something :)
The primary idea is that tasks will generate a list of TAP results
which can be farther processed into ResultsDB submissions or Bodhi
comments (for now). While TAP isn't scrictly a dictionary, it is a
reasonably well defined output format and when TAP13 is used with
embedded YAML, we end up with something that doesn't diverge too much
from a dictionary.
Example TAP output (stolen from D26):
ok - $CHECKNAME for Koji build xchat-0.5-1.fc20
---
details:
output: |-
xchat.x86_64: W: file-not-utf8 /usr/share/doc/xchat/ChangeLog
xchat.x86_64: W: non-conffile-in-etc
/etc/gconf/schem/apps_xchat_url_handler.schemas
xchat.x86_64: W: no-manual-page-for-binary xchat
item: xchat-0.5-1.fc20
outcome: PASSED
summary: 5 ERRORS, 10 WARNINGS
type: koji_build
...
not ok - $CHECKNAME for Bodhi update foobar-1.2-3.fc20 # FAIL
Just a note, this line and the "ok" line below should be aligned with the very
first line.
---
item: foobar-1.2-3.fc20
outcome: FAILED
summary: dependency error
type: bodhi_update
...
ok - $CHECKNAME for Yum repository f20-updates
---
item: f20-updates
outcome: INFO
summary: 2 stale updates
type: yum_repository
...
We will need to add some additional lines here, e.g. log_url (computed from build_id).
I'd like to do that transparently in our library without the user needing to care
about anything.
Example ResultsDB post (stolen from T84):
{
"outcome": 'NEEDS_INSPECTION',
"testcase_name": 'upgradepath',
"summary": 'not ok upgradepath for build foo-1.2-3.fc20'
"result_data": {
'item': 'foo-1.2-3.fc20',
'type': 'koji_build',
},
"job_id": <resultsdb id>,
"log_url": <path to log files>
}
Thanks for the overview, Tim. I think the overall idea is good, and this is the way
forward.