Excerpts from Nick Coghlan's message of 2013-01-02 11:53:20 +1000:
On 12/21/2012 12:52 PM, Raymond Mancy wrote:
* report results and upload logs
I don't think results should be a necessity. I wonder if the LC should only
require the API to expose a way of setting the status of a task. e.g.
Running/Completed.
Because the UI of beaker is pretty tightly coupled to the idea of a result,
perhaps we would have to provide some default which indicates no
particular result either way, e.g N/A.
People who would be comfortable using beaker in this way would just be using
it for it's inventory resources, scheduling algorithms, and provisioning capablities,
and would be interested in sending their test results and logs elewhere.
Of course I don't imagine this to be the majority of use cases, and people
would also be able to upload a result and logs as per normal, but I don't
think it should be necessary.
I'd think of this part of the API as "report progress" rather than
"report results". Even if they don't tell us whether or not the task
succeeded, they still need to tell us when it is done.
Okay, we could fully de-couple reporting progress (task X is completed)
from reporting results (task X result Y is pass). Actually now that
I look at it, they are already completely de-coupled -- task_stop is
separate from task_result. I missed that in my first draft.
I think right now if the harness stops a task without recording any
results then it will just become Completed/New, which might be
reasonable if the harness doesn't want to say whether the task passed or
failed. We could also introduce some new result distinct from New, such
as None or Unspecified, if we wanted to make it more explicit.
I would prefer N/A (not applicable)
For navigation purposes, we may want to provide a way for alternate
test harnesses/result servers to add an "external results URL" at the
job/recipe/task level (with the appropriate granularity being an open
question).
Yes I like this idea of allowing the harness to submit arbitrary URLs as
results. On the Beaker side we could probably just treat that the same
as a recipe-task log (or recipe-task-result log) but instead of being
stored by Beaker it is just a reference to a URL.
This is a great idea.