Excerpts from Nick Coghlan's message of 2013-01-02 11:53:20 +1000:
On 12/21/2012 12:52 PM, Raymond Mancy wrote:
- report results and upload logs
I don't think results should be a necessity. I wonder if the LC should only require the API to expose a way of setting the status of a task. e.g. Running/Completed.
Because the UI of beaker is pretty tightly coupled to the idea of a result, perhaps we would have to provide some default which indicates no particular result either way, e.g N/A. People who would be comfortable using beaker in this way would just be using it for it's inventory resources, scheduling algorithms, and provisioning capablities, and would be interested in sending their test results and logs elewhere.
Of course I don't imagine this to be the majority of use cases, and people would also be able to upload a result and logs as per normal, but I don't think it should be necessary.
I'd think of this part of the API as "report progress" rather than "report results". Even if they don't tell us whether or not the task succeeded, they still need to tell us when it is done.
Okay, we could fully de-couple reporting progress (task X is completed) from reporting results (task X result Y is pass). Actually now that I look at it, they are already completely de-coupled -- task_stop is separate from task_result. I missed that in my first draft.
I think right now if the harness stops a task without recording any results then it will just become Completed/New, which might be reasonable if the harness doesn't want to say whether the task passed or failed. We could also introduce some new result distinct from New, such as None or Unspecified, if we wanted to make it more explicit.
For navigation purposes, we may want to provide a way for alternate test harnesses/result servers to add an "external results URL" at the job/recipe/task level (with the appropriate granularity being an open question).
Yes I like this idea of allowing the harness to submit arbitrary URLs as results. On the Beaker side we could probably just treat that the same as a recipe-task log (or recipe-task-result log) but instead of being stored by Beaker it is just a reference to a URL.