On Wed, Sep 6, 2017 at 2:07 PM, Frederic Lepied <flepied(a)redhat.com> wrote:
On Wed, Sep 6, 2017 at 7:12 AM, Kamil Paral <kparal(a)redhat.com>
wrote:
> On Wed, Sep 6, 2017 at 12:33 PM, Kamil Paral <kparal(a)redhat.com> wrote:
>
>> Third, SI says:
>>
>>> The playbook and its test suite or test framework:
>>> MUST place the main readable output of the test suite into a test.log
>>> file in the artifacts variable folder. This MUST happen even if some of the
>>> test suites fail.
>>>
>>
>> However, the default example doesn't contain any reference to *test.log*
>> and uses *output* file instead. Shouldn't those examples above redirect
>> to *test.log* instead?
>>
>
> It turns out I've been looking at old cached wiki content. After
> refreshing, I see that *test.log* is referenced instead.
>
>
>
>> Or is *test.log* not supposed to be an stdout+stderr output of the main
>> test process? I don't exactly understand the definition from SI, what *main
>> readable output* is supposed to be. When the test suite consists of X
>> individual tests, are they supposed to concatenate their output to
>> *test.log*?
>>
>
> These questions still stand, though.
>
>
You should see the test.log file as the main file to debug the run so it
should contain the information to debug issues. So it's up to the test
implementer to decide if it's stdout or stderr or both according to what he
needs to debug. Does it make sense?
Fred
It does, thanks. Will you include this clarification in the spec? The main
important information for me here is that the purpose of that file is to
*debug* the execution, and most often it'll be stdout+stderr. Because
"readable output", as it currently written, might be understood as human or
machine readable, i.e. a json with detailed results, which is something
completely different.