RFC: Taskotron task description format

[Kamil Paral]

I've finally read it as well :) Comments below.

> At the moment, that task yaml file for rpmlint [1] looks like:
> 
>   dependencies:
>       - rpmlint
>       - libtaskbot
> 
>   input:
>       args: envr,arch

These arguments are provided by the event trigger, right? So, the event trigger offers a set of arguments and we pick some of them inside "input" section; they can be then used in other sections. Is that right? ('arch' is not actually used anywhere in this demo file).

> 
>   preparation:
>       koji: download $envr

This is great, a preparation phase is exactly what I wanted.

> 
>   execution:
>       python: run_rpmlint.py $workdir

1) You don't actually use $workdir in run_rpmlint.py. I guess that's a bug? Or will we guarantee that CWD is set to $workdir?

2) Can we have multiple execution lines that are run sequentially (or in parallel?) and then consider the aggregate result as a test result? One example is in our outdated rats_install test:
for ks in ['fedora','updates','updates-testing']:
    job.run_test('rats_install', tag=ks, ks=ks, **autoqa_args)
We can of course define this as three separate tests, but sometimes running a few different variants together in a single test run might make sense.

> 
>   post:
>       shell: clean $workdir

How is this supposed to work? 'clean' is not an existing command, is that a keyword?

Will we support running arbitrary shell commands or custom scripts (e.g. './myscript') in this or other phases (like preparation)?

> 
>   report:
>       resultdb: something

I guess it's possible to run and report multiple results from a single test run? I.e. upgradepath checking 20 builds and reporting 20 separate test results.

Can I also report a single result from several execution lines?


I know this is just a demo and we will probably need to discover the answers for many of the questions above collectively. I was trying to think about different use cases a maybe discover some missing ones.