I've finally read it as well :) Comments below.
At the moment, that task yaml file for rpmlint [1] looks like:
dependencies: - rpmlint - libtaskbot
input: args: envr,arch
These arguments are provided by the event trigger, right? So, the event trigger offers a set of arguments and we pick some of them inside "input" section; they can be then used in other sections. Is that right? ('arch' is not actually used anywhere in this demo file).
preparation: koji: download $envr
This is great, a preparation phase is exactly what I wanted.
execution: python: run_rpmlint.py $workdir
1) You don't actually use $workdir in run_rpmlint.py. I guess that's a bug? Or will we guarantee that CWD is set to $workdir?
2) Can we have multiple execution lines that are run sequentially (or in parallel?) and then consider the aggregate result as a test result? One example is in our outdated rats_install test: for ks in ['fedora','updates','updates-testing']: job.run_test('rats_install', tag=ks, ks=ks, **autoqa_args) We can of course define this as three separate tests, but sometimes running a few different variants together in a single test run might make sense.
post: shell: clean $workdir
How is this supposed to work? 'clean' is not an existing command, is that a keyword?
Will we support running arbitrary shell commands or custom scripts (e.g. './myscript') in this or other phases (like preparation)?
report: resultdb: something
I guess it's possible to run and report multiple results from a single test run? I.e. upgradepath checking 20 builds and reporting 20 separate test results.
Can I also report a single result from several execution lines?
I know this is just a demo and we will probably need to discover the answers for many of the questions above collectively. I was trying to think about different use cases a maybe discover some missing ones.