As a developer I want to easily execute/enable tests

[Petr Šplíchal]

Hi!

I am writing today to share a proposal for how we can make it
easier to enable tests in Fedora CI.

We've heard the user story mentioned in the subject many times.
Fedora developers have frequently mentioned the difficulties which
they face with enabling tests in Fedora CI. Some people don't
understand the complex yaml syntax. Other people find that they
can't easily reproduce issues revealed by Fedora CI on their
laptops. People have asked us to provide a tool for local testing:

    https://pagure.io/fedora-ci/general/issue/4

A few of us have been working on a tool which we hope will improve
the situation considerably. The proof of concept below shows how
everyday developer tasks (related to testing) might look in the
near future:

Enable basic smoke test for my component:

    dnf install -y tmt
    tmt init --template mini
    vim plans/example.fmf
    git add . && git commit -m tests && git push

Create a new test based on a shell/beakerlib template:

    tmt test create --template shell tests/basic
    tmt test create --template beakerlib tests/advanced

Run all/selected tests safely in a virtual machine:

    tmt run
    tmt run test --name smoke

Execute tests in my preferred environment:

    tmt run --all provision --how container
    tmt run --all provision --how virtual
    tmt run --all provision --how local
    tmt run --all provision --how ...

I would like to ask for your feedback about this. What do you
think? Is this something that would be useful for you? What do
you think of the syntax? We would love to hear back from you
concerning what essential features you might expect from such a
tool. It would also be helpful to know if there is anything that
should be changed early in the tool design.

For more examples and frequently asked questions have a look here:

    https://tmt.readthedocs.io/en/latest/examples.html
    https://docs.fedoraproject.org/en-US/ci/tmt/

Looking forward to your feedback and ideas! :)

psss...