tmt hint 02: Under The Hood

[Petr Šplíchal]

Hi!

Based on the feedback from users we've put together the second
chapter of the Test Management Tool guide: Let's together dive a
little bit more and look Under The Hood [0] of tmt to see how
plans, tests and stories work together on a couple of examples:

Plans [1] are used to enable testing and group relevant tests
together. They describe how to discover tests for execution, how
to provision the environment, how to prepare it for testing, how
to execute tests, report results and finally how to finish the
test job.

provision:
        how: container
        image: fedora:33
    prepare:
        how: install
        package: wget
    execute:
        how: tmt
        script: wget http://example.org/

Tests [2] define attributes which are closely related to
individual test cases such as the test script, framework,
directory path where the test should be executed, maximum test
duration or packages required to run the test.

summary: Fetch an example web page
    test: wget http://example.org/
    require: wget
    duration: 1m

Stories [3] can be used to track implementation, test and
documentation coverage for individual features or requirements.
Thanks to this you can track everything in one place, including
the project implementation progress.

story:
        As a user I want to see more detailed information for
        particular command.
    example:
      - tmt test show -v
      - tmt test show -vvv
      - tmt test show --verbose

Core [4] attributes cover general metadata such as summary or
description for describing the content, the enabled attribute for
disabling and enabling tests, plans and stories and the link key
which can be used for tracking relations between objects.

description:
        Different verbose levels can be enabled by using the
        option several times.
    link:
      - implemented-by: /tmt/cli.py
      - documented-by: /tmt/cli.py
      - verified-by: /tests/core/dry

Have a look at the whole chapter [0] to learn more details and get
some more context. See the Fedora Guide [6] to learn even more
about enabling tmt tests in the CI.

psss...

P.S. The tmt-1.7.0 [5] is available in Fedora repos and contains a
couple of new interesting features such as environment files,
parametrized plans, improved image guessing, experimental support
for reboot in the middle of the test plus a bunch of other
improvements and bugfixes.

[0] https://tmt.readthedocs.io/en/latest/guide.html#under-the-hood
[1] https://tmt.readthedocs.io/en/latest/guide.html#plans
[2] https://tmt.readthedocs.io/en/latest/guide.html#tests
[3] https://tmt.readthedocs.io/en/latest/guide.html#stories
[4] https://tmt.readthedocs.io/en/latest/guide.html#core
[5] https://github.com/psss/tmt/releases/tag/1.7.0
[6] https://docs.fedoraproject.org/en-US/ci/tmt/