[release] Testing Farm 2023-08.1 with tmt-1.26.1

[Miroslav Vadkerti]

Testing Farm release 2023-08.1
https://issues.redhat.com/projects/TFT/versions/12410869 is deployed 🎉.
🎮 Upgrades

-

tmt https://tmt.readthedocs.io/en/stable/ updated from 1.24 to 1.26.1
   https://github.com/teemtee/tmt/releases/tag/1.26.1, see also release
   notes for 1.25 https://github.com/teemtee/tmt/releases/tag/1.25.0 and
   1.26 https://github.com/teemtee/tmt/releases/tag/1.26.0.
   -

Artemis https://gitlab.com/testing-farm/artemis upgraded from v0.0.57
   to v0.0.63 https://gitlab.com/testing-farm/artemis/-/releases/v0.0.63,
   see also release notes for v0.0.58
   https://gitlab.com/testing-farm/artemis/-/releases/v0.0.58, v0.0.59
   https://gitlab.com/testing-farm/artemis/-/releases/v0.0.59, v0.0.60
   https://gitlab.com/testing-farm/artemis/-/releases/v0.0.60, v0.0.61
   https://gitlab.com/testing-farm/artemis/-/releases/v0.0.61 and v0.0.62
   https://gitlab.com/testing-farm/artemis/-/releases/v0.0.62.

*🎯 Highlights*

-

Testing Farm reservations are officially available to all Testing Farm
   users. Reservations make it easy to reserve any machine with the same test
   environment setup as Testing Farm uses. We advise users to use this
   capability to debug their tests in the same test environment of our users
   like RHEL CI, Fedora CI and CentOS Stream CI. See documentation
   https://docs.testing-farm.io/general/0.1/cli.html#reserve for more
   details.
   -

Support for serial console log of the provisioned machines. See
   documentation
   https://docs.testing-farm.io/general/0.1/test-results.html#console-log
   for more datails. (TFT-892 https://issues.redhat.com/browse/TFT-892)
   -

Testing Farm now supports testing of VM images crated by Packit via
   ImageBuilder https://packit.dev/docs/cli/build/in-image-builder. (
   TFT-1980 https://issues.redhat.com/browse/TFT-1980)
   -

Support for tmt reportportal plugin. Users can now request reporting
   results to any reportportal instance. (testing-farm/worker!561
   https://gitlab.com/testing-farm/gluetool-modules/-/merge_requests/561)
   -

Support skipping of artifacts installation via API. This should greatly
   improve the stability of multiple sidatag builds in RHEL CI. (TFT-978
   https://issues.redhat.com/browse/TFT-978)
   -

Testing Farm worker is now based on Fedora, solving some long-term
   incompatibility issues in Fedora when rpm-build needs to be used to extract
   tests from rpm sources. (TFT-2121
   https://issues.redhat.com/browse/TFT-2121)
   -

Support for RHEL-8.10.0 and RHEL-9.4.0. (TFT-2124
   https://issues.redhat.com/browse/TFT-2124)
   -

Per user reliability and execution time dashboards are now possible for
   our users. (TFT-2138 https://issues.redhat.com/browse/TFT-2138)
   -

Testing Farm will be doing asynchronous updates of tmt to mitigate
   incompatibility issues.
   -

Support for all provision.fmf from STI use cases are now covered. STI
   migration guide in tmt documentation covers the migration
   https://tmt.readthedocs.io/en/latest/questions.html#migrating-provision-fmf.
   (TFT-767 https://issues.redhat.com/browse/TFT-767)
   -

Testing Farm will now refuse invalid composes in the API. Greatly
   reducing the risk of users passing invalid composes. (TFT-1723
   https://issues.redhat.com/browse/TFT-1723)
   -

Test result notes are now correctly displayed in our results viewer. (
   TFT-1883 https://issues.redhat.com/browse/TFT-1883)
   -

A new xunit_url field was added in the request details. We will be
   dropping the xunit field in the next releases, causing slow load times
   in our results viewer. (TFT-1907
   https://issues.redhat.com/browse/TFT-1907)
   -

i686 packages of tested artifacts are now available for the tests to
   consume. (TFT-2001 https://issues.redhat.com/browse/TFT-2001)
   -

The API now supports test.tmt as a shortcut to test.fmf for the tmt
   tests specification. (TFT-2032
   https://issues.redhat.com/browse/TFT-2032)
   -

Hardware requirements seen by Artemis are now shown in the pipeline.log.
   (TFT-2079 https://issues.redhat.com/browse/TFT-2079)
   -

Large plans are now collapsed in Oculus results viewer, improving load
   time for huge plans. (TFT-2140
   https://issues.redhat.com/browse/TFT-2140)

*🐞 *Bugfixes

-

Beaker provisioner now provisions Server variant for RHEL6 and RHEL7. (
   TFT-2086 https://issues.redhat.com/browse/TFT-2086)
   -

In case a repository provides multiple packages, only the latest package
   is installed. (TFT-2102 https://issues.redhat.com/browse/TFT-2102)
   -

The repoquery command used during artifact installation is now run on
   Testing Farm worker instead in the testing environment making the
   installation more stable. (TFT-2109
   https://issues.redhat.com/browse/TFT-2109)
   -

Correct provisioning with multiple network cards for ppc64le and s390x. (
   TFT-2199 https://issues.redhat.com/browse/TFT-2199)
   -

Installing of brew build on Rocky Linux and Alma Linux was fixed. (
   TFT-2231 https://issues.redhat.com/browse/TFT-2231)
   -

The tag repository
   https://docs.testing-farm.io/general/0.1/test-environment.html#_tag_repository
   for RHEL now includes modularity content. (TFT-2230
   https://issues.redhat.com/browse/TFT-2230)
   -

Support for UEFI in AWS for RHEL7 and later. Users can now get UEFI
   booted machines outside of Beaker \o/. (TFT-741
   https://issues.redhat.com/browse/TFT-741)
   -

Tag repository is now correctly added for CentOS Stream builds. (TFT-2139
   https://issues.redhat.com/browse/TFT-2139)
   -

Empty plans filtered out by plan-filter or test-filter are now skipped
   from scheduling. (TFT-1673 https://issues.redhat.com/browse/TFT-1673)
   -

Fix kickstart options can be now passed via the Testing Farm API. (
   TFT-2099 https://issues.redhat.com/browse/TFT-2099)

📦️ <#user-content-packages> Packages

List of important packages bundled in the worker image.

❯ podman run --entrypoint rpm quay.io/testing-farm/worker:2023-08.1 -q
tmt standard-test-roles ansible-core podman beakerlib | sort |
uniqansible-core-2.14.8-1.fc38.noarchbeakerlib-1.29.3-2.fc38.noarchpodman-4.6.2-1.fc38.x86_64standard-test-roles-4.11-2.fc38.noarchtmt-1.26.1-1.fc38.noarch

*🔍 Statistics*

-

Error rate ~ 3.82% from start of August
   http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=12&orgId=1&from=1690840800000&to=1693432799000
   -

~ 65k testing requests in August
   http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=8&orgId=1&from=1690840800000&to=1693432799000
   -

Average queued time is unavailable due to change of metrics, will resume
   in the next release