[release] Testing Farm 2024-02.1

[Jan Havlin]

 Hi,

Testing Farm release 2024-02.1
<https://docs.testing-farm.io/Testing%20Farm/0.1/releases/2024-02.1.html>
is deployed 🎉.
💔 Breaking Changes

The multihost testing from now uses tmt run provision --update-missing, see
docs <https://tmt.readthedocs.io/en/stable/stories/cli.html#update-missing>;.
This means it will update the image attribute in provision step with the
environemnts[].compose API field only if it is not set in the plan. This
might break your current multihost plans. Please update your plans
accordingly.
🚀 Upgrades

Testing Farm CLI <https://docs.testing-farm.io/Testing%20Farm/0.1/cli.html>
tool updated from v0.0.16 to v0.0.17
<https://gitlab.com/testing-farm/cli/-/releases/v0.0.17>;, please update
your installation.
🎯 Highlights

Created RFD2
<https://docs.testing-farm.io/Testing%20Farm/0.1/rfd/rfd2-testing-farm-rep...
for the Testing Farm in config repo which allows configuring secrets in the
repository(TFT-1813 <https://issues.redhat.com/browse/TFT-1813>). Feel free
to provide feedback to this planned major feature of Testing Farm.

Added support by selecting tmt tests by name via the --test option in CLI
or test.tmt.test_name field in the API (TFT-2421
<https://issues.redhat.com/browse/TFT-2421>).

RHEL-9.5.0 available on Red Hat ranch (TFT-2434
<https://issues.redhat.com/browse/TFT-2434>).

CentOS Stream 10 available on Public ranch (TFT-2442
<https://issues.redhat.com/browse/TFT-2442>).

Multi-host now uses tmt run provision --update-missing improving the
multi-host testing by filling in only missing compose (TFT-2379
<https://issues.redhat.com/browse/TFT-2379>).

Testing Farm now prints a warning when how: artemis is used in provision
step (TFT-2306 <https://issues.redhat.com/browse/TFT-2306>) and crashes
when anything else other than how: container or how is unspecified (TFT-2320
<https://issues.redhat.com/browse/TFT-2320>).

Fedora CI docs <https://docs.fedoraproject.org/en-US/ci/testing-farm/> now
document Testing Farm as their testing infrastructure (TFT-938
<https://issues.redhat.com/browse/TFT-938>).

Added support for configuring how many plans should be executed in parallel
via the --parallel-limit option in the CLI or
settings.pipeline.parallel-limit field in the API (TFT-2262
<https://issues.redhat.com/browse/TFT-2262>).

Automated importing of updates-nightly composes (TFT-2401
<https://issues.redhat.com/browse/TFT-2401>).

Added an artifacts link to results viewer page of a running TF test (
TFT-2405 <https://issues.redhat.com/browse/TFT-2405>).

Xunit is generated also in case of pipeline errors (TFT-544
<https://issues.redhat.com/browse/TFT-544>).

Time attribute is now shown in xunit and results viewer (TFT-2007
<https://issues.redhat.com/browse/TFT-2007>).
🐞 Bugfixes

Test environment variable TESTING_FARM_REQUEST_USERNAME is now available
again (TFT-2433 <https://issues.redhat.com/browse/TFT-2433>).

Leakage of secrets in testout.log was fixed (TFT-2462
<https://issues.redhat.com/browse/TFT-2462>).

Fixed a bug where volumes from deployments in EKS clusters were not tagged (
TFT-2313 <https://issues.redhat.com/browse/TFT-2313>).

Fixed showing incomplete pipeline.log file in the artifacts (TFT-2451
<https://issues.redhat.com/browse/TFT-2451>).

Fixed incorrect Beaker filter for virtualization.is-supported and aarch64 (
TFT-2453 <https://issues.redhat.com/browse/TFT-2453>).

Fixed an issue with artifact artifaction at the end of the pipeline (
TFT-2413 <https://issues.redhat.com/browse/TFT-2413>).
🫂 Contributors

We would like to thank the following external contributors for contributing
to Testing Farm.

   -

   Inessa Vasilevskaya — in Testing Farm CLI, arch is passed into tmt
   context (cli!97 <https://gitlab.com/testing-farm/cli/-/merge_requests/97>
   ).

📦️ Packages

List of important packages bundled in the worker image.

❯ podman run --entrypoint rpm quay.io/testing-farm/worker:2024-02.1 -q
tmt standard-test-roles ansible-core podman beakerlib | sort | uniq
ansible-core-2.14.11-2.fc38.noarch
beakerlib-1.29.3-2.fc38.noarch
podman-4.9.3-2.fc38.x86_64
standard-test-roles-4.11-2.fc38.noarch
tmt-1.31.0-1.fc38.noarch

🔎 Stats

Testing Farm is on track to surpass 1.1M requests per year
<https://stats.testing-farm.io/d/dpYooDIVk/testing-farm-all-time-stats> \o/.

Error rate in February was approximately 2.88%
<http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=12&...
.

Around 98k testing requests
<http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=8&o...
were recorded in February.

Average queue time in February was 1.33 minutes on the Public ranch
<http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=26&...
and 3.54 minutes on the Red Hat ranch
<http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=27&...
.