Hi Miro,
Thanks for the reply!
On Wed, Apr 14, 2021 at 02:16:02PM +0200, Miroslav Vadkerti wrote:
Hi Coiby,
On Tue, Apr 13, 2021 at 12:57 PM Coiby Xu <coxu(a)redhat.com> wrote:
> On Tue, Apr 06, 2021 at 05:05:03PM +0200, Petr Šplíchal wrote:
> >On Tue, 6 Apr 2021 at 11:28, Coiby Xu <coxu(a)redhat.com> wrote:
> >
> >> Hi Tomas,
> >>
> >> Thank you for the reply!
> >>
> >> On Tue, Apr 06, 2021 at 10:19:51AM +0200, Tomas Tomecek wrote:
> >> >Hi Coiby, thanks for reaching out!
> >> >
> >> >Packit service is using the Testing Farm (TF) project to run tests. TF
> >> >itself uses tmt and fmf projects [tmt]. So if you want to run your
> >> >tests in the packit workflow, you'd need to make sure they run
> >> >correctly with tmt tool itself. I am CCing Petr Splichal and Miro
> >> >Vadkerti, the lead developers of tmt and TF projects. Petr is actually
> >> >running periodic onboarding sessions where he's helping engineers
to
> >> >onboard their projects. I'm not sure if TF supports testing VM
reboots
> >> >right now, I will defer answering that question to the guys.
> >> >
> >> >[tmt]
https://docs.fedoraproject.org/en-US/ci/tmt/#_summary
> >> >
https://tmt.readthedocs.io/en/stable/
> >> >
> >>
> >> Last week I played with tmt to run the shell tests. For kdump tests, an
> >> important question is whether TF supports testing VM reboots. So I wait
> >> for Petr Splichal and Miro Vadkerti's answer.
> >>
> >
> >Hi!
> >
> >As for now, reboot during the test execution itself is not supported in
> >tmt. Here's the issue where we discuss the feature:
> >
> > -
https://github.com/psss/tmt/issues/281
> >
> >Reboot is possible using an ansible playbook during the prepare step:
> >
> > -
https://tmt.readthedocs.io/en/latest/spec/plans.html#ansible
> >
> >However, I'm not sure this approach would be sufficient for the use case
> >you've described (e.g. several vm instances). I guess Miro will provide
> >some more details from the TFT point of view.
> >
>
> Hi Petr,
>
> Before Miro provides more details, I'd like to run the tests inside a
> docker
> and manage multiple VM instances by the tests themselves to evaluate
> the benefits of Testing Farm.
I expect in this reply that "manage multiple VM instances by tests" means,
that your tests will take care of provisioning of the VMs via available
/dev/kvm (e.g using libvirt, qemu-kvm, or similar ...) on the test host ...
If that is not what you are expecting, I guess I will need to reply again
...
Yes, what the kdump tests [2] do is to use qemu-kvm to start multiple VM
instances. This part of work has already be implemented by my collogue.
But this is not I'm expecting. I'm afraid you need you reply again.
Since Packit supports running tests inside a VM, I expect packet to
provision multiple VM instances for a single test and make the console
output of the VMs accessible to test. If this works, one great benefit
of using Packit is I can run the sames tests for both upstream and
downstream kexec-tools and dracut. Because commits backported from
upstream dracut could also break kexec-tools.
Btw, the kdump tests have already implemented a framework to provision
multiple VM instances and watch the console output. Maybe I can help
implement these features for Packit?
[2]
https://src.fedoraproject.org/rpms/kexec-tools/blob/f34/f/tests
....
Currently, for Fedora CI and Packit, we use AWS EC2 instances, which do not
support
nested virtualization and we do not yet have bare metal support with AWS :/.
I see. So a AWS EC2 instance doesn't support starting a VM via qemu,
right?
So ... the only way to make this work **now** is to use internal
infrastructure,
but that is outside of Packit, and you would need to call
Testing Farm service yourself (from Github actions for example), scripting
the communication.
That way, you could, from upstream pull request, provision internal
infrastructure and return back results. The only drawback of this solution
is that the artifacts storage are not accessible from the public, you will
just get an xunit XML with Pass/Fail results, but the links will lead to
internal
Red Hat infrastructure.
Unfortunately, as we do not yet have any users doing exactly this, this
would
be a prototype for your team. But should be possible ...
If the internal artifact storage is a no-go for you, we do not have a
possible
solution right now. But we will be glad to talk about possibilities, like
prototyping the use of AWS bare metal instances ....
If the problem is that this goes around Packit, we can have a talk if this
feature
(run tests against internal infrastucture) is something we could also
support via Packit.
This feature is nice to have but I could prefer what has been described
before.
But this also needs some additional work ...
Unfortunately we are a bit behind with docs, so the best sources we now
have are:
https://packit.dev/docs/testing-farm/
https://docs.fedoraproject.org/en-US/ci/tmt/
Thanks for recommending these docs!
But what I wrote here is not included there ...
Best regards,
/M
> Tomas said you are running periodic
> onboarding sessions. When will you launch the next session? Can you share
> some materials with me since
docs.fedoraproject.org/en-US/ci seems to be
> outdated?
>
--
Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / BaseOS QE / OSCI
IRC mvadkert #tft #baseosci #osci :: GPG 0x7B5B2E95
TPB-C 2C215 :: Mobile +420 773 944 252
Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR
--
Best regards,
Coiby