On Wed, Apr 14, 2021 at 3:57 PM Coiby Xu <coxu(a)redhat.com> wrote:
Thanks for the reply!
On Wed, Apr 14, 2021 at 02:16:02PM +0200, Miroslav Vadkerti wrote:
>On Tue, Apr 13, 2021 at 12:57 PM Coiby Xu <coxu(a)redhat.com> wrote:
>> On Tue, Apr 06, 2021 at 05:05:03PM +0200, Petr Šplíchal wrote:
>> >On Tue, 6 Apr 2021 at 11:28, Coiby Xu <coxu(a)redhat.com> wrote:
>> >> Hi Tomas,
>> >> Thank you for the reply!
>> >> On Tue, Apr 06, 2021 at 10:19:51AM +0200, Tomas Tomecek wrote:
>> >> >Hi Coiby, thanks for reaching out!
>> >> >
>> >> >Packit service is using the Testing Farm (TF) project to run
>> >> >itself uses tmt and fmf projects [tmt]. So if you want to run your
>> >> >tests in the packit workflow, you'd need to make sure they run
>> >> >correctly with tmt tool itself. I am CCing Petr Splichal and Miro
>> >> >Vadkerti, the lead developers of tmt and TF projects. Petr is
>> >> >running periodic onboarding sessions where he's helping
>> >> >onboard their projects. I'm not sure if TF supports testing VM
>> >> >right now, I will defer answering that question to the guys.
>> >> >
>> >> >[tmt] https://docs.fedoraproject.org/en-US/ci/tmt/#_summary
>> >> > https://tmt.readthedocs.io/en/stable/
>> >> >
>> >> Last week I played with tmt to run the shell tests. For kdump tests,
>> >> important question is whether TF supports testing VM reboots. So I
>> >> for Petr Splichal and Miro Vadkerti's answer.
>> >As for now, reboot during the test execution itself is not supported in
>> >tmt. Here's the issue where we discuss the feature:
>> > - https://github.com/psss/tmt/issues/281
>> >Reboot is possible using an ansible playbook during the prepare step:
>> > - https://tmt.readthedocs.io/en/latest/spec/plans.html#ansible
>> >However, I'm not sure this approach would be sufficient for the use
>> >you've described (e.g. several vm instances). I guess Miro will provide
>> >some more details from the TFT point of view.
>> Hi Petr,
>> Before Miro provides more details, I'd like to run the tests inside a
>> and manage multiple VM instances by the tests themselves to evaluate
>> the benefits of Testing Farm.
>I expect in this reply that "manage multiple VM instances by tests" means,
>that your tests will take care of provisioning of the VMs via available
>/dev/kvm (e.g using libvirt, qemu-kvm, or similar ...) on the test host
>If that is not what you are expecting, I guess I will need to reply again
Yes, what the kdump tests  do is to use qemu-kvm to start multiple VM
instances. This part of work has already be implemented by my collogue.
But this is not I'm expecting. I'm afraid you need you reply again.
Since Packit supports running tests inside a VM, I expect packet to
provision multiple VM instances for a single test and make the console
output of the VMs accessible to test.
Unfortunately multihost support is not yet implemented.
All testing we do is single host.
If this works, one great benefit
of using Packit is I can run the sames tests for both upstream and
downstream kexec-tools and dracut. Because commits backported from
upstream dracut could also break kexec-tools.
Yeah, we would like definitely to see multihost supported ...
Btw, the kdump tests have already implemented a framework to provision
multiple VM instances and watch the console output. Maybe I can help
implement these features for Packit?
Not Packit. We first need to outline the syntax of multihost
test definition in tmt. Then implement it in the tmt tool. As
the last step we need to make sure this also works in Testing Farm,
the service which runs tmt tests for Packit.
Of course, we are very happy for any contribution. So if you have
some ideas, how this could work, share your experiences that would
>Currently, for Fedora CI and Packit, we use AWS EC2 instances, which do
>nested virtualization and we do not yet have bare metal support with AWS
I see. So a AWS EC2 instance doesn't support starting a VM via qemu,
Yes, AWS EC2 does not support nested virtualization in VMs,
just on bare metal.
>So ... the only way to make this work **now** is to use internal
>but that is outside of Packit, and you would need to call
>Testing Farm service yourself (from Github actions for example), scripting
>That way, you could, from upstream pull request, provision internal
>infrastructure and return back results. The only drawback of this solution
>is that the artifacts storage are not accessible from the public, you will
>just get an xunit XML with Pass/Fail results, but the links will lead to
>Red Hat infrastructure.
>Unfortunately, as we do not yet have any users doing exactly this, this
>be a prototype for your team. But should be possible ...
>If the internal artifact storage is a no-go for you, we do not have a
>solution right now. But we will be glad to talk about possibilities, like
>prototyping the use of AWS bare metal instances ....
>If the problem is that this goes around Packit, we can have a talk if this
>(run tests against internal infrastucture) is something we could also
>support via Packit.
This feature is nice to have but I could prefer what has been described
Me too :) Proper tmt support would be the best.
>But this also needs some additional work ...
>Unfortunately we are a bit behind with docs, so the best sources we now
Thanks for recommending these docs!
>But what I wrote here is not included there ...
>> Tomas said you are running periodic
>> onboarding sessions. When will you launch the next session? Can you
>> some materials with me since docs.fedoraproject.org/en-US/ci
>Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / BaseOS QE /
>IRC mvadkert #tft #baseosci #osci :: GPG 0x7B5B2E95
>TPB-C 2C215 :: Mobile +420 773 944 252
>Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR
Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / BaseOS QE / OSCI
IRC mvadkert #tft #baseosci #osci :: GPG 0x7B5B2E95
TPB-C 2C215 :: Mobile +420 773 944 252
Red Hat Czech s.r.o, Purkyňova 115, 612 00, Brno, CR