On Wed, Apr 19, 2017 at 12:26 PM, Stef Walter <stefw(a)redhat.com> wrote:
On 19.04.2017 11:29, Frederic Lepied wrote:
>
> On Wed, Apr 19, 2017 at 10:54 AM, Stef Walter <stefw(a)redhat.com
> <mailto:stefw@redhat.com>> wrote:
>
> Looks like Ansible has been chosen as the way we're going to invoke
> tests stored in dist-git.
>
>
https://fedoraproject.org/wiki/Changes/InvokingTests
> <
https://fedoraproject.org/wiki/Changes/InvokingTests>
>
https://fedoraproject.org/wiki/Changes/InvokingTestsAnsible
> <
https://fedoraproject.org/wiki/Changes/InvokingTestsAnsible>
>
>
> Even for those who voted for alternatives, the good news is that
almost
> all of both the packaged tests and the autopkgtest style tests can be
> wrapped in the Ansible style tests.
>
> I did some work to try out the Ansible proposal and finish it up
where
> there were rough edges or mysterious parts. I talked about these
changes
> with Pingou, Ari, and Martin.
>
> If you're interested in the details of what was changed or finished
up
> in the Ansible proposal:
>
>
https://fedoraproject.org/w/index.php?title=Changes%
2FInvokingTestsAnsible&diff=491284&oldid=490960
> <
https://fedoraproject.org/w/index.php?title=Changes%
2FInvokingTestsAnsible&diff=491284&oldid=490960>
>
>
> In addition some examples have been written. In particular, the sed
> tests here work both in-situ and against a composed Fedora Atomic
host
> image:
>
>
https://fedoraproject.org/wiki/Changes/InvokingTestsAnsible#Examples
> <
https://fedoraproject.org/wiki/Changes/
InvokingTestsAnsible#Examples>
>
>
https://fedoraproject.org/wiki/Changes/
InvokingTestsAnsible#Example:_Beakerlib_based_test
> <
https://fedoraproject.org/wiki/Changes/
InvokingTestsAnsible#Example:_Beakerlib_based_test>
>
>
> There'll be more updates to the examples on those pages, including
> finishing them up for the standard as it is, and making some parts of
> them more Ansibley.
>
>
>
> The proposition I proposed hasn't been really reviewed. I don't know the
> exact process for taking decisions like this but it seems rushed.
I'll add my review. Have you reached out folks to review it? Make sure
to email them?
Yes I sent an email but got a few out of office replies :-(
Shouldn't we send an email to fedora-devel to have more feedback?
The conclusion here is one driven by those who are going to do the work
migrating and wrapping the tests. By and large they have indicated they
prefer doing that with Ansible as the standard interface. It's up to
alternatives to reach out to them (see those who voted) and talk with
them to change their minds.
> I'm insisting on my proposal because that's more a superset of the 2
> other propositions than an alternative: you can package your tests or
> write them in ansible without any problem with the added values of
> having metadata about the tests like needing a real system, needing root
> access etc. and also being able to reuse tests from the Debian and
> Ubuntu communities.
This works with the three proposals in various combinations. The
staging, invocation and results can be wrapped either the following ways:
* autopkgtest + control can wrap and execute tests written in Ansible
* Ansible can execute tests wrapped in autopkgtest + control files
* Ansible tests can execute packaged tests
* Packaged tests can execute Ansible tests
* autopkgtest + control can execute packaged tests
In fact we already see some of these combinations in the examples. For
example GLib2 or Cockpit package their own tests in RPMs. These are then
executed by the tests laid out as Ansible tests.
I agree with this except that we cannot discover which packages are
involved for a particular test using the ansible approach. The only package
that will trigger the test will be the package that is being built while
with the other 2 approaches you can have a finer granularity in the other
packages needed for a particular test (with the control approach) or for
the all the tests (with the packaging approach).
Fred