Using tags to indicate context in which tests are able to run

[Stef Walter]

Many of the tests that are being written for dist-git packages or
modules have special assumptions about where they can execute. For
example, tests may interact with packages not present in an Atomic Host,
or run things that don't make sense in a container.

While some of this info can be inferred from the "test subject" in the
spec [0], there was a need for a rigorous way for packagers to what a
test was designed for.

To that end, Miroslav, Merlin and I worked on a change to the spec,
which would allow for tests to indicate in which context they are
designed to operate.

We'd use Ansible Tags for this. So far there are three tags "atomic",
"container" and "classic". Here's the updated page:

https://fedoraproject.org/wiki/Changes/InvokingTestsThree

And here's the specific changes:

https://fedoraproject.org/w/index.php?title=Changes%2FInvokingTestsThree&...

And here's an example of it in use:

https://upstreamfirst.fedorainfracloud.org/bash/pull-request/2

Broadly speaking the changes are backwards compatible, although tests
will want to start to tag themselves to at least operate in the
"classic" context.

Does this make sense? If so, next steps are to:

1. Merge those changes back into the main spec:

https://fedoraproject.org/wiki/Changes/InvokingTests

2. Update the tutorial style documentation:

https://fedoraproject.org/wiki/CI/Tests

3. Work through the current set of staged tests and put tags in.

Cheers,

Stef

[0] https://fedoraproject.org/wiki/Changes/InvokingTests

[Stef Walter]

So I've tried this out more and opened pull requests against acl, at,
attr, audit, bash, ca-certificates and gzip repositories here:

https://upstreamfirst.fedorainfracloud.org/browse/projects/

I've opened pull requests there ^^

In addition I've updated the spec and documentation for the tag changes:

https://fedoraproject.org/wiki/Changes/InvokingTests
https://fedoraproject.org/wiki/CI/Tests

Thanks for taking a look.

Stef

On 24.07.2017 17:43, Stef Walter wrote:
> Many of the tests that are being written for dist-git packages or
> modules have special assumptions about where they can execute. For
> example, tests may interact with packages not present in an Atomic Host,
> or run things that don't make sense in a container.
>
> While some of this info can be inferred from the "test subject" in the
> spec [0], there was a need for a rigorous way for packagers to what a
> test was designed for.
>
> To that end, Miroslav, Merlin and I worked on a change to the spec,
> which would allow for tests to indicate in which context they are
> designed to operate.
>
> We'd use Ansible Tags for this. So far there are three tags "atomic",
> "container" and "classic". Here's the updated page:
>
> https://fedoraproject.org/wiki/Changes/InvokingTestsThree
>
> And here's the specific changes:
>
> https://fedoraproject.org/w/index.php?title=Changes%2FInvokingTestsThree&...
>
> And here's an example of it in use:
>
> https://upstreamfirst.fedorainfracloud.org/bash/pull-request/2
>
> Broadly speaking the changes are backwards compatible, although tests
> will want to start to tag themselves to at least operate in the
> "classic" context.
>
> Does this make sense? If so, next steps are to:
>
> Merge those changes back into the main spec:
>
> https://fedoraproject.org/wiki/Changes/InvokingTests
>
> Update the tutorial style documentation:
>
> https://fedoraproject.org/wiki/CI/Tests
>
> Work through the current set of staged tests and put tags in.
>
> Cheers,
>
> Stef
>
> [0] https://fedoraproject.org/wiki/Changes/InvokingTests
>
> CI mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]