On 22/08/16, Stephen John Smoogen wrote:
On 22 August 2016 at 02:12, Kushal Das <mail(a)kushaldas.in>
wrote:
> Hi,
>
> We want to introduce new Autocloud test cases. This will not require
> change in any configure or package version. But it will create a new
> tarball of testcases for Autocloud which is accessible at [1].
>
> I would also request to decide if we can keep the tests outside of the
> release freeze so that we can add/update the tests as required.
>
> [1]
>
http://infrastructure.fedoraproject.org/infra/autocloud/tunirtests.tar.gz
>
>
In order to give an informed opinion, I need to know:
1) What are the upsides of doing this?
We will be able to update the two week
atomic related tests regularly. As the
technology gets regular updates (read super new changes), updating the
tests will also be required to keep up with the upstream. It is being
consumed for automated two week atomic releases.
2) What are the downsides of doing this?
Someone may write
wrong test case which somehow no one noticed, this may
create false failures, but this is a standard problem in any testing I
guess.
3) If the downsides happen what is the plan on backing out whatever
broke?
Any thing breaks, we generally have to manually verify the issue, and
update tests if required.
4) What effects could the downsides have on the alpha deliverables.
tl;dr This does not affect Fedora releases other than two week atomic.
This is formally being used only for two week atomic, for standard
Fedora releases, we still verify the images manually with the official
QA tests. Autocloud in the other hand does a lot extra tests for two
week atomic releases.
Kushal
--
Fedora Cloud Engineer
CPython Core Developer
https://kushaldas.in
https://dgplug.org