On 23 August 2016 at 14:41, Kushal Das <mail(a)kushaldas.in>
wrote:
> On 22/08/16, Stephen John Smoogen wrote:
>> On 22 August 2016 at 02:12, Kushal Das <mail(a)kushaldas.in> wrote:
>> > Hi,
>> >
>> > We want to introduce new Autocloud test cases. This will not require
>> > change in any configure or package version. But it will create a new
>> > tarball of testcases for Autocloud which is accessible at [1].
>> >
>> > I would also request to decide if we can keep the tests outside of the
>> > release freeze so that we can add/update the tests as required.
>> >
>> > [1]
>> >
http://infrastructure.fedoraproject.org/infra/autocloud/tunirtests.tar.gz
>> >
>> >
>>
>> In order to give an informed opinion, I need to know:
>>
>> 1) What are the upsides of doing this?
> We will be able to update the two week atomic related tests regularly. As the
> technology gets regular updates (read super new changes), updating the
> tests will also be required to keep up with the upstream. It is being
> consumed for automated two week atomic releases.
>
>> 2) What are the downsides of doing this?
> Someone may write wrong test case which somehow no one noticed, this may
> create false failures, but this is a standard problem in any testing I
> guess.
>
>> 3) If the downsides happen what is the plan on backing out whatever broke?
> Any thing breaks, we generally have to manually verify the issue, and
> update tests if required.
>
>> 4) What effects could the downsides have on the alpha deliverables.
> tl;dr This does not affect Fedora releases other than two week atomic.
>
> This is formally being used only for two week atomic, for standard
> Fedora releases, we still verify the images manually with the official
> QA tests. Autocloud in the other hand does a lot extra tests for two
> week atomic releases.
Thank you. The above makes it to me a non-blocker on alpha and can be +1
I still need one more +1 for this :)
Kushal
--
Fedora Cloud Engineer
CPython Core Developer