Automatic Smoketests for the Cloud Images: What to Test?

Sandro "red" Mathys red at fedoraproject.org
Fri Apr 4 04:34:09 UTC 2014


On Fri, Mar 7, 2014 at 8:20 PM, Vitaly Kuznetsov <vitty at redhat.com> wrote:
> "Sandro \"red\" Mathys" <red at fedoraproject.org> writes:
>
>> On Fri, Mar 7, 2014 at 7:32 PM, Vitaly Kuznetsov <vitty at redhat.com> wrote:
>>>> So we have the RedHatQE tests, Taskotron and CentOS's CI. Can anyone
>>>> of the people involved (at the Red Hat side, I guess) well me why we
>>>> have 3 systems for 1 task?
>>>
>>> (my personal opinion) I think we rather have plenty of tasks, not
>>> one. Afaict (after 5 min. of reading Taskotron's development plan
>>> https://fedoraproject.org/wiki/User:Tflink/taskotron_development_plan)
>>> Taskotron is designed to replace AutoQA in the first place.
>>> RHEL's Cloud Image Validation was developed several years ago when the
>>> following task was on the table: we have many AWS regions, many images,
>>> different architectures, we need to try different hardware types and
>>> AWS-specific features (e.g. attach EBS on the fly or test AWS-specific
>>> content delivery) and finally we need to aggregate the result. Existing
>>> test infrastructure was built around Beaker which is not that well
>>> suited for the job and creating a separate tool was considered a
>>> reasonable trade-off.
>>
>> Well, "one" task as in "do cloud image QA".
>>
>> Thanks, for sharing that insights, really helpful to help my
>> understanding. So, do you currently test EC2 only? (Not saying that's
>> necessarily bad / too little).
>
> Now it is EC2-only but Google's ComputeEngine was on the horizon.
>
>>
>> Now, we do have the RHQE stuff in place and it's already used for
>> testing Fedora images...that's good. Is that fully automated? Or to
>> what extend?
>
> You run the tool with the data (AMI IDs, region, arch) and get the
> result in a meanwhile. It can be fully-automated once we have this data
> announced via fedmsg or in any other automated way (now I just read
> mailing list and if there are any images announced by Dennis I run the
> tool).
>
>>
>>>> When I took ownership of this "external
>>>> need" (for the Fedora cloud product) I was under the impression we
>>>> only just (are going to) have Taskotron and everyone knows it's THE
>>>> way to go.
>>>
>>> I personally love collaboration. It would be awesome if we could avoid
>>> spreading resources on '3 systems for 1 task'. I definitely want to know
>>> more about Taskotron and its movement towards cloud image testing.
>>
>> That's why I was a bit confused to find there's actually 3 systems.
>> Collaboration is certainly great, but that's not how it's done so
>> let's try to improve on this.
>>
>> So, would you recommend to keep using your tools or rather go with
>> Taskotron? Or do we do some things in one and others in the other? Or
>> do we try to fully implement your tests in Taskotron and drop doing
>> the tests with your tools?
>>
>
> Well, it depends on what's our future plan. IMHO once we have images
> announced via fedmsg we can have all basic things covered by the existing
> tool (and I'm definitely in for integration and support process for the
> tool) and it won't take us long to set everything up. With regards to
> Taskotron I want to know more on how this 'cloud integration' is planned
> as (if I'm not mistaken) there's no code written yet. If merging here
> seems reasonable then I'm in. I'll try reaching out to Tim & others on
> fedora-qa-devel list.

So, what's the status here? Tim's responses to this thread show no
cloud integration code has been written yet and he's open to have
valid integrated in Taskotron, particularly if helping hands do most
of the work so he can keep focusing on other open tasks. Could you
work on that, Vitaly?

>> Also, Karanbir, what's your (i.e. CentOS's) story? You say you already
>> have a CI system running but shared little other information. What CI
>> system? Did you already implement image tests? What kind of
>> collaboration would you suggest here?


More information about the cloud mailing list