Automatic Smoketests for the Cloud Images: What to Test?

[Sandro "red" Mathys]

On Fri, Apr 4, 2014 at 1:35 PM, Tim Ski <marshyski at gmail.com> wrote:
> I said I would help with this but someone else took the lead. What's the 411
> haha???

Where did you say so? I haven't seen any such message and neither did
the cloud list's archive. Anyway, what exactly would you like to help
with?

> On Apr 4, 2014 12:34 AM, "Sandro "red" Mathys" <red at fedoraproject.org>
> wrote:
>>
>> On Fri, Mar 7, 2014 at 8:20 PM, Vitaly Kuznetsov <vitty at redhat.com> wrote:
>> > "Sandro \"red\" Mathys" <red at fedoraproject.org> writes:
>> >
>> >> On Fri, Mar 7, 2014 at 7:32 PM, Vitaly Kuznetsov <vitty at redhat.com>
>> >> wrote:
>> >>>> So we have the RedHatQE tests, Taskotron and CentOS's CI. Can anyone
>> >>>> of the people involved (at the Red Hat side, I guess) well me why we
>> >>>> have 3 systems for 1 task?
>> >>>
>> >>> (my personal opinion) I think we rather have plenty of tasks, not
>> >>> one. Afaict (after 5 min. of reading Taskotron's development plan
>> >>> https://fedoraproject.org/wiki/User:Tflink/taskotron_development_plan)
>> >>> Taskotron is designed to replace AutoQA in the first place.
>> >>> RHEL's Cloud Image Validation was developed several years ago when the
>> >>> following task was on the table: we have many AWS regions, many
>> >>> images,
>> >>> different architectures, we need to try different hardware types and
>> >>> AWS-specific features (e.g. attach EBS on the fly or test AWS-specific
>> >>> content delivery) and finally we need to aggregate the result.
>> >>> Existing
>> >>> test infrastructure was built around Beaker which is not that well
>> >>> suited for the job and creating a separate tool was considered a
>> >>> reasonable trade-off.
>> >>
>> >> Well, "one" task as in "do cloud image QA".
>> >>
>> >> Thanks, for sharing that insights, really helpful to help my
>> >> understanding. So, do you currently test EC2 only? (Not saying that's
>> >> necessarily bad / too little).
>> >
>> > Now it is EC2-only but Google's ComputeEngine was on the horizon.
>> >
>> >>
>> >> Now, we do have the RHQE stuff in place and it's already used for
>> >> testing Fedora images...that's good. Is that fully automated? Or to
>> >> what extend?
>> >
>> > You run the tool with the data (AMI IDs, region, arch) and get the
>> > result in a meanwhile. It can be fully-automated once we have this data
>> > announced via fedmsg or in any other automated way (now I just read
>> > mailing list and if there are any images announced by Dennis I run the
>> > tool).
>> >
>> >>
>> >>>> When I took ownership of this "external
>> >>>> need" (for the Fedora cloud product) I was under the impression we
>> >>>> only just (are going to) have Taskotron and everyone knows it's THE
>> >>>> way to go.
>> >>>
>> >>> I personally love collaboration. It would be awesome if we could avoid
>> >>> spreading resources on '3 systems for 1 task'. I definitely want to
>> >>> know
>> >>> more about Taskotron and its movement towards cloud image testing.
>> >>
>> >> That's why I was a bit confused to find there's actually 3 systems.
>> >> Collaboration is certainly great, but that's not how it's done so
>> >> let's try to improve on this.
>> >>
>> >> So, would you recommend to keep using your tools or rather go with
>> >> Taskotron? Or do we do some things in one and others in the other? Or
>> >> do we try to fully implement your tests in Taskotron and drop doing
>> >> the tests with your tools?
>> >>
>> >
>> > Well, it depends on what's our future plan. IMHO once we have images
>> > announced via fedmsg we can have all basic things covered by the
>> > existing
>> > tool (and I'm definitely in for integration and support process for the
>> > tool) and it won't take us long to set everything up. With regards to
>> > Taskotron I want to know more on how this 'cloud integration' is planned
>> > as (if I'm not mistaken) there's no code written yet. If merging here
>> > seems reasonable then I'm in. I'll try reaching out to Tim & others on
>> > fedora-qa-devel list.
>>
>> So, what's the status here? Tim's responses to this thread show no
>> cloud integration code has been written yet and he's open to have
>> valid integrated in Taskotron, particularly if helping hands do most
>> of the work so he can keep focusing on other open tasks. Could you
>> work on that, Vitaly?
>>
>> >> Also, Karanbir, what's your (i.e. CentOS's) story? You say you already
>> >> have a CI system running but shared little other information. What CI
>> >> system? Did you already implement image tests? What kind of
>> >> collaboration would you suggest here?
>> _______________________________________________
>> cloud mailing list
>> cloud at lists.fedoraproject.org
>> https://admin.fedoraproject.org/mailman/listinfo/cloud
>> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>
>
> _______________________________________________
> cloud mailing list
> cloud at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>