Automatic Smoketests for the Cloud Images: What to Test?

[Tim Ski]

I said I would help with this but someone else took the lead. What's the
411 haha???
On Apr 4, 2014 12:34 AM, "Sandro "red" Mathys" <red at fedoraproject.org>
wrote:

> On Fri, Mar 7, 2014 at 8:20 PM, Vitaly Kuznetsov <vitty at redhat.com> wrote:
> > "Sandro \"red\" Mathys" <red at fedoraproject.org> writes:
> >
> >> On Fri, Mar 7, 2014 at 7:32 PM, Vitaly Kuznetsov <vitty at redhat.com>
> wrote:
> >>>> So we have the RedHatQE tests, Taskotron and CentOS's CI. Can anyone
> >>>> of the people involved (at the Red Hat side, I guess) well me why we
> >>>> have 3 systems for 1 task?
> >>>
> >>> (my personal opinion) I think we rather have plenty of tasks, not
> >>> one. Afaict (after 5 min. of reading Taskotron's development plan
> >>> https://fedoraproject.org/wiki/User:Tflink/taskotron_development_plan)
> >>> Taskotron is designed to replace AutoQA in the first place.
> >>> RHEL's Cloud Image Validation was developed several years ago when the
> >>> following task was on the table: we have many AWS regions, many images,
> >>> different architectures, we need to try different hardware types and
> >>> AWS-specific features (e.g. attach EBS on the fly or test AWS-specific
> >>> content delivery) and finally we need to aggregate the result. Existing
> >>> test infrastructure was built around Beaker which is not that well
> >>> suited for the job and creating a separate tool was considered a
> >>> reasonable trade-off.
> >>
> >> Well, "one" task as in "do cloud image QA".
> >>
> >> Thanks, for sharing that insights, really helpful to help my
> >> understanding. So, do you currently test EC2 only? (Not saying that's
> >> necessarily bad / too little).
> >
> > Now it is EC2-only but Google's ComputeEngine was on the horizon.
> >
> >>
> >> Now, we do have the RHQE stuff in place and it's already used for
> >> testing Fedora images...that's good. Is that fully automated? Or to
> >> what extend?
> >
> > You run the tool with the data (AMI IDs, region, arch) and get the
> > result in a meanwhile. It can be fully-automated once we have this data
> > announced via fedmsg or in any other automated way (now I just read
> > mailing list and if there are any images announced by Dennis I run the
> > tool).
> >
> >>
> >>>> When I took ownership of this "external
> >>>> need" (for the Fedora cloud product) I was under the impression we
> >>>> only just (are going to) have Taskotron and everyone knows it's THE
> >>>> way to go.
> >>>
> >>> I personally love collaboration. It would be awesome if we could avoid
> >>> spreading resources on '3 systems for 1 task'. I definitely want to
> know
> >>> more about Taskotron and its movement towards cloud image testing.
> >>
> >> That's why I was a bit confused to find there's actually 3 systems.
> >> Collaboration is certainly great, but that's not how it's done so
> >> let's try to improve on this.
> >>
> >> So, would you recommend to keep using your tools or rather go with
> >> Taskotron? Or do we do some things in one and others in the other? Or
> >> do we try to fully implement your tests in Taskotron and drop doing
> >> the tests with your tools?
> >>
> >
> > Well, it depends on what's our future plan. IMHO once we have images
> > announced via fedmsg we can have all basic things covered by the existing
> > tool (and I'm definitely in for integration and support process for the
> > tool) and it won't take us long to set everything up. With regards to
> > Taskotron I want to know more on how this 'cloud integration' is planned
> > as (if I'm not mistaken) there's no code written yet. If merging here
> > seems reasonable then I'm in. I'll try reaching out to Tim & others on
> > fedora-qa-devel list.
>
> So, what's the status here? Tim's responses to this thread show no
> cloud integration code has been written yet and he's open to have
> valid integrated in Taskotron, particularly if helping hands do most
> of the work so he can keep focusing on other open tasks. Could you
> work on that, Vitaly?
>
> >> Also, Karanbir, what's your (i.e. CentOS's) story? You say you already
> >> have a CI system running but shared little other information. What CI
> >> system? Did you already implement image tests? What kind of
> >> collaboration would you suggest here?
> _______________________________________________
> cloud mailing list
> cloud at lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/cloud
> Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/cloud/attachments/20140404/516cb1a5/attachment.html>