Taskbot and Automation

[Tim Flink]

On Tue, 28 May 2013 07:21:39 -0400 (EDT)
Kamil Paral <kparal at redhat.com> wrote:

> > Before spending a bunch more time on this, I want to see what the
> > general thoughts were on the approach.
> > 
> >  - Does the general concept make sense for Fedora?
> >  - Is this something we should pursue?
> 
> I already commented in the blog a bit, so just a recap: I feel we
> need to move forward in this direction sooner or later, because the
> lack of basic automation is a really pain in Fedora. Also a lot of
> people are asking for it and could leverage such a system. Your
> currently proposed concept (when leaving out technical details, e.g.
> whether something should use JSON or REST, which I'm not really
> experienced enough to discuss properly) looks very good. Of course we
> will need to think deeply about each part to make sure we didn't
> forget any important use case.

I'm approaching this with the assumption that I'm going to miss some
use case that will eventually be important. One of the reasons why I'm
so strongly for JSON-via-REST is that I want loose coupling between
components so that if/when we end up wanting to replace one part (the
scheduler, the execution engine, the results storage system etc.), we
can do that without any more pain than absolutely necessary.

In my mind, we can either pretend to be clairvoyant, attempt to
anticipate everything and fail or we can plan to change over time and
iterate as the need presents itself.

> I'd love to re-use any many bits as possible (either already
> developer by us or some existing third-party tools/libraries),
> because re-inventing every wheel just to create a "perfect fit" is
> extremely costly.

I have no desire to get rid of good code or to re-invent the wheel just
because we can or due to NIH syndrome. At the same time, I'm willing to
at least consider throwing out anything we need to in order to make a
better system.

I don't think that making a perfect system is reasonable but at the
same time, I don't want to be handicapped for the sake of being more
efficient on paper. After a certain point, the maintenance and
workaround cost can outweigh the value of not having to re-implement.

> > 
> > If so, I think that some of the next steps should be:
> > 
> >  - move the code somewhere so that others can contribute
> 
> Fedora Hosted or Github? I am not opposed to using non Fedora hosted
> services, and Github provides some nice bonuses, like integrated
> patches reviews.

Good question and I wish I had a good answer. Personally, I prefer
bitbucket to github if we're talking about closed source services but I
suspect that I might the only one. 

I also think that reviewboard has improved dramatically since we were
using it for autoqa so the code review part isn't as big of a deal as
it used to be.

> >  - migrate the proof-of-concept system to either autoqa-stg or
> > fedora cloud systems (95% of the setup is ansible-ized so migration
> > isn't too painful)
> 
> Autoqa-stg is not needed at the moment, I have no objections to tear
> it down completely.

Yeah, I didn't think there would be much objection to doing so. I think
that will be best utilized as a test bed for different system
management options.

> >  - polish the bits that are there so that they actually do most of
> > the things I'm talking about
> >  - do some investigation to be somewhat sure that we're not ignoring
> >    existing tools (autotest is first on my list, beaker is probably
> >    worth exploring a bit)
> 
> This comparison will not be easy, the tools are large with lots of
> features. It might be beneficial to include their developers (e.g.
> lmr from autotest) in the discussions what we need and what the tools
> can offer. We have a lot of experience with autotest, but I somewhat
> expect that there are many more features that we haven't even tried
> to discover.

Yeah, I have no illusions that the comparison will be straight forward.
The systems have very little in common outside of being written in
python and the fact that from a high level, they both execute jobs. I
suspect that a lot of it is going to come down to how much we want
flexibility.

Autotest is by far more featureful than buildbot from a test execution
perspective - it already has a test running system, it already has
concepts of more complicated results and supports jobs on remote and
multiple remote systems (probably more, those are the ones that come to
mind right away). At the moment, I think buildbot is a better fit for
the idea of a loosely coupled system of components joined by json and
rest but creating such a system will likely require more initial work
to fill some gaps in functionality.

Both will require code to get started, both have their strengths and
weaknesses but I suspect that any comparison could turn into a separate
thread.

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/qa-devel/attachments/20130528/8e3f897c/attachment.sig>