Integration per-component tests in Fedora

[Matthew Miller]

On Tue, Oct 21, 2014 at 01:53:43PM +0200, Honza Horak wrote:
> >There was previously some discussion on this (I think on -devel? I'll
> >look for the thread)
> Ah, thanks, I probably found it:
> https://lists.fedoraproject.org/pipermail/devel/2014-January/193498.html

Yes, there it was. Thanks. :)


> So, the only disadvantage I found was that keeping tests further
> from components' sources might make users not keeping tests in sync
> with packages.

Adam's point in that thread was (and I think I'm representing it
fairly) that spec files are already horrendous enough, and that we
don't want to add "learn how to package RPMs" to the bar to cross in
order to contribute tests.

(And I guess along the same lines, if you're not a maintainer of the
package, adding tests causes you to add extra packaging work for the
people who are the maintainers.)

Once we have disposable/relatively-safe test sandboxes, I was thinking
that possibly anyone in some low-threshold test-writer group could have
commit access to all packages, and all tests would run, but each test's
impact would be decided by a higher-threshold group. For example, if I
create a test for httpd, it would first send results just to me and to
some big bucket of all results. Then, it could be increased to send
failures to the maintainers and/or to bodhi and/or to the devel or
testing lists, and with varying levels of impact in bodhi (block, -1,
0, or whatever.)


-- 
Matthew Miller
<mattdm at fedoraproject.org>
Fedora Project Leader