On 04. 03. 19 12:34, Pierre-Yves Chibon wrote:
On Sun, Mar 03, 2019 at 11:45:31AM +0100, Miro Hrončok wrote:
> On 01. 03. 19 22:19, Ben Cotton wrote:
>> '''The CI system, the tests and the decision on which tests are used
>> to gate upon are out of scope for the present document.'''
>
> This is both good (specifying explicitly what is this change about and what
> it is not about) and bad...
>
> Since the CI system is not part of this change, we cannot say this change is
> not complete if the CI system is not complete. So later we say we have
> rawhide gating and we'll all be \o/ \o/ \o/ yet nobody will care that the CI
> is unfortunately:
>
> * not locally reproducible
https://pagure.io/fedora-ci/general/issue/4
> * only working on on x86_64
https://pagure.io/fedora-ci/general/issue/16
> * unreadable
https://pagure.io/fedora-ci/general/issue/2
> * unreliable (I file 1.75 issues per month on average)
> * understaffed (my personal observation)
> * tedious to start using (we focus on standards instead of UX)
> * untested (the (sometimes fatal) issues I face go unnoticed until I'm hit)
>
> My concern is that the CI experience still feels a bit rough and I'd rather
> see us focusing on making it better. This can of course be done in parallel
> with this change, yet I feel that we are building this on water.
>
> Note that I don't blame the CI people, they are very helpful and great to
> work with, they just don't have time/resources/people.
I think you are raising very good points and that they should be looked into,
however, in the design of gating rawhide we tried to be test system agnostic,
so, as Adam already pointed out, you could gate on results any of our test
systems (we currently have three: the CI pipeline, taskotron, OpenQA) and
possibly other in the future.
So that leaves 2 things:
* Fedora Ci with the above problems
* OpenQA
I honestly don't except many packagers writing their own OpenQA tests.
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok