On Sun, Mar 03, 2019 at 11:45:31AM +0100, Miro HronĨok wrote:
On 01. 03. 19 22:19, Ben Cotton wrote:
> '''The CI system, the tests and the decision on which tests are used
> to gate upon are out of scope for the present document.'''
This is both good (specifying explicitly what is this change about and what
it is not about) and bad...
Since the CI system is not part of this change, we cannot say this change is
not complete if the CI system is not complete. So later we say we have
rawhide gating and we'll all be \o/ \o/ \o/ yet nobody will care that the CI
is unfortunately:
* not locally reproducible
https://pagure.io/fedora-ci/general/issue/4
* only working on on x86_64
https://pagure.io/fedora-ci/general/issue/16
* unreadable
https://pagure.io/fedora-ci/general/issue/2
* unreliable (I file 1.75 issues per month on average)
* understaffed (my personal observation)
* tedious to start using (we focus on standards instead of UX)
* untested (the (sometimes fatal) issues I face go unnoticed until I'm hit)
My concern is that the CI experience still feels a bit rough and I'd rather
see us focusing on making it better. This can of course be done in parallel
with this change, yet I feel that we are building this on water.
Note that I don't blame the CI people, they are very helpful and great to
work with, they just don't have time/resources/people.
I think you are raising very good points and that they should be looked into,
however, in the design of gating rawhide we tried to be test system agnostic,
so, as Adam already pointed out, you could gate on results any of our test
systems (we currently have three: the CI pipeline, taskotron, OpenQA) and
possibly other in the future.
I believe Dominik is working on a new CI initiative at the council level and
I'll let him reply if these (or some of these) concerns can be covered by this
new initiative.
Thanks again,
Pierre