On 02/23/2018 09:02 AM, Petr Vobornik wrote:
Hi all,
Felipe made nightly testing working as PRs in freeipa main Git Hub repo.
e.g.:
https://github.com/freeipa/freeipa/pull/1624
https://github.com/freeipa/freeipa/pull/1623
This is good. We can see the results publicly in comparison to
"upstream Jenkins". I also believe that it will be more stable. But
time will tell it. A disadvantage is more noise in mail notifications,
but that is not that drastic.
Let's discuss some aspects. First one is definitions of the tests
(.freeipa-pr-ci.yaml) and how to extend them.
We currently use 3 definitions:
* gating, the definition in repo
* nightly master tests
* nighly rawhide tests
Gating definition is in repo. But both nightly are not.
What is the way how other team members can extend this definition?
Where are they kept?
Could we also put them into a repo?
What about the following solution:
1. create a directory with tests definitions, e.g. /ipatests/definitions
2. put these 3 files there
- gating
- nightly_master
- nightly_rawhide
3. make .freeipa-pr-ci.yaml a symlink to /ipatest/definitions/gating
4. for nightly testing do PR which would change only the symlink
Sounds a good solution for me. We could go that way. This would
require a small change in the github/openclose_pr script, but that's
easy.
> Benefits:
> - anybody can extend nightly test suite
> - the nighlty test suite can be extended right away in a PR which adds the tests
> - nightly tests won't fail on merge conflict when somebody extends
> current gating spec
>
> Possible alternative is to make it more complex and use something like
> includes instead of symlinks, but that would need update of PR-CI and
> I don't see a real benefit in comparison to the symlink solution.
>