On 02/26/2018 06:32 AM, Petr Vobornik wrote:
Hi thanks for the feedback, comments inline
On Mon, Feb 26, 2018 at 8:59 AM, Standa Laznicka <slaznick(a)redhat.com> wrote:
> On 02/23/2018 05:26 PM, Robbie Harwood via FreeIPA-devel wrote:
>
> Petr Vobornik via FreeIPA-devel <freeipa-devel(a)lists.fedorahosted.org>
> writes:
>
> Felipe made nightly testing working as PRs in freeipa main Git Hub
> repo.
>
> Is there really not a better way to do this than spamming freeipa-devel
> with two more PRs every day?
I guess it can be optimized a bit so that notification mails are a bit
more useful. I agree that notification mails about opening and closing
PR don't bring any added value. Maybe it can be suppressed in
https://github.com/freeipa/freeipa-tools/tree/master/github-email-notific...
On the other hand, what is desired is to show results of nightly
testing to project maintainers and contributors. Because tests are
useful only if their results are visible and it drives actions.
Therefore I find beneficial to send a mail with test failures. This is
not working yet. A possible technical solution might be to create a
task depending on all other tasks which would get the results and
would create a comment with them thus creating a notification mail.
>
>
> +1, it messes up the PR queue, too, either make it use the same PR or use
> another repo. The current state is unbearable.
I don't see a benefit in moving it to other repo. It would lose the
visibility and therefore would become half-useless.
Actually, if we want to use the PR CI resources, we can't move it to
another repo. The runners points to freeipa/freeipa repo.
PR CI was designed to watch the PRs list, so it would not be simple to
do not use this way of running tests.
IMHO the easiest way is:
- Run nightly PRs two or three times per week (Mon and Friday or Mon,
Wed, Fry)
- Keep the PRs on freeipa/freeipa repo
- people create some filter in their email client if they don't want to
receive the notifications
As Petr already mentioned, it's important to keep them on GitHub because
it's easy to have a historic, it keep them public available and it
already uses all of our infrastructure. If we "hide" the results, we
would have the same problem that Jenkins has: it's not easy to see the
results.
>
> Reusing PRs could be a way to limit some noise. But if I'm correct old
> prs are shown back in the queue. IMO the goal here should be to show
> nightly testing on a first PR page. One way is to have fewer PRs ;).
> Other optimization could be e.g. to have weekly nightly prs - meaning
> one PR would be updated 5-7 times.
>
> IMO it is also useless to run nightly testing over weekend as no one
> is currently pushing any patches during the time. So I'd test on
> Friday evening and then on a Monday evening.
>
> Could you describe in more details what you mean by "messes up the PR
> queue" or "unbearable? These are quite hard words but their meaning is
> not specific. Could be interpreted in many ways. It's good to express
> feeling but it is hard to grasp. What is the effect it creates from
> your perspective?
>
>>
>>
>> Travis has cronjob support; wouldn't this be a better fit there? Why
>> does it need to be a PR?
>
> PR is a way how to show results and coordinate test runners. The
> testing infra used for it doesn't currently work without PR. How it
> works now:
https://github.com/freeipa/freeipa-pr-ci/tree/master/doc I
> have no idea how travis cron jobs work. But if it uses Travis infra
> then it is a no-go as Travis cannot handle our load.
>
>>
>>
>> I second the opinion that github PR is just a bad place to place nightly CI
>> run results to and it already shows.
>
> Could you explain what you mean by "it already shows"?
>
>>
>>
>> Like I suspect many users, I will be muting these.
>>
>> Thanks,
>> --Robbie
>
>