What to do about ELN-related "spam"?

[Fabio Valentini]

Hi everybody,

I'm unsure how to deal with this, so I'm asking here if anybody has ideas.

- I'm getting serious spam from fedora-notif on IRC about successful -
and a lot more failed - ELN builds for my packages (triggered either
by ELN developers or automatically by their CI)
- bodhi is spamming bugzilla by changing the "Fixed in Version" field
every time a new ELN build succeds with a new %dist value (one
bugzilla email each for eln106, eln107, eln108, etc.)

At this point, almost half of the bugzilla mail I get and a majority
of the IRC notifications are "ELN spam" that I do not know how to deal
with / is not actionable for me. It's now so bad that I'd like to get
the packages that are failing to build (multiple times a day) in ELN
fixed just so I don't get spammed with "failed to build" notifications
anymore ...

Fabio

[Petr Šabata]

On Thu, Dec 17, 2020 at 12:27 PM Fabio Valentini <decathorpe(a)gmail.com&gt; wrote:
>  Hi everybody,
>
>  I'm unsure how to deal with this, so I'm asking here if anybody has ideas.
>
>  - I'm getting serious spam from fedora-notif on IRC about successful -
>  and a lot more failed - ELN builds for my packages (triggered either
>  by ELN developers or automatically by their CI) 
I hope successful builds aren't too much of a pain; failed builds
should preferably be addressed, ideally by you. Unless it's clearly
not a problem in your package.

>  - bodhi is spamming bugzilla by changing the "Fixed in
> Version" field
>  every time a new ELN build succeds with a new %dist value (one
>  bugzilla email each for eln106, eln107, eln108, etc.) 
This sounds wrong to me but maybe it's relevant for potential ELN
consumers to know if a bug was also addressed in ELN. Is that an
issue?

>  At this point, almost half of the bugzilla mail I get and a majority
>  of the IRC notifications are "ELN spam" that I do not know how to deal
>  with / is not actionable for me. It's now so bad that I'd like to get
>  the packages that are failing to build (multiple times a day) in ELN
>  fixed just so I don't get spammed with "failed to build" notifications
>  anymore ... 
I wonder why you're getting so many build notifications. These should
only be triggered when you update your package in Rawhide. But again,
fixing build failures would generally be a good thing.

P

[Dan Čermák]

Petr Šabata <contyk(a)redhat.com&gt; writes:

>  On Thu, Dec 17, 2020 at 12:27 PM Fabio Valentini
> <decathorpe(a)gmail.com&gt; wrote:
> >
> > Hi everybody,
> >
> > I'm unsure how to deal with this, so I'm asking here if anybody has ideas.
> >
> > - I'm getting serious spam from fedora-notif on IRC about successful -
> > and a lot more failed - ELN builds for my packages (triggered either
> > by ELN developers or automatically by their CI)
>
>  I hope successful builds aren't too much of a pain; failed builds
>  should preferably be addressed, ideally by you. Unless it's clearly
>  not a problem in your package. 
Wasn't there an agreement that I, as an average packager, don't have to
care about ELN at all?


Cheers,

Dan

[Fabio Valentini]

On Thu, Dec 17, 2020 at 12:37 PM Petr Šabata <contyk(a)redhat.com&gt; wrote:
>  On Thu, Dec 17, 2020 at 12:27 PM Fabio Valentini <decathorpe(a)gmail.com&gt; wrote:
>  >
>  > Hi everybody,
>  >
>  > I'm unsure how to deal with this, so I'm asking here if anybody has ideas.
>  >
>  > - I'm getting serious spam from fedora-notif on IRC about successful -
>  > and a lot more failed - ELN builds for my packages (triggered either
>  > by ELN developers or automatically by their CI)
>
>  I hope successful builds aren't too much of a pain; failed builds
>  should preferably be addressed, ideally by you. Unless it's clearly
>  not a problem in your package.
>
>  > - bodhi is spamming bugzilla by changing the "Fixed in Version" field
>  > every time a new ELN build succeds with a new %dist value (one
>  > bugzilla email each for eln106, eln107, eln108, etc.)
>
>  This sounds wrong to me but maybe it's relevant for potential ELN
>  consumers to know if a bug was also addressed in ELN. Is that an
>  issue?
>
>  > At this point, almost half of the bugzilla mail I get and a majority
>  > of the IRC notifications are "ELN spam" that I do not know how to deal
>  > with / is not actionable for me. It's now so bad that I'd like to get
>  > the packages that are failing to build (multiple times a day) in ELN
>  > fixed just so I don't get spammed with "failed to build"
> notifications
>  > anymore ...
>
>  I wonder why you're getting so many build notifications. These should
>  only be triggered when you update your package in Rawhide. But again,
>  fixing build failures would generally be a good thing. 
I maintain 400 (co-maintain 1600) packages and have submitted almost
300 package updates in the past month alone. That's probably the
reason for the amount.

The build failures are certainly not my problem, because the packages
built (and still build) fine in rawhide.

I assume they fail because they're getting built in the wrong order in
ELN (I use side tags for order-sensitive builds), or they're missing
new dependencies that are not in ELN yet.
I can do nothing about either of those things, only wait until the CI
submits the failed builds often enough until they land in the correct
order, which takes n! tries in the "random" order the CI submits the
builds, resulting in a large number of failed koji builds.

Fabio

[Aleksandra Fedorova]

Hi, Fabio,

On Thu, Dec 17, 2020 at 1:22 PM Fabio Valentini <decathorpe(a)gmail.com&gt; wrote:
>  On Thu, Dec 17, 2020 at 12:37 PM Petr Šabata <contyk(a)redhat.com&gt; wrote:
>  >
>  > On Thu, Dec 17, 2020 at 12:27 PM Fabio Valentini <decathorpe(a)gmail.com&gt;
> wrote:
>  > >
>  > > Hi everybody,
>  > >
>  > > I'm unsure how to deal with this, so I'm asking here if anybody has
> ideas.
>  > >
>  > > - I'm getting serious spam from fedora-notif on IRC about successful -
>  > > and a lot more failed - ELN builds for my packages (triggered either
>  > > by ELN developers or automatically by their CI)
>  >
>  > I hope successful builds aren't too much of a pain; failed builds
>  > should preferably be addressed, ideally by you. Unless it's clearly
>  > not a problem in your package.
>  >
>  > > - bodhi is spamming bugzilla by changing the "Fixed in Version"
> field
>  > > every time a new ELN build succeds with a new %dist value (one
>  > > bugzilla email each for eln106, eln107, eln108, etc.)
>  >
>  > This sounds wrong to me but maybe it's relevant for potential ELN
>  > consumers to know if a bug was also addressed in ELN. Is that an
>  > issue?
>  >
>  > > At this point, almost half of the bugzilla mail I get and a majority
>  > > of the IRC notifications are "ELN spam" that I do not know how to
> deal
>  > > with / is not actionable for me. It's now so bad that I'd like to get
>  > > the packages that are failing to build (multiple times a day) in ELN
>  > > fixed just so I don't get spammed with "failed to build"
> notifications
>  > > anymore ...
>  >
>  > I wonder why you're getting so many build notifications. These should
>  > only be triggered when you update your package in Rawhide. But again,
>  > fixing build failures would generally be a good thing.
>
>  I maintain 400 (co-maintain 1600) packages and have submitted almost
>  300 package updates in the past month alone. That's probably the
>  reason for the amount.
>
>  The build failures are certainly not my problem, because the packages
>  built (and still build) fine in rawhide.
>
>  I assume they fail because they're getting built in the wrong order in
>  ELN (I use side tags for order-sensitive builds), or they're missing
>  new dependencies that are not in ELN yet.
>  I can do nothing about either of those things, only wait until the CI
>  submits the failed builds often enough until they land in the correct
>  order, which takes n! tries in the "random" order the CI submits the
>  builds, resulting in a large number of failed koji builds. 
Unfortunately there is no way to manage these notifications from the
ELN SIG side. The ELN automation doesn't send any messages on its own.
(We do file BZ's for ELN FTBFS issues for RHEL developers under RHEL
Product umbrella in bugzilla, but this is done on agreement with RHEL
Engineering)

The notifications you get are sent by Koji and Bodhi.

The issue was reported for Bodhi some time ago[1]

The IRC notifications can be controlled via Notifications App.
Afaik, by default they are configured so that you get notified about
all builds for packages you maintain, no matter the target or owner of
the build. But it can be changed in the app interface.

We welcome contributors to the ELN SIG, but we have no intent to push
any responsibility regarding ELN builds on people who have no interest
in them. It is only the limitation of the current messaging
infrastructure, which creates the inconvenience.

[1] https://github.com/fedora-infra/bodhi/issues/4094
[2] https://apps.fedoraproject.org/notifications


>  Fabio
>  _______________________________________________
>  devel mailing list -- devel(a)lists.fedoraproject.org
>  To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail] 
-- 
Aleksandra Fedorova
bookwar on IRC

[Vít Ondruch]

Dne 17. 12. 20 v 12:37 Petr Šabata napsal(a):
>  On Thu, Dec 17, 2020 at 12:27 PM Fabio Valentini
> <decathorpe(a)gmail.com&gt; wrote:
> > Hi everybody,
> >
> > I'm unsure how to deal with this, so I'm asking here if anybody has ideas.
> >
> > - I'm getting serious spam from fedora-notif on IRC about successful -
> > and a lot more failed - ELN builds for my packages (triggered either
> > by ELN developers or automatically by their CI)
>  I hope successful builds aren't too much of a pain; failed builds
>  should preferably be addressed, ideally by you. Unless it's clearly
>  not a problem in your package. 

Addressing FTBFS is indeed good idea, but what I don't understand is why 
in the times rubygem-shoulda-matchers was broken in Fedora for months, 
ELN tried rebuild every two hours. There is certainly space for 
improvement. Once packages is FTBFS in Fedora, it probably won't start 
magically build in ELN.


Vít

[Tom Hughes]

On 17/12/2020 11:23, Fabio Valentini wrote:

>  - I'm getting serious spam from fedora-notif on IRC about
> successful -
>  and a lot more failed - ELN builds for my packages (triggered either
>  by ELN developers or automatically by their CI) 
I use email notification rather than IRC so I just have an
exim filter rule that drops anything matching:

   $h_x-fedmsg-category is "buildsys" and $h_subject contains ".eln"

Along with a similar rule for modularity builds.

You should be able to do something similar upstream in the
notifications app I guess - the downside of customising things
there is that you stop getting any changes to the default rules.

>  - bodhi is spamming bugzilla by changing the "Fixed in
> Version" field
>  every time a new ELN build succeds with a new %dist value (one
>  bugzilla email each for eln106, eln107, eln108, etc.) 
I haven't noticed this one happening.

Tom

-- 
Tom Hughes (tom(a)compton.nu)
http://compton.nu/