koji / bodhi issues status update

[Kevin Fenzi]

As some of you may know, we have been having issues with koji and bodhi
over the holidays. :( koji would sometimes not tag builds or error them
with odd error messages and bodhi wasn't pushing updates. 

I'm happy to report that the underlying issue seems to be fixed now. 
We hopefully will have a more detailed root cause next week.

However, due to the koji issues builds were in a state where bodhi
ejected many of them from updates pushes. We are working on cleaning
up the state of those updates and there's no need for maintainers to do
anything with those updates at this time.

Once we get the ejected builds cleaned up we should be able to resume
normal bodhi updates pushes. 

Thanks everyone for your patience on this issue!

kevin

[Clement Verna]

On Fri, 3 Jan 2020 at 22:41, Kevin Fenzi <kevin(a)scrye.com&gt; wrote:

>  As some of you may know, we have been having issues with koji and
> bodhi
>  over the holidays. :( koji would sometimes not tag builds or error them
>  with odd error messages and bodhi wasn't pushing updates.
>
>  I'm happy to report that the underlying issue seems to be fixed now.
>  We hopefully will have a more detailed root cause next week.
>
>  However, due to the koji issues builds were in a state where bodhi
>  ejected many of them from updates pushes. We are working on cleaning
>  up the state of those updates and there's no need for maintainers to do
>  anything with those updates at this time.
>
>  Once we get the ejected builds cleaned up we should be able to resume
>  normal bodhi updates pushes.
>  
Hi all,

I think that we now have dealt with most of the ejected updates. It seems
that there are still a few rawhide updates stuck in pending so I ll be
looking at unblocking these.

Please let us know if you believed that we have missed something.

Thanks


>  Thanks everyone for your patience on this issue!
>
>  kevin
>  _______________________________________________
>  devel-announce mailing list -- devel-announce(a)lists.fedoraproject.org
>  To unsubscribe send an email to
>  devel-announce-leave(a)lists.fedoraproject.org
>  Fedora Code of Conduct:
>  https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>  List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>  List Archives:
>  https://lists.fedoraproject.org/archives/list/[e-mail]...
>

[Kevin Fenzi]

On Sun, Jan 05, 2020 at 05:27:19PM +0100, Clement Verna wrote:
>  On Fri, 3 Jan 2020 at 22:41, Kevin Fenzi <kevin(a)scrye.com&gt;
> wrote:
>  
>  > As some of you may know, we have been having issues with koji and bodhi
>  > over the holidays. :( koji would sometimes not tag builds or error them
>  > with odd error messages and bodhi wasn't pushing updates.
>  >
>  > I'm happy to report that the underlying issue seems to be fixed now.
>  > We hopefully will have a more detailed root cause next week.
>  >
>  > However, due to the koji issues builds were in a state where bodhi
>  > ejected many of them from updates pushes. We are working on cleaning
>  > up the state of those updates and there's no need for maintainers to do
>  > anything with those updates at this time.
>  >
>  > Once we get the ejected builds cleaned up we should be able to resume
>  > normal bodhi updates pushes.
>  >
>  
>  Hi all,
>  
>  I think that we now have dealt with most of the ejected updates. It seems
>  that there are still a few rawhide updates stuck in pending so I ll be
>  looking at unblocking these.
>  
>  Please let us know if you believed that we have missed something. 
Additionally there were a number of builds in koji in "BUILDING" state,
even though they had failed/had no active tasks building them. 

I went ahead and canceled them all, so now maintainers should be able to
resubmit or just do a new build. List of those by maintainer is: 

cabal-rpm-1.0.3-1.fc32                                   petersen          BUILDING
cobbler-2.8.5-1.el7                                      orion             BUILDING
desktopfolder-1.1.2-1.fc31                               atim              BUILDING
efl-1.23.1-1.fc32                                        spot              BUILDING
fleet-commander-admin-0.15.0-1.fc30                      ogutierrez        BUILDING
gstreamer1-plugins-bad-free-1.16.2-1.fc32                wtaymans          BUILDING
gstreamer1-plugins-base-1.16.2-2.fc31                    wtaymans          BUILDING
heimdal-7.7.0-2.epel8.playground                         abo               BUILDING
im-chooser-1.7.3-1.epel8.playground                      tagoh             BUILDING
imsettings-1.8.2-1.fc32                                  tagoh             BUILDING
js8call-2.1.1-1.fc32                                     hobbes1069        BUILDING
libbpf-0.0.6-1.fc32                                      jolsa             BUILDING
libwacom-1.2-2.fc32                                      whot              BUILDING
mellowplayer-3.5.8-1.20191227git9fd6cee.fc32             martinkg          BUILDING
module-build-macros-0.1-1.module_f32+7264+508b1e07       mbs/mbs.fedoraproject.org 
BUILDING
module-build-macros-0.1-1.module_f32+7272+cab9d0cd       mbs/mbs.fedoraproject.org 
BUILDING
musique-1.7-1.fc32                                       lbazan            BUILDING
mypaint-2.0.0-0.4.beta.0.fc32                            avsej             BUILDING
osc-source_validator-0.19-2.fc31                         ngompa            BUILDING
pdf-stapler-1.0.0-1.fc32                                 aarem             BUILDING
perl-Config-Model-2.138-1.fc32                           eseyman           BUILDING
python-avocado-74.0-1.module_f31+7253+5196ffdf           mbs/mbs.fedoraproject.org 
BUILDING
python-chaospy-3.0.17-1.fc30                             lbazan            BUILDING
python-django-threadedcomments-1.2-8.fc30                lbazan            BUILDING
python-lz4-3.0.2-1.fc30                                  jgu               BUILDING
python-lz4-3.0.2-1.fc31                                  jgu               BUILDING
python-lz4-3.0.2-1.fc32                                  jgu               BUILDING
python-mypy_extensions-0.4.1-5.fc32                      ignatenkobrain    BUILDING
python-paho-mqtt-1.5.0-2.fc31                            fab               BUILDING
python-pycares-3.1.0-fix3.1.fc31                         fantom            BUILDING
python-pyelectro-0.1.10-1.fc32                           major             BUILDING
python-versioneer-0.18-2.fc32                            nonamedotc        BUILDING
rubygem-pg-1.2.0-1.fc32                                  jaruga            BUILDING
ucblogo-6.1-1.fc31                                       jrincayc          BUILDING
vertica-python-0.10.1-1.el7                              kubo              BUILDING

I don't want to blindly resubmit in case folks already bumped release
and make newer builds. If thats the case, you have nothing to do here. 

kevin