Failed builds for no apparent reasons

[Miro Hrončok]

Hey,

I have coupe dozens (or hundreds?) builds like this:

https://copr.fedorainfracloud.org/coprs/g/python/python3.8/build/936777/

Any idea what went wrong?

[Pavel Raiskup]

On Tuesday, June 18, 2019 12:05:58 AM CEST Miro Hrončok wrote:
> Hey,
>
> I have coupe dozens (or hundreds?) builds like this:
>
> https://copr.fedorainfracloud.org/coprs/g/python/python3.8/build/936777/
>
> Any idea what went wrong?

Thanks for the report.  Long story short, there were IO problems on
copr-dist-git machine.  Sorry for inconvenience.

Longer sum up:
This was caused by rsync started yesterday to move the dist-git data to new
storage.  We have one script (related to cgit,
/var/lib/copr-dist-git/cgit_pkg_list) which is suffering from IO performance for
some time, but it was not noticeable before (rsync made the problems visible).
Plus there's https://pagure.io/copr/copr/issue/807 which multiplies those two
above.

Pavel

[Miro Hrončok]

On 18. 06. 19 9:43, Pavel Raiskup wrote:
> On Tuesday, June 18, 2019 12:05:58 AM CEST Miro Hrončok wrote:
> > Hey,
> >
> > I have coupe dozens (or hundreds?) builds like this:
> >
> > https://copr.fedorainfracloud.org/coprs/g/python/python3.8/build/936777/
> >
> > Any idea what went wrong?
>
> Thanks for the report.  Long story short, there were IO problems on
> copr-dist-git machine.  Sorry for inconvenience.
>
> Longer sum up:
> This was caused by rsync started yesterday to move the dist-git data to new
> storage.  We have one script (related to cgit,
> /var/lib/copr-dist-git/cgit_pkg_list) which is suffering from IO performance for
> some time, but it was not noticeable before (rsync made the problems visible).
> Plus there's https://pagure.io/copr/copr/issue/807 which multiplies those two
> above.

Thanks. From the past tense I assume this has been fixed?

[Pavel Raiskup]

On Tuesday, June 18, 2019 10:07:19 AM CEST Miro Hrončok wrote:
> On 18. 06. 19 9:43, Pavel Raiskup wrote:
> > On Tuesday, June 18, 2019 12:05:58 AM CEST Miro Hrončok wrote:
> > > Hey,
> > >
> > > I have coupe dozens (or hundreds?) builds like this:
> > >
> > > https://copr.fedorainfracloud.org/coprs/g/python/python3.8/build/936777/
> > >
> > > Any idea what went wrong?
> >
> > Thanks for the report.  Long story short, there were IO problems on
> > copr-dist-git machine.  Sorry for inconvenience.
> >
> > Longer sum up:
> > This was caused by rsync started yesterday to move the dist-git data to new
> > storage.  We have one script (related to cgit,
> > /var/lib/copr-dist-git/cgit_pkg_list) which is suffering from IO performance for
> > some time, but it was not noticeable before (rsync made the problems visible).
> > Plus there's https://pagure.io/copr/copr/issue/807 which multiplies those two
> > above.
>
> Thanks. From the past tense I assume this has been fixed?

I disabled the cgit_pkg_list for now, so yes.  Please report a bug if you see
the same problem again.

Pavel

[Miro Hrončok]

On 18. 06. 19 10:09, Pavel Raiskup wrote:
> On Tuesday, June 18, 2019 10:07:19 AM CEST Miro Hrončok wrote:
> > On 18. 06. 19 9:43, Pavel Raiskup wrote:
> > > On Tuesday, June 18, 2019 12:05:58 AM CEST Miro Hrončok wrote:
> > > > Hey,
> > > >
> > > > I have coupe dozens (or hundreds?) builds like this:
> > > >
> > > > https://copr.fedorainfracloud.org/coprs/g/python/python3.8/build/936777/
> > > >
> > > > Any idea what went wrong?
> > >
> > > Thanks for the report.  Long story short, there were IO problems on
> > > copr-dist-git machine.  Sorry for inconvenience.
> > >
> > > Longer sum up:
> > > This was caused by rsync started yesterday to move the dist-git data to new
> > > storage.  We have one script (related to cgit,
> > > /var/lib/copr-dist-git/cgit_pkg_list) which is suffering from IO performance for
> > > some time, but it was not noticeable before (rsync made the problems visible).
> > > Plus there's https://pagure.io/copr/copr/issue/807 which multiplies those two
> > > above.
> >
> > Thanks. From the past tense I assume this has been fixed?
>
> I disabled the cgit_pkg_list for now, so yes.  Please report a bug if you see
> the same problem again.

Thank you!