Minor Copr release 2020-06-19

[Pavel Raiskup]

Hey all,

a new versions of copr packages were deployed now to production.  There are
several small bugfixes (most of them were already hot-fix patched in production
before), but there are also two major things worth mentioning:

1. The build task scheduler (priority counting) was fixed once more, now
   for real, I believe.  When you submit a background job, it _will_ be
   processed later than your normal builds (or at the same time, if there's
   capacity, there are no synchronization mechanisms, yet).  Also source builds
   have precedence over binary RPM builds again (see issues #1414 and #1415
   for more info).

2. Newly, we don't always run createrepo_c after each build (or delete request).
   Iff there are a concurrent createrepo_c requests on the same directory
   (multiple builds finished at a similar time in the same project chroot) we will
   likely group them into a batch and execute the createrepo_c only once for the
   whole batch.  This saves some resources in general, but the major speedup
   will be in large projects (where createrepo_c used to block us historically).

Especially point 2 was not really trivial.  In case you experience any bugs,
please report it as usual (and as always, you can regenerate the repository
manually from the web-UI when necessary).

Also, today's Fedora outage [1] allowed me to fix one small problem (I got a
good reproducer) in our VM allocation mechanism, so the Copr's build performance
shouldn't degrade over time again as it happened several times during the past
few days.

[1] https://pagure.io/fedora-infrastructure/issue/9051

Happy building!
Pavel

[Iñaki Ucar]

Thanks for the update! We will test all this thoroughly. ;-)

El sáb., 20 jun. 2020 0:29, Pavel Raiskup  escribió:
> Hey all,
>
> a new versions of copr packages were deployed now to production.  There are
> several small bugfixes (most of them were already hot-fix patched in
> production
> before), but there are also two major things worth mentioning:
> The build task scheduler (priority counting) was fixed once more, now
> for real, I believe.  When you submit a background job, it _will_ be
> processed later than your normal builds (or at the same time, if there's
> capacity, there are no synchronization mechanisms, yet).  Also source
>
> builds
>    have precedence over binary RPM builds again (see issues #1414 and #1415
>    for more info).
> Newly, we don't always run createrepo_c after each build (or delete
>
> request).
>    Iff there are a concurrent createrepo_c requests on the same directory
>    (multiple builds finished at a similar time in the same project chroot)
> we will
>    likely group them into a batch and execute the createrepo_c only once
> for the
>    whole batch.  This saves some resources in general, but the major
> speedup
>    will be in large projects (where createrepo_c used to block us
> historically).
>
> Especially point 2 was not really trivial.  In case you experience any
> bugs,
> please report it as usual (and as always, you can regenerate the repository
> manually from the web-UI when necessary).
>
> Also, today's Fedora outage [1] allowed me to fix one small problem (I got
> a
> good reproducer) in our VM allocation mechanism, so the Copr's build
> performance
> shouldn't degrade over time again as it happened several times during the
> past
> few days.
>
> [1] https://pagure.io/fedora-infrastructure/issue/9051
>
> Happy building!
> Pavel
>
> copr-devel mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedorahosted.org/archives/list/[e-mail].o...

[Miro Hrončok]

On 20. 06. 20 0:29, Pavel Raiskup wrote:
> Newly, we don't always run createrepo_c after each build (or delete request).
>  Iff there are a concurrent createrepo_c requests on the same directory
>  (multiple builds finished at a similar time in the same project chroot) we will
>  likely group them into a batch and execute the createrepo_c only once for the
>  whole batch.  This saves some resources in general, but the major speedup
>  will be in large projects (where createrepo_c used to block us historically).

Thank you so much for this.

[Iñaki Ucar]

On Sat, 20 Jun 2020 at 00:29, Pavel Raiskup  wrote:
> Hey all,
>
> a new versions of copr packages were deployed now to production.  There are
> several small bugfixes (most of them were already hot-fix patched in production
> before), but there are also two major things worth mentioning:
>
> The build task scheduler (priority counting) was fixed once more, now
> for real, I believe.  When you submit a background job, it _will_ be
> processed later than your normal builds (or at the same time, if there's
> capacity, there are no synchronization mechanisms, yet).  Also source builds
> have precedence over binary RPM builds again (see issues #1414 and #1415
> for more info).
>
> Newly, we don't always run createrepo_c after each build (or delete request).
> Iff there are a concurrent createrepo_c requests on the same directory
> (multiple builds finished at a similar time in the same project chroot) we will
> likely group them into a batch and execute the createrepo_c only once for the
> whole batch.  This saves some resources in general, but the major speedup
> will be in large projects (where createrepo_c used to block us historically).

Screenshot of the batch I sent yesterday. Thanks to 1), the number of
pending tasks steadily grows very fast, and thanks to 2) builds are
processed much faster than before. Great work!