koji buildsystem changes

[Kevin Fenzi]

Greetings everyone. I thought I would share some recent changes to
koji (The Fedora buildsystem):

* All s390x, x86_64, and ppc64le builders have been upgraded to Fedora33

* aarch64 and armv7 builder should be upgraded soon. 

* I have enabled bootstrap mode for all builds. In this mode koji uses
the builder tools to construct a 'bootstrap' repo from the target os
tools, which it then uses to create the src.rpm and do the build. 

For now this is needed for epel7/8 builds after builders moved to f33
due to rpm changing to default to sqlite rpm db. It will likely be
needed for many other things moving forward and allows easier landing of
new features while keeping other branches working as expected.

This has caused one common issue: rhel7 rpm does not understand the
newer changelog format with timezone, so spec files merged from fedora
will break. 

* We have been having some koji database issues nightly when the koji-gc
(garbage collection) scripts run (8-10UTC). We are working hard to track
this down and fix it. Appologies for slowdowns in the mean time.
Progress on this can be followed in 
https://pagure.io/fedora-infrastructure/issue/9528

* Finally releng is looking at establishing a sidetag cleanup policy. 
A reminder that sidetags should be short lived and only created when
needed. koji must generate buildroot repos for every single sidetag.
( You can list all your sidetags with 'fedpkg list-side-tags --mine' )

As a first cut we are going to delete sidetags that have been
empty for more than 14 days. We are considering a upper limit for 
time for sidetags as well. Input welcome to devel list or 
https://pagure.io/releng/issue/9805

Thanks everyone, stay safe out there!

kevin