Server Updates and Reboots

[Mark O'Brien]

Greetings all,

Apologies for the late notice on this message. I previously sent it to the
infrastructure list and should have sent it here at the same time.

We will be doing a mass update/reboot this week. We haven't done
one in a couple of months and would like to get all up to date before the
upcoming beta freeze.

Rough plan:

* Monday (2022-02-14): *.stg.iad2.fedoraproject.org. update/reboot all
staging hosts. We need to watch for the openshift clusters and make sure
they come back up correctly.

* Tuesday (2022-02-15): any non outage causing hosts. This would include
proxies and their virthosts since we can take them out of dns. Also
might include some virthosts in iad2 as long as all the guests on them
have counterparts on other virthosts to take the load.

* *Wed (2022-02-16): Outage starting at 21UTC.*
Everything else left that we haven't done yet we do here.
Outage ticket: https://pagure.io/fedora-infrastructure/issue/10536

This is mostly on Kevin and I, since it needs sysadmin-main, but our
SOP/doc on this is horribly out of date if someone(s) would like to work
on updating it. (
https://docs.fedoraproject.org/en-US/infra/sysadmin_guide/massupgrade/ )
Also there's talking to users during the outage and testing things after we
are done.

Thanks,

Mark