batcave01 move to rhel9 - 2023-07-13 21UTC

[Kevin Fenzi]

Hey folks. I keep putting it off, but I need to move it forward, so I am
proposing to move batcave01 to rhel9 tomorrow, starting at 21UTC.

My plan is roughly:

- sync data from batcave01 to batcave02
- at 21UTC take down services on batcave01
- sync data again
- take batcave01 down.
- rename batcave02 to batcave01 and re-ip it to the right IP
- reboot it back up as batcave01
- Fix any issues that come up.

This means that the ssh host key for batcave01 WILL CHANGE!
If you are using https://admin.fedoraproject.org/ssh_known_hosts
in your ~/.ssh/known_hosts it shouldn't matter. If you aren't you may
have to accept the new changed host key.

For the time when I am switching things, EPEL builds may fail.
This is because koji pulls RHEL packages from batcave01.
I would expect this to be a very short window.

The playbook completes fine on batcave02 and as far as I can see things
should work fine there, but there may be some broken items.
Please file tickets/let me know if you find anything like that.

In the event that things go very poorly for some reason, I can always
bring the old vm back up and we can revert to it.

Thanks,

kevin

[Stephen Smoogen]

On Wed, 12 Jul 2023 at 12:40, Kevin Fenzi  wrote:
> Hey folks. I keep putting it off, but I need to move it forward, so I am
> proposing to move batcave01 to rhel9 tomorrow, starting at 21UTC.
>
> My plan is roughly:

reviewed and +1. [I am not sure I will be awake enough to help out at
2100UTC so I am trying to help now.]
> sync data from batcave01 to batcave02
> at 21UTC take down services on batcave01
> sync data again
> take batcave01 down.
> rename batcave02 to batcave01 and re-ip it to the right IP
> reboot it back up as batcave01
> Fix any issues that come up.
>
> This means that the ssh host key for batcave01 WILL CHANGE!
> If you are using https://admin.fedoraproject.org/ssh_known_hosts
> in your ~/.ssh/known_hosts it shouldn't matter. If you aren't you may
> have to accept the new changed host key.
>
> For the time when I am switching things, EPEL builds may fail.
> This is because koji pulls RHEL packages from batcave01.
> I would expect this to be a very short window.
>
> The playbook completes fine on batcave02 and as far as I can see things
> should work fine there, but there may be some broken items.
> Please file tickets/let me know if you find anything like that.
>
> In the event that things go very poorly for some reason, I can always
> bring the old vm back up and we can revert to it.
>
> Thanks,
>
> kevin
> _______________________________________________
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to
> [e-mail]
> Fedora Code of Conduct:
> https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
> https://lists.fedoraproject.org/archives/list/[e-mail]...
> Do not reply to spam, report it:
> https://pagure.io/fedora-infrastructure/new_issue

[Michal Konecny]

+1, thanks for doing this

On 12. 07. 23 18:39, Kevin Fenzi wrote:
> Hey folks. I keep putting it off, but I need to move it forward, so I am
> proposing to move batcave01 to rhel9 tomorrow, starting at 21UTC.
>
> My plan is roughly:
> sync data from batcave01 to batcave02
> at 21UTC take down services on batcave01
> sync data again
> take batcave01 down.
> rename batcave02 to batcave01 and re-ip it to the right IP
> reboot it back up as batcave01
> Fix any issues that come up.
>
> This means that the ssh host key for batcave01 WILL CHANGE!
> If you are using https://admin.fedoraproject.org/ssh_known_hosts
> in your ~/.ssh/known_hosts it shouldn't matter. If you aren't you may
> have to accept the new changed host key.
>
> For the time when I am switching things, EPEL builds may fail.
> This is because koji pulls RHEL packages from batcave01.
> I would expect this to be a very short window.
>
> The playbook completes fine on batcave02 and as far as I can see things
> should work fine there, but there may be some broken items.
> Please file tickets/let me know if you find anything like that.
>
> In the event that things go very poorly for some reason, I can always
> bring the old vm back up and we can revert to it.
>
> Thanks,
>
> kevin
>
> infrastructure mailing list -- [e-mail]
> To unsubscribe send an email to [e-mail]
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedoraproject.org/archives/list/[e-mail]...
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue

[Kevin Fenzi]

This is now done and I think everything is working.

I am sure I missed some things, so do file tickets/let us know when you
see things that are broken. ;)

kevin

[Aurelien Bompard]

> This is now done and I think everything is working.

Congrats!