Software Factory 3.8 upgrade fedora.softwarefactory-project.io: 2023-02-08 and 2023-02-09 from 14:00 to 18:00 UTC

[Nicolas Hicher]

Hello folks,

We plan to perform an upgrade of Software Factory deployments with 
version 3.8.

We will upgrade softwarefactory-project.io on 2023-02-08 from 14:00 to 
18:00 UTC and plan to upgrade fedora.softwarefactory-project.io on 
2023-02-09  from 14:00 to 18:00 UTC

For zuul services are hosted on softwarefactory-project.io, 
fedora.softwarefactory-project.io  will feature Zuul [1] and Nodepool 
[2] last versions Wednesday.

The affected services will be:
- Zuul CI not running jobs for pagure.io or src.fedoraproject.org.

Regards,
Nicolas, on behalf of the Software Factory Operation Team

[1] https://zuul-ci.org/docs/zuul/latest/releasenotes.html#relnotes-8-1-0
[2] 
https://zuul-ci.org/docs/nodepool/latest/releasenotes.html#relnotes-8-1-0

[Nicolas Hicher]

Hello,

Unfortunately We couldnot finish zuul'supgrade today for we have issues 
restartingthe zuul-schedulerservice.We are hitting a lot of 
configuration errors due to some incompatibilities between the previous 
version of Zuul we were running, and version 8.1 that the upgrade is 
deploying.Some tenants are loaded but we have to force push updatesin 
some config repositories for that. We have issueswith several 
configuration errorsin the review.rdo zuul tenant.It doesn't help either 
that we're also hitting rate limits with the github API. We are 
confident most issues will be resolved in due time.

Sincemost of the team islocatedin Europe, we will continue the upgrade 
tomorrow morning.

Regards,
Nicolas, on behalf of the Software Factory Operation Team

On 2/6/23 11:09, Nicolas Hicher wrote:
> Hello folks,
>
> We plan to perform an upgrade of Software Factory deployments with 
> version 3.8.
>
> We will upgrade softwarefactory-project.io on 2023-02-08 from 14:00 to 
> 18:00 UTC and plan to upgrade fedora.softwarefactory-project.io on 
> 2023-02-09  from 14:00 to 18:00 UTC
>
> For zuul services are hosted on softwarefactory-project.io, 
> fedora.softwarefactory-project.io  will feature Zuul [1] and Nodepool 
> [2] last versions Wednesday.
>
> The affected services will be:
> Zuul CI not running jobs for pagure.io or src.fedoraproject.org.
>
> Regards,
> Nicolas, on behalf of the Software Factory Operation Team
>
> [1] https://zuul-ci.org/docs/zuul/latest/releasenotes.html#relnotes-8-1-0
> [2] 
> https://zuul-ci.org/docs/nodepool/latest/releasenotes.html#relnotes-8-1-0

[Nicolas Hicher]

Hello folks,

We finalized zuul and nodepool upgrade today and now everything should 
works as expected.

We didn't have time to manage the sf.3.8 upgrade for 
fedora.softwarefactory-project.io today, we will do the upgrade next 
Tuesday from 14:00 to 18:00 UTC

Regards,
Nicolas, on behalf of the Software Factory Operation Team

On 2/8/23 16:35, Nicolas Hicher wrote:
> Hello,
>
> Unfortunately We couldnot finish zuul'supgrade today for we have 
> issues restartingthe zuul-schedulerservice.We are hitting a lot of 
> configuration errors due to some incompatibilities between the 
> previous version of Zuul we were running, and version 8.1 that the 
> upgrade is deploying.Some tenants are loaded but we have to force push 
> updatesin some config repositories for that. We have issueswith 
> several configuration errorsin the review.rdo zuul tenant.It doesn't 
> help either that we're also hitting rate limits with the github API. 
> We are confident most issues will be resolved in due time.
>
> Sincemost of the team islocatedin Europe, we will continue the upgrade 
> tomorrow morning.
>
> Regards,
> Nicolas, on behalf of the Software Factory Operation Team
> On 2/6/23 11:09, Nicolas Hicher wrote:
> > Hello folks,
> >
> > We plan to perform an upgrade of Software Factory deployments with 
> > version 3.8.
> >
> > We will upgrade softwarefactory-project.io on 2023-02-08 from 14:00 
> > to 18:00 UTC and plan to upgrade fedora.softwarefactory-project.io on 
> > 2023-02-09  from 14:00 to 18:00 UTC
> >
> > For zuul services are hosted on softwarefactory-project.io, 
> > fedora.softwarefactory-project.io  will feature Zuul [1] and Nodepool 
> > [2] last versions Wednesday.
> >
> > The affected services will be:
> > Zuul CI not running jobs for pagure.io or src.fedoraproject.org.
> >
> > Regards,
> > Nicolas, on behalf of the Software Factory Operation Team
> >
> > [1] 
> > https://zuul-ci.org/docs/zuul/latest/releasenotes.html#relnotes-8-1-0
> > [2] 
> > https://zuul-ci.org/docs/nodepool/latest/releasenotes.html#relnotes-8-1-0