Feedback on secondary architecute promotion requirements draft

[Josh Boyer]

On Tue, Apr 3, 2012 at 2:58 PM, Brendan Conoboy <blc at redhat.com> wrote:
> On 04/03/2012 09:07 AM, Josh Boyer wrote:
>>
>>  From a koji perspective, there really isn't much benefit to step 2.
>> What needs to happen is the RPMs from the secondary hub need to be
>> copied to the primary in the correct NVR directories in the hub's
>> storage.  That can happen in the background for quite a bit, but at
>> some point the hub would need to be taken offline to sync up the last
>> few builds, and then switch the builders over.
>>
>> Having a staging hub just means you have to copy and move the builders
>> twice.  This is mostly due to how koji builders can only talk to one
>> hub at a time and one hub only.
>
>
> Where do you envision the builders being in this scenario?  I see the steps
> being something like this:
>
> 1. SA builders and/or hub are located outside PHX.
> 2 option a. Builders come up in PHX, hub stays in original location.
> 2 option b. Staging hub comes up in PHX, builders stay in original location.
> 3. Both staging hub and builders come up in PHX
> 4. When appropriate move from staging hub to primary hub.


I hope when you refer to "staging hub" you mean "the hub being used
as the SA hub.

> Having everything take place in PHX prior to the switchover has numerous
> benefits.  These 3 come to mind immediately:
>
> 1. Fast local network will represent true build times (vs transfering rpms
> across the external network).

Yes.  Which is why you ship the SA hub to PHX.

I'm not saying this is a _requirement_, but it is the most expedient
option.  Otherwise you have a hell of a lot of data to get to PHX
anyway.

> 2. Realistic load assessment.  If, hypothetically primary koji and staging
> koji are both virtual machines on the same underlying hardware you'll know
> if the hardware can handle the load.  Also network, disk, etc.
> 3. Comparable infrastructure reliability.
>
> Switching koji hubs twice does incur a bit more work, but it may also
> provide better results.

I don't see how.  The hub characteristics are the least of the worries
in this entire scenario.  The builders are going to dominate the time
spent, and you aren't going to get exactly comparable statistics by
using a "staging" hub on identical hardware/VM config because that
staging hub isn't going to be building both PA and the soon-to-be PA
SA.

Really, staging hubs seem like a waste of time to me.

josh