serverbeach hardware refresh plans / hosted VFAD
skvidal at fedoraproject.org
Fri Nov 11 14:55:58 UTC 2011
On Thu, 10 Nov 2011 14:45:16 -0700
Kevin Fenzi <kevin at scrye.com> wrote:
> I talked about this at todays meeting, but I thought I would note it
> here for further input and also see if there was interest in doing a
> Virtual Fedora Activity Day next week to finish out hosted plans.
> We have 5 servers at serverbeach (Thanks for donating them SB Folks!)
> They were setup long ago and are getting a bit old in the tooth, so we
> are getting 5 new machines to migrate our stuff to. This will allow us
> to be on hardware thats newer/better supported and also move to new OS
> and re-arrange things. We must migrate to those new machines by
> 2011-12-11. (around a month from now).
> serverbeach01 -> a mirror
> serverbeach02 -> collab01 (all our lists.fedoraproject.org lists).
> serverbeach03 -> collab02 (a warm spare of collab01, it rsyncs from
> there every hour).
> serverbeach04 -> hosted01 (all fedorahosted.org)
> serverbeach05 -> hosted02 (a warm spare, but currently down and not
> serverbeach06 -> collab03
> serverbeach07 -> collab04
> (move these to rhel6 for both host/guest. Also, look at setting up
> drbd for /srv on them, so we can easily switch over from 03 to 04 in
> case of failure. If we do do drbd, I'd like to schedule at least one
> failover to make sure it really is all working as expected. ns01 can
> go on either. ) We will also need to add drbd rpms to our repo.
> That leaves us with hosted to deal with, and serverbeach08/09/10 to do
> it in (currently). We have discussed lots of plans with hosted, I'd
> like to propose meeting up on IRC next Wed to do a virtual Fedora
> Activity Day around planning that. Or we could just try and work out
> details here on list.
> Things we have discussed:
> * Moving lists.fedorahosted.org to collab*
> * Moving projects to a CNAME or aliased setup so we can then move them
> to whatever backend host we like. (what exact changes?)
> * Making whatever changes we need now so we could move to spinning out
> some projects into "the cloud" instances. What are those specific
> * Backups of hosted01 take ~6 hours, which is not very good. ;( (thats
> incrementals, full backups are much slower... days).
> * Hosted01 is pretty heavily loaded. Some of which may just disappear
> when we move to rhel6 (I know some of it will).
> * Ideally we could bring up projects on a new instance in the event of
> a failure of a node.
> I'd like to have a SOP with steps for migrating hosted instances soon.
1. I think we need some volunteer projects. Do we know of any
enterprising folks who would like to be suckers.... err test subjects?
2. For collab2 and hosted2 - would it be possible to run both of
those secondary machines on one of the new sb boxes? Disk-space wise we
should be able to make it work - and it from a failover standpoint it
feels unlikely we would need to failover both hosted1 and collab1 at
the same time.
3. Then do backups from collab2 and hosted2 - until we have our stuff
together for hosted-tng.
> So, we can say, 'here's what we are going to change and why'. Then we
> can get setup and start moving projects that opt-in, then have a flag
> day before our migration window to just move everyone else.
I suspect we'll need to waddle them all through a little at a time.
Remember getting rid of cvs.fedoraproject.org? like that :(
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 190 bytes
Desc: not available
Url : http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20111111/5f86ba1d/attachment.bin
More information about the infrastructure