OOM killer on bapp02 again

Pierre-Yves Chibon pingou at pingoured.fr
Mon Mar 16 12:38:41 UTC 2015


On Mon, Mar 16, 2015 at 06:31:17AM -0600, Kevin Fenzi wrote:
> On Mon, 16 Mar 2015 10:08:28 +0100
> Adrian Reber <adrian at lisas.de> wrote:
> 
> > On Fri, Mar 13, 2015 at 07:16:44AM -0600, Kevin Fenzi wrote:
> > > We could do this... however at this point, I think I'd like to just
> > > push to get mm2 out in production. I'm sure it will have issues too,
> > > but at least they will not be this one hopefully. ;) 
> > 
> > So what is actually necessary to get MM2 out? As I am probably one of
> > the few users of the MM2 administrative functions maybe I can help to
> > get the last missing bits of MM2 ready.
> 
> That would be great. ;) 
> 
> I would really love to get MM2 fully rolled out before Beta freeze if
> we can. 
> 
> For the mirrorlist servers: 
> 
> * We need a /etc/tmpfiles.d/mirrormanager2-mirrorlist.conf file with: 
> d /var/run/mirrormanager 0775 mirrormanager apache
> in it. 

I need to fix the spec file for this

> * Then I think we can convert all the mirrorlists over to it. The one
>   we have had in rotation has been just fine. In fact it's had a lot
>   less issues than the mm1 ones. ;) 
> 
> For the frontend:
> 
> * We have a stg setup: 
> https://admin.stg.fedoraproject.org/mirrormanager2/

Note: I had to restart apache on it this morning, I didn't see anything in the
logs but as I was away last week I don't if that was expected or not.

> Can you login and look around and see if the data/setup seems ok from a
> high level glance?

One point where you might be of great help is with testing the xml-rpc endpoint.
It should be 100% backward compatible (but I let you guess what the keyword is
in that sentence :)).


> For the backend: 
> 
> * I think we need some more testing, but I am not at all sure how to do
>   this. If we do switch over to it, we could have some time to fix
>   things on it, as discovering updates and new rawhide composes and
>   such isn't super time critical, but we would have to make sure it
>   works in a day or so. 
> 
> * We need to finish fedmsg stuff on it. It's still sending some crons
>   where it's erroring on fedmsg items. We need to sort that out before
>   rolling to production. 
> 
> For the crawler: 
> 
> * We have a staging instance, but I am not sure what state it's in. 
> Perhaps we could check this and make it crawl for a day as a test? 

In theory it should crawl, in practice I didn't check if it does so correctly.

> For the big picture: 
> 
> * We can switch mirrorlists anytime. 
> * For frontend/backend/crawler, we should make production versions of
>   them and then look at scheduling a day when many of us can be around
>   to debug things, switch over to using them and in the event of big
>   problems revert back to mm1. 

The DB schema changed a little but not enough that going forth and back would be
a big problem imho (and we could also copy the mirrormanager db into a
mirrormanger2, making the rollback even easier).

Pierre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://lists.fedoraproject.org/pipermail/infrastructure/attachments/20150316/2defe65c/attachment-0001.sig>


More information about the infrastructure mailing list