[Fedora-infrastructure-list] Hardware Trouble (?) on app1.fedora.phx.redhat.com

[Luke Macken]

On Sun, Sep 10, 2006 at 08:49:13PM -0400, Jeffrey Tadlock wrote:
> Warren Togami wrote:
> > Both before and after the data center migration to a new rack and new
> > switch, we have occasionally been experiencing network trouble to
> > app1.fedora.phx.redhat.com.
> >
> > Since this happened both before and after the new switch, could this
> > perhaps be hardware trouble?
> >
> > Any opinions of what we should do about this?  Perhaps...
> >
> > - More closely monitor, with ping logs over time?
> 
> Closer monitoring would probably be good.  Seeing what is happening from 
> the console when these unusual events occur might also provide some 
> insight as to what is really happening.  From the IRC log it looks like 
> lmacken was able to produce some "oddities" with an nmap scan of app1 
> and trying to flush iptables?

I have experienced this issue with a couple of other machines in the
colo, although I cannot remember exactly which ones (proxy[1-4] iirc).

I have been successful in reproducing this issue by just `nmap app1`
from bastion, and also `sudo /sbin/iptables -F` on app1 itself.

I have yet to try these against app1 without any iptables chains loaded
(since I cannot flush them).  I'm still trying to get the Dell Remote
Console switch stuff working so I can get into the KVM and flush them,
but I haven't had much time to play with it.

> The firewalls on these boxes have been a little unusual to say the least 
> in the time I have worked on these boxes.  Maybe just getting some of 
> the Pyroman configs rolled out will clear some of this up.

I'm ready to deploy pyroman on the app servers at any time, but I have
been hesitating because I would like a safetynet just in case something
explodes (either through the KVM or the cyclade consoles (anyone have
the new information on this?)).

luke