System Crash Troubleshooting

Tim Alberts talberts at msiscales.com
Mon Feb 27 23:17:22 UTC 2006


I have a problem with system crash/lockups.  I've been running Linux for several years and have seen this problem time and time again.  The system log messages at the end of this mail show what I am asking about.  I have several times in the past had a box running and after some time of stable operation, the system will start locking up.  Rebooting usually gets the thing running for a couple hours.  However, the problem never goes away, I have to re-install the OS.  I have never been able to diagnose the problem.  The only thing that seems consistent is the system messages below.  Unfortunately, these messages can run for weeks before the system actually becomes unstable and starts locking up so I never thought much of these messages.

My general question is, do the messages below indicate a problem that can be the cause of my system crash/lockups.  If so, how do I start troubleshooting this?

Also, as shown in the logs below, the process crond was opening and closing sessions.  However, it appears something happened with init and/or xinetd that caused crond to stop logging sessions for user root.  I don't get the sessions for root anymore (I do still get the sessions for postgres), what could have happened?

Note that I am not running PostgreSQL server so I don't know what could possibly have permission/access to open a session for it?

My guess is that my system has been hacked/infected and this is a virus of some kind.

..

Feb 12 04:05:06 msi2 crond(pam_unix)[9463]: session closed for user root
Feb 12 04:10:01 msi2 crond(pam_unix)[9539]: session opened for user root by (uid=0)
Feb 12 04:10:01 msi2 crond(pam_unix)[9541]: session opened for user root by (uid=0)
Feb 12 04:10:01 msi2 crond(pam_unix)[9540]: session opened for user root by (uid=0)
Feb 12 04:10:02 msi2 crond(pam_unix)[9540]: session closed for user root
Feb 12 04:10:02 msi2 crond(pam_unix)[9539]: session closed for user root

..

Feb 14 17:05:01 msi2 crond(pam_unix)[2468]: session opened for user root by (uid=0)
Feb 14 17:05:01 msi2 crond(pam_unix)[2469]: session opened for user root by (uid=0)
Feb 14 17:05:02 msi2 crond(pam_unix)[2468]: session closed for user root
Feb 14 17:05:03 msi2 su(pam_unix)[2496]: session opened for user postgres by (uid=0)
Feb 14 17:05:03 msi2 su(pam_unix)[2496]: session closed for user postgres
Feb 14 17:05:04 msi2 crond(pam_unix)[2469]: session closed for user root
Feb 14 17:09:03 msi2 ntpd[2362]: synchronized to 212.79.244.34, stratum 2
Feb 14 17:09:55 msi2 kernel: loop: loaded (max 8 devices)
Feb 14 17:10:01 msi2 crond(pam_unix)[2608]: session opened for user root by (uid=0)
Feb 14 17:10:01 msi2 crond(pam_unix)[2609]: session opened for user root by (uid=0)
Feb 14 17:10:01 msi2 crond(pam_unix)[2610]: session opened for user root by (uid=0)
Feb 14 17:10:02 msi2 crond(pam_unix)[2609]: session closed for user root
Feb 14 17:10:03 msi2 crond(pam_unix)[2608]: session closed for user root
Feb 14 17:10:06 msi2 su(pam_unix)[2648]: session opened for user postgres by (uid=0)
Feb 14 17:10:06 msi2 su(pam_unix)[2648]: session closed for user postgres
Feb 14 17:10:08 msi2 crond(pam_unix)[2610]: session closed for user root
Feb 14 17:11:27 msi2 init: Trying to re-exec init
Feb 14 17:12:11 msi2 xinetd[2346]: Starting reconfiguration
Feb 14 17:12:12 msi2 xinetd[2346]: Swapping defaults
Feb 14 17:12:12 msi2 xinetd[2346]: Reconfigured: new=0 old=0 dropped=0 (services)
Feb 14 17:13:02 msi2 xinetd[2346]: Starting reconfiguration
Feb 14 17:13:02 msi2 xinetd[2346]: Swapping defaults
Feb 14 17:13:02 msi2 xinetd[2346]: Reconfigured: new=0 old=0 dropped=0 (services)
Feb 14 17:13:48 msi2 named[2012]: client 192.168.0.2#32777: update 'inside.msi/IN' denied
Feb 14 17:13:48 msi2 dhcpd: if RWPowerBookG4.inside.msi IN A rrset doesn't exist add RWPowerBookG4.inside.msi 43200 IN A 192.168.0.177: timed out.
Feb 14 17:13:48 msi2 dhcpd: Wrote 40 leases to leases file.
Feb 14 17:13:48 msi2 dhcpd: DHCPREQUEST for 192.168.0.177 from 00:14:51:28:d6:a2 (RWPowerBookG4) via eth1
Feb 14 17:13:48 msi2 dhcpd: DHCPACK on 192.168.0.177 to 00:14:51:28:d6:a2 (RWPowerBookG4) via eth1
Feb 14 17:15:05 msi2 su(pam_unix)[3822]: session opened for user postgres by (uid=0)
Feb 14 17:15:06 msi2 su(pam_unix)[3822]: session closed for user postgres
Feb 14 17:20:03 msi2 su(pam_unix)[3882]: session opened for user postgres by (uid=0)
Feb 14 17:20:03 msi2 su(pam_unix)[3882]: session closed for user postgres

..

Feb 26 06:15:04 msi2 su(pam_unix)[20826]: session closed for user postgres
Feb 26 06:20:03 msi2 su(pam_unix)[20886]: session opened for user postgres by (uid=0)
Feb 26 06:20:03 msi2 su(pam_unix)[20886]: session closed for user postgres
Feb 26 06:25:03 msi2 su(pam_unix)[20943]: session opened for user postgres by (uid=0)
Feb 26 06:25:03 msi2 su(pam_unix)[20943]: session closed for user postgres
Feb 26 06:30:03 msi2 su(pam_unix)[21003]: session opened for user postgres by (uid=0)
Feb 26 06:30:03 msi2 su(pam_unix)[21003]: session closed for user postgres
Feb 26 06:35:03 msi2 su(pam_unix)[21062]: session opened for user postgres by (uid=0)
Feb 26 06:35:03 msi2 su(pam_unix)[21062]: session closed for user postgres
Feb 26 06:40:03 msi2 su(pam_unix)[21122]: session opened for user postgres by (uid=0)
Feb 26 06:40:03 msi2 su(pam_unix)[21122]: session closed for user postgres
Feb 26 06:45:03 msi2 su(pam_unix)[21179]: session opened for user postgres by (uid=0)



More information about the users mailing list