How to find and clear zombie processes w/o rebooting?

[Dan Thurman]

Using F8, I have found that Nautilus sometimes hangs and
runs at 100% CPU.  Force killing Nautilus's drive windows
was the only way to recover.  I am not sure that force killing
this drive window is related to the zombie that I founding using
top.

I also discovered that my swap was increased to 2GB of 4GB
and has stayed there ever since.

So before I accuse Nautilus as being the zombie process, how
do I locate it using ps or some other tool to find out what is
going on and to what process the zombie was?

In the same breadth, is there a way to clear zombie processes
without being forced to reboot the system which would certainly
remove the zombie process and clear the swap space?

Thanks!
Dan

[Patrick O'Callaghan]

On Thu, 2008-06-26 at 12:34 +1000, Cameron Simpson wrote:
>    xterm &
>    xclock &
>    window-manager
>  
>  So, that is a shell script. It kicks off a terminal, a clock and a
>  window manager. It waits for that last. When the window manager exits
>  (usually from the quit/logout menu item), the session ends, and
>  gdm/xdm
>  kills the X server and starts a new server for a new login prompt.
>  
>  Now.. if the terminal or clock exit, they will be zombies. 
In fact the Shell waits for forked processes, so they won't be zombies.

poc

[Cameron Simpson]

On 25Jun2008 20:49, Daniel B. Thurman <dant(a)cdkkt.com&gt; wrote:
>  Well! Thanks for the explanation! 
No worries.

>  I guess I will just reboot - problem solved! 
No, just deferred. Presumably it will happen again.

Next time it happens, try doing a "ps elf" and examine the RSS (resident)
and VSZ (virtual size) columns. They may point the way to an unreasonably
large process. You can also tell "top" to sort its display by process
memory size, and to show "idle" processes (press "i"), which it
normally
doesn't bother to display.

Cheers,
-- 
Cameron Simpson <cs(a)zip.com.au&gt; DoD#743
http://www.cskk.ezoshosting.com/cs/

Why is it whatever we don't understand is called a 'thing'? -
"Bones" McCoy

[Cameron Simpson]

On 25Jun2008 23:45, Patrick O'Callaghan <pocallaghan(a)gmail.com&gt; wrote:
| On Thu, 2008-06-26 at 12:34 +1000, Cameron Simpson wrote:
| >   xterm &
| >   xclock &
| >   window-manager
| > 
| > So, that is a shell script. It kicks off a terminal, a clock and a
| > window manager. It waits for that last. When the window manager exits
| > (usually from the quit/logout menu item), the session ends, and
| > gdm/xdm
| > kills the X server and starts a new server for a new login prompt.
| > 
| > Now.. if the terminal or clock exit, they will be zombies.
| 
| In fact the Shell waits for forked processes, so they won't be zombies.

Indeed. However if it execs the window manager, the window manager usually
won't. But point taken.
-- 
Cameron Simpson <cs(a)zip.com.au&gt; DoD#743
http://www.cskk.ezoshosting.com/cs/

Anarchy is not lack of order. Anarchy is lack of ORDERS.

[Dan Thurman]

Craig White wrote:
>  On Wed, 2008-06-25 at 18:33 -0700, Daniel B. Thurman wrote:
>  > Using F8, I have found that Nautilus sometimes hangs and
>  > runs at 100% CPU.  Force killing Nautilus's drive windows
>  > was the only way to recover.  I am not sure that force killing
>  > this drive window is related to the zombie that I founding using
>  > top.
>  >
>  > I also discovered that my swap was increased to 2GB of 4GB
>  > and has stayed there ever since.
>  >
>  > So before I accuse Nautilus as being the zombie process, how
>  > do I locate it using ps or some other tool to find out what is
>  > going on and to what process the zombie was?
>  >
>  > In the same breadth, is there a way to clear zombie processes
>  > without being forced to reboot the system which would certainly
>  > remove the zombie process and clear the swap space?
>  ----
>  you have to kill the parent - you can locate the parent process by
>  running 'ps auxwwf'
>
>  Craig
>  
Interesting! 

Looks like the parent is related to gdm-binary?
Strange about my comment previously - hit 'reset' :D
Wonder what is going on with Xsession and why it died.

This is what I found:
===================
root      3704  0.0  0.1  18416  2096 ?        Ss   Jun24   0:00 
/usr/sbin/gdm-binary -nodaemon
root      3797  0.0  0.0  17896  2064 ?        S    Jun24   0:00  \_ 
/usr/sbin/gdm-binary -nodaemon
root      3801  2.4  2.2  63512 47528 tty7     Ss+  Jun24  40:15      \_ 
/usr/bin/X :0 -br -audit 0 -auth /var/gdm/:0.Xauth -nolisten tcp vt7
dant      3854  0.0  0.1  33768  4104 ?        Ssl  Jun24   0:01      \_ 
/usr/bin/gnome-session
dant      3902  0.0  0.0      0     0 ?        Z    Jun24   
0:00          \_ [Xsession] <defunct>
dant      3956  0.0  0.0   6328   252 ?        Ss   Jun24   
0:00          \_ /usr/bin/ssh-agent /usr/bin/dbus-launch 
--exit-with-session /etc/X11/xinit/Xclients
dant      3976  0.3  0.1  53684  3188 ?        Sl   Jun24   
5:01          \_ /usr/bin/pulseaudio --log-target=syslog
dant      3982  0.0  0.0   5216  1300 ?        S    Jun24   
0:00          |   \_ /usr/libexec/pulse/gconf-helper
dant      3986  0.0  0.5  45136 10576 ?        S    Jun24   
0:36          \_ metacity --sm-client-id 
110a010008000120153982800000046910003
dant      3989  0.0  0.7  75252 15236 ?        S    Jun24   
0:09          \_ gnome-panel --sm-config-prefix /gnome-panel-imrZa5/ 
--sm-client-id 110a010008000120153982700000046910001 --screen 0
dant      4013  0.7  0.3  65148  7292 ?        S    Jun24  
12:56          \_ gkrellm --sm-client-id 
110a010008000120794573900000093780003
dant      4026  0.0  0.7 127592 14612 ?        Sl   Jun24   
1:33          \_ gnome-terminal --sm-config-prefix 
/gnome-terminal-1dtJD1/ --sm-client-id 
110a010008000120153989800000046910008 --screen 0 
--window-with-profile-internal-id=Default --show-menubar 
--role=gnome-terminal-5108--2081150439-1201539898 --active --geometry 
80x24+135+141 --title dant@gold:~ --working-directory /home/dant --zoom 
1 --window-with-profile-internal-id=Default --show-menubar 
--role=gnome-terminal-5108-574728165-1201539901 --active --geometry 
80x24+135+550 --title dant@gold:~ --working-directory /home/dant --zoom 
1 --window-with-profile-internal-id=Default --show-menubar 
--role=gnome-terminal-5108-1143529827-1201539901 --active --geometry 
80x24+799+141 --title dant@gold:~ --working-directory /home/dant --zoom 
1 --window-with-profile-internal-id=Default --show-menubar 
--role=gnome-terminal-5108--1131700843-1201539902 --active --geometry 
80x24+799+550 --title dant@gold:~ --working-directory /home/dant --zoom 1
dant      4191  0.0  0.0   2616   628 ?        S    Jun24   
0:00          |   \_ gnome-pty-helper
dant      4192  0.0  0.0   5616  1056 pts/1    Ss   Jun24   
0:00          |   \_ bash
root      5533  0.0  0.0   4716   916 pts/1    S    Jun24   
0:00          |   |   \_ su
root      5539  0.0  0.0   5620  1608 pts/1    S    Jun24   
0:00          |   |       \_ bash
root     23634  0.0  0.0   4016   828 pts/1    S+   18:34   
0:00          |   |           \_ man ps
root     23654  0.0  0.0   4652  1092 pts/1    S+   18:34   
0:00          |   |               \_ sh -c /usr/bin/bzip2 -c -d 
/var/cache/man/cat1/ps.1.bz2 | /usr/bin/less -is
root     23656  0.0  0.0   4372   932 pts/1    S+   18:34   
0:00          |   |                   \_ /usr/bin/less -is
dant      4198  0.0  0.0   5616  2016 pts/2    Ss   Jun24   
0:00          |   \_ bash
root     23690  0.0  0.0   4716  1232 pts/2    S    18:36   
0:00          |   |   \_ su
root     23697  0.0  0.1   5620  2480 pts/2    S    18:36   
0:00          |   |       \_ bash
root     24340  1.0  0.0   4652   980 pts/2    R+   19:02   
0:00          |   |           \_ ps auxwwf
dant      4203  0.0  0.0   5616  2036 pts/3    Ss+  Jun24   
0:00          |   \_ bash
dant      4215  0.0  0.0   5616  1056 pts/4    Ss+  Jun24   
0:00          |   \_ bash
dant      5267  0.0  0.0   5608  1056 pts/6    Ss   Jun24   
0:00          |   \_ bash
root      5329  0.0  0.0   4716   916 pts/6    S    Jun24   
0:00          |   |   \_ su
root      5339  0.0  0.0   5620  1912 pts/6    S+   Jun24   
0:00          |   |       \_ bash
dant     10269  0.0  0.0   5616  1056 pts/5    Ss   Jun24   
0:00          |   \_ bash
root     10328  0.0  0.0   4716   936 pts/5    S    Jun24   
0:00          |       \_ su
root     10335  0.0  0.0   5912  1728 pts/5    S    Jun24   
0:00          |           \_ bash
root     24065  0.0  0.0   3824   528 pts/5    S+   18:51   
0:00          |               \_ tail -f /var/log/messages
dant      4040 19.5 30.2 3125956 624924 ?      Sl   Jun24 
326:57          \_ strigidaemon
dant      4044  0.0  0.1  28888  2504 ?        SNl  Jun24   
0:00          \_ pinot-dbus-daemon
dant      4185  0.0  0.0   3796   372 ?        S    Jun24   
0:00          |   \_ /bin/cat
dant      4046  0.0  0.2  69920  5456 ?        S    Jun24   
0:16          \_ mugshot --no-show-window
dant      4054  0.0  0.2  20480  4796 ?        S    Jun24   
0:00          \_ tracker-applet
dant      4055  0.0  0.1  39268  3696 ?        SNl  Jun24   
0:17          \_ trackerd
dant      4061  0.0  0.0   4148  1792 ?        S    Jun24   
0:00          \_ online-prefs-sync-daemon
dant      4064  0.0  0.4  78908  8976 ?        S    Jun24   
0:00          \_ /usr/bin/python -tt /usr/bin/puplet
dant      4065  0.0  0.2  28192  4588 ?        S    Jun24   
0:00          \_ python /usr/share/system-config-printer/applet.py
dant      4072  0.0  0.2  99360  4148 ?        S    Jun24   
0:00          \_ nm-applet --sm-disable
dant      4078  0.0  0.1  65020  4012 ?        S    Jun24   
0:00          \_ padevchooser
dant      4118  0.0  0.1  17476  3416 ?        S    Jun24   
0:00          \_ /usr/bin/pam-panel-icon --sm-client-id 
110a010008000120153983200000046910006
root      4136  0.0  0.0   1956   580 ?        S    Jun24   
0:04          |   \_ /sbin/pam_timestamp_check -d root
dant     23276  0.5  2.0 142092 42544 ?        S    18:18   
0:15          \_ nautilus --sm-config-prefix /nautilus-vN3JUZ/ 
--sm-client-id 110a010008000120153982700000046910002 --screen 0

[Dan Thurman]

Cameron Simpson wrote:
>  On 25Jun2008 19:08, Daniel B. Thurman <dant(a)cdkkt.com&gt; wrote:
>  > Looks like the parent is related to gdm-binary?
>  > Strange about my comment previously - hit 'reset' :D
>  > Wonder what is going on with Xsession and why it died.
>  >
>  > This is what I found:
>  > ===================
>  > root      3704  0.0  0.1  18416  2096 ?        Ss   Jun24   0:00 
>  /usr/sbin/gdm-binary -nodaemon
>  > root      3797  0.0  0.0  17896  2064 ?        S    Jun24   0:00  \_ 
>  /usr/sbin/gdm-binary -nodaemon
>  > root      3801  2.4  2.2  63512 47528 tty7     Ss+  Jun24  
>  40:15      \_ /usr/bin/X :0 -br -audit 0 -auth /var/gdm/:0.Xauth 
>  -nolisten tcp vt7
>
>  > dant      3854  0.0  0.1  33768  4104 ?        Ssl  Jun24   
>  0:01      \_ /usr/bin/gnome-session
>  > dant      3902  0.0  0.0      0     0 ?        Z    Jun24   
>  0:00          \_ [Xsession] <defunct>
>  > dant      3956  0.0  0.0   6328   252 ?        Ss   Jun24   
>  0:00          \_ /usr/bin/ssh-agent /usr/bin/dbus-launch 
>  --exit-with-session /etc/X11/xinit/Xclients
>
>  [...]
>
>  Ok, this mean gnome-session forked a run of Xsession and didn't wait for
>  it. Xsession has exited, but whatever gnome-session really waits for
>  (typically the window manager) has not yet exited.
>
>  This is really common in xsession/xinitrc type shell scripts. You see
>  people write their sessions like this:
>
>    xterm &
>    xclock &
>    window-manager
>
>  So, that is a shell script. It kicks off a terminal, a clock and a
>  window manager. It waits for that last. When the window manager exits
>  (usually from the quit/logout menu item), the session ends, and gdm/xdm
>  kills the X server and starts a new server for a new login prompt.
>
>  Now.. if the terminal or clock exit, they will be zombies. The script is
>  may be waiting only for the window manager, expecially if the last line
>  goes:
>
>    exec window-manager
>
>  because the window manager will _replace_ the script, and thus be the
>  parent of the terminal and clock, but it will not be waiting for them.
>
>  It is normally desirable to run "tools" (the terminal, the clock,
>  nautilus etc) as children of a throw away subshell, thus:
>
>    ( xterm &
>      xclock &
>    )
>    exec window manager
>
>  That way the tools are forked off from a subshell, the subshell exits,
>  and the children get inherited by init (_not_ the top shell!) whose
>  purpose in life is to catch such children and reap them, to prevent them
>  becoming zombies.
>
>  So:
>    - it is really a problem in gnome-session
>    - it isn't important, and is not related to your resource problems
>  
Well! Thanks for the explanation!
I guess I will just reboot - problem solved!

Thanks alot!
Dan

[Cameron Simpson]

On 25Jun2008 19:08, Daniel B. Thurman <dant(a)cdkkt.com&gt; wrote:
>  Looks like the parent is related to gdm-binary?
>  Strange about my comment previously - hit 'reset' :D
>  Wonder what is going on with Xsession and why it died.
>
>  This is what I found:
>  ===================
>  root      3704  0.0  0.1  18416  2096 ?        Ss   Jun24   0:00 /usr/sbin/gdm-binary
> -nodaemon
>  root      3797  0.0  0.0  17896  2064 ?        S    Jun24   0:00  \_ /usr/sbin/gdm-binary
> -nodaemon
>  root      3801  2.4  2.2  63512 47528 tty7     Ss+  Jun24  40:15      \_ /usr/bin/X :0
> -br -audit 0 -auth /var/gdm/:0.Xauth -nolisten tcp vt7
>  dant      3854  0.0  0.1  33768  4104 ?        Ssl  Jun24   0:01      \_
> /usr/bin/gnome-session
>  dant      3902  0.0  0.0      0     0 ?        Z    Jun24   0:00          \_ [Xsession]
> <defunct>
>  dant      3956  0.0  0.0   6328   252 ?        Ss   Jun24   0:00          \_
> /usr/bin/ssh-agent /usr/bin/dbus-launch --exit-with-session /etc/X11/xinit/Xclients
[...]

Ok, this mean gnome-session forked a run of Xsession and didn't wait for
it. Xsession has exited, but whatever gnome-session really waits for
(typically the window manager) has not yet exited.

This is really common in xsession/xinitrc type shell scripts. You see
people write their sessions like this:

  xterm &
  xclock &
  window-manager

So, that is a shell script. It kicks off a terminal, a clock and a
window manager. It waits for that last. When the window manager exits
(usually from the quit/logout menu item), the session ends, and gdm/xdm
kills the X server and starts a new server for a new login prompt.

Now.. if the terminal or clock exit, they will be zombies. The script is
may be waiting only for the window manager, expecially if the last line
goes:

  exec window-manager

because the window manager will _replace_ the script, and thus be the
parent of the terminal and clock, but it will not be waiting for them.

It is normally desirable to run "tools" (the terminal, the clock,
nautilus etc) as children of a throw away subshell, thus:

  ( xterm &
    xclock &
  )
  exec window manager

That way the tools are forked off from a subshell, the subshell exits,
and the children get inherited by init (_not_ the top shell!) whose
purpose in life is to catch such children and reap them, to prevent them
becoming zombies.

So:
  - it is really a problem in gnome-session
  - it isn't important, and is not related to your resource problems

Cheers,
-- 
Cameron Simpson <cs(a)zip.com.au&gt; DoD#743
http://www.cskk.ezoshosting.com/cs/

In article <CF3rw3.n0x(a)eskimo.com&gt; pirih(a)eskimo.com (Chris Pirih) writes:
| Wotsa zerk?

It's a portable hemispherical perforated spooge flange.
        - Chuck Rogers, car377(a)torreys.att.com

[Dan Thurman]

Cameron Simpson wrote:
>  On 25Jun2008 18:33, Daniel B. Thurman <dant(a)cdkkt.com&gt; wrote:
>  > Using F8, I have found that Nautilus sometimes hangs and
>  > runs at 100% CPU.  Force killing Nautilus's drive windows
>  > was the only way to recover.  I am not sure that force killing
>  > this drive window is related to the zombie that I founding using
>  > top.
>  > I also discovered that my swap was increased to 2GB of 4GB
>  > and has stayed there ever since.
>
>  Zombies are just process slots of exit()ed processes awaiting collection.
>  They do not consume CPU time or swap space.
>
>  > So before I accuse Nautilus as being the zombie process, how
>  > do I locate it using ps or some other tool to find out what is
>  > going on and to what process the zombie was?
>
>  Zombies are in the Z state. They are exited programs whose parents have
>  not called wait() to collect the exit status.
>
>  > In the same breadth, is there a way to clear zombie processes
>  > without being forced to reboot the system which would certainly
>  > remove the zombie process and clear the swap space?
>
>  Removing zombies will do NOTHING about your swap space, since they are
>  not consuming it. (Of course a reboot will clear the swap, as it does
>  everything else.)
>
>  They only way to clear a zombies process slot (which is all a zombie
>  process actually is) is to find the zombie's parent process (using ps'
>  "f" (forest) option, for example, or just looking at the PPID column)
>  and either causing the parent to do a wait() for the zombie if possible,
>  or to kill the parent (which will cause the zombies to be inherited by
>  process 1, which will then wait() for it, cleaning it up).
>
>  Zombies are essentially untidy but harmless. They are not eating you
>  machine's brains (ram/swap/cpu).
>
>  Cheers,
>  -- 
>  Cameron Simpson <cs(a)zip.com.au&gt; DoD#743
>  http://www.cskk.ezoshosting.com/cs/
>
>  Zombies don't get pumped.       - Jake, in rec.climbing
>  
I like your signature ;)

I did a ps aux | grep Z :
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
dant      3902  0.0  0.0      0     0 ?        Z    Jun24   0:00 
[Xsession] <defunct>

Um - ok.  Something "burped" with the Xsession.  Wonder what happened.

I noticed that my system became slightly sluggish and I have no clue what is
causing it.  I do know that a reboot will bring the system into a 
'snappy' state
and my script that converts ape to mp3 and splits mp3 with a provided cue
file and it really rips.  But then over the last few days, it started 
slowing down
quite noticeably.

Ah- I think I will just do it the BGM$ way: just hit "reset" :D

Thanks for the pointer,
Dan

[Craig White]

On Wed, 2008-06-25 at 18:33 -0700, Daniel B. Thurman wrote:
>  Using F8, I have found that Nautilus sometimes hangs and
>  runs at 100% CPU.  Force killing Nautilus's drive windows
>  was the only way to recover.  I am not sure that force killing
>  this drive window is related to the zombie that I founding using
>  top.
>  
>  I also discovered that my swap was increased to 2GB of 4GB
>  and has stayed there ever since.
>  
>  So before I accuse Nautilus as being the zombie process, how
>  do I locate it using ps or some other tool to find out what is
>  going on and to what process the zombie was?
>  
>  In the same breadth, is there a way to clear zombie processes
>  without being forced to reboot the system which would certainly
>  remove the zombie process and clear the swap space? 
----
you have to kill the parent - you can locate the parent process by
running 'ps auxwwf'

Craig

[Cameron Simpson]

On 25Jun2008 18:33, Daniel B. Thurman <dant(a)cdkkt.com&gt; wrote:
>  Using F8, I have found that Nautilus sometimes hangs and
>  runs at 100% CPU.  Force killing Nautilus's drive windows
>  was the only way to recover.  I am not sure that force killing
>  this drive window is related to the zombie that I founding using
>  top.
>  I also discovered that my swap was increased to 2GB of 4GB
>  and has stayed there ever since. 
Zombies are just process slots of exit()ed processes awaiting collection.
They do not consume CPU time or swap space.

>  So before I accuse Nautilus as being the zombie process, how
>  do I locate it using ps or some other tool to find out what is
>  going on and to what process the zombie was? 
Zombies are in the Z state. They are exited programs whose parents have
not called wait() to collect the exit status.

>  In the same breadth, is there a way to clear zombie processes
>  without being forced to reboot the system which would certainly
>  remove the zombie process and clear the swap space? 
Removing zombies will do NOTHING about your swap space, since they are
not consuming it. (Of course a reboot will clear the swap, as it does
everything else.)

They only way to clear a zombies process slot (which is all a zombie
process actually is) is to find the zombie's parent process (using ps'
"f" (forest) option, for example, or just looking at the PPID column)
and either causing the parent to do a wait() for the zombie if possible,
or to kill the parent (which will cause the zombies to be inherited by
process 1, which will then wait() for it, cleaning it up).

Zombies are essentially untidy but harmless. They are not eating you
machine's brains (ram/swap/cpu).

Cheers,
-- 
Cameron Simpson <cs(a)zip.com.au&gt; DoD#743
http://www.cskk.ezoshosting.com/cs/

Zombies don't get pumped.       - Jake, in rec.climbing