Shutdown hangs on "Turning off quotas"

[Patrick]

On Fri, 2007-04-13 at 21:04 +0100, Keith G. Robertson-Turner wrote: 
> Verily I say unto thee, that Patrick spake thusly:
> > Hi,
> > 
> > Intermittently when I shutdown my laptop with up-to-date FC6 the
> > shutdown process hangs at "Turning off quotas". It just sits there in
> > thin air. Also tried waiting for a few minutes but it seems to hang
> > forever. Anyone know what's causing this and how I might fix this?
> 
> Try this:
> 
> Go to runlevel 3
> ~]# init 3
> 
> Have a look at /etc/init.d/halt
> ~]# less /etc/init.d/halt
> 
> One step at a time, execute the sequence of steps in the "halt" script.
> 
> Keep an eye on /var/log/messages
> 
> In particular, watch what happens when you manually run this:
> 
> ~]# /sbin/quotaoff -aug
> 
> Run ps and look for zombies:
> 
> ~]# ps -eLf | grep defunct
> 
> Try to manually remove modules, one at a time, and see if it makes any
> difference:
> 
> ~]# lsmod | less
> ~]# rmmod <module name>
> 
> I'd guess that something isn't letting go of its resources when it's
> told to die by killall during halt. Check carefully what you are running
> during the course of a session, and see if *not* running any particular
> thing makes a difference.

The hanging quotaoff -aug command seems to be caused by an nfs share
that was mounted when I was logged into the Gnome desktop as a normal
user. Here is an strace of /sbin/quotaoff -aug after logging out of the
Gnome desktop (with the share still mounted and the icon visible on the
Desktop) and switching to init 3:

[root at localhost ~]# /sbin/quotaoff -aug

[root at localhost ~]# strace /sbin/quotaoff -aug
execve("/sbin/quotaoff", ["/sbin/quotaoff", "-aug"], [/* 25 vars */]) = 0
brk(0)                                  = 0x9fca000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f14000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=77643, ...}) = 0
mmap2(NULL, 77643, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7f01000
close(3)                                = 0
open("/lib/libc.so.6", O_RDONLY)        = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\240|\000"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1576920, ...}) = 0
mmap2(0x7b4000, 1295780, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7b4000
mmap2(0x8eb000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x137) = 0x8eb000
mmap2(0x8ee000, 9636, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x8ee000
close(3)                                = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f00000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb7f006c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0x8eb000, 8192, PROT_READ)     = 0
mprotect(0x7b0000, 4096, PROT_READ)     = 0
munmap(0xb7f01000, 77643)               = 0
brk(0)                                  = 0x9fca000
brk(0x9feb000)                          = 0x9feb000
open("/usr/lib/locale/locale-archive", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=55524816, ...}) = 0
mmap2(NULL, 2097152, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7d00000
close(3)                                = 0
rt_sigaction(SIGSEGV, {SIG_DFL}, {SIG_DFL}, 8) = 0
stat64("/proc/fs/xfs/stat", 0xbfab49d4) = -1 ENOENT (No such file or directory)
quotactl(Q_XGETQSTAT|USRQUOTA, NULL, 0, NULL) = -1 EFAULT (Bad address)
stat64("/proc/sys/fs/quota", {st_mode=S_IFDIR|0555, st_size=0, ...}) = 0
rt_sigaction(SIGSEGV, {SIG_DFL}, NULL, 8) = 0
open("/etc/mtab", O_RDONLY)             = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=390, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f13000
read(3, "/dev/sda6 / ext3 rw 0 0\nproc /pr"..., 4096) = 390
lstat64("/mnt", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat64("/mnt/data1",

And here is just hangs. I tried a manual umount /mnt/data1 and it gave
an error and would not unmount the share.

The nfs share entry in /etc/fstab:
box1:/home/patrick      /mnt/data1              nfs     user,rw,exec,nosuid,soft,intr,rsize=8192,wsize=8192     0 0
I'm not too sure about all these settings. Picked them up while googling
around.

I did some tests:

WORKS: log into Gnome Desktop, do not mount the nfs share & shutdown.
WORKS: log into the Gnome Desktop, mount the nfs share, unmount the nfs
share by right clicking on it and selecting unmount then shut down.
DOES NOT WORK: log into the Gnome Desktop, mount the nfs share and
select shutdown.

Any suggestions?

Thanks and regards,
Patrick