Seems like some F7 machines that sit for awhile do this.
# yum clean all Loading "installonlyn" plugin rpmdb: Lock table is out of available locker entries rpmdb: Unknown locker ID: 912 error: db4 error(22) from db->close: Invalid argument error: cannot open Packages index using db3 - Cannot allocate memory (12) error: cannot open Packages database in /var/lib/rpm Traceback (most recent call last): File "/usr/bin/yum", line 29, in <module> yummain.main(sys.argv[1:]) File "/usr/share/yum-cli/yummain.py", line 82, in main base.getOptionsConfig(args) File "/usr/share/yum-cli/cli.py", line 146, in getOptionsConfig errorlevel=opts.errorlevel) File "/usr/lib/python2.5/site-packages/yum/__init__.py", line 153, in _getConfig self._conf = config.readMainConfig(startupconf) File "/usr/lib/python2.5/site-packages/yum/config.py", line 601, in readMainConfig yumvars['releasever'] = _getsysver(startupconf.installroot, startupconf.distroverpkg) File "/usr/lib/python2.5/site-packages/yum/config.py", line 664, in _getsysver idx = ts.dbMatch('provides', distroverpkg) TypeError: rpmdb open failed
Skunk Worx skunkworx@verizon.net writes:
Seems like some F7 machines that sit for awhile do this.
# yum clean all Loading "installonlyn" plugin rpmdb: Lock table is out of available locker entries rpmdb: Unknown locker ID: 912 error: db4 error(22) from db->close: Invalid argument error: cannot open Packages index using db3 - Cannot allocate memory (12) error: cannot open Packages database in /var/lib/rpm
Turn selinux off (not just permissive but disabled), reboot and the problem will go away.
I found that yum/rpm would lock up within a day if I had selinux set to permissive. With it is disabled completely the problem appears gone, even after many days uptime.
-wolfgang
Wolfgang S. Rupprecht wrote:
Skunk Worx skunkworx@verizon.net writes:
Seems like some F7 machines that sit for awhile do this.
# yum clean all Loading "installonlyn" plugin rpmdb: Lock table is out of available locker entries rpmdb: Unknown locker ID: 912 error: db4 error(22) from db->close: Invalid argument error: cannot open Packages index using db3 - Cannot allocate memory (12) error: cannot open Packages database in /var/lib/rpm
Turn selinux off (not just permissive but disabled), reboot and the problem will go away.
I found that yum/rpm would lock up within a day if I had selinux set to permissive. With it is disabled completely the problem appears gone, even after many days uptime.
-wolfgang
Did you see avc messages in /var/log/audit/audit.log?
Daniel J Walsh dwalsh@redhat.com writes:
Did you see avc messages in /var/log/audit/audit.log?
I see quite a few avc messages. If you want I can mail or bz the whole file or just the avc lines.
# wc audit.log 8996 144947 1946140 audit.log # grep avc: audit.log | wc 2614 40929 585751 #
I run nfs4 and postfix with delivery into ~username/Mailbox. Maybe this isn't a heavily tested configuration.
-wolfgang
Wolfgang S. Rupprecht wrote:
Daniel J Walsh dwalsh@redhat.com writes:
Did you see avc messages in /var/log/audit/audit.log?
I see quite a few avc messages. If you want I can mail or bz the whole file or just the avc lines.
# wc audit.log 8996 144947 1946140 audit.log # grep avc: audit.log | wc 2614 40929 585751 #
I run nfs4 and postfix with delivery into ~username/Mailbox. Maybe this isn't a heavily tested configuration.
-wolfgang
Yes please send me the audit.log
Daniel J Walsh wrote:
Wolfgang S. Rupprecht wrote:
Skunk Worx skunkworx@verizon.net writes:
Seems like some F7 machines that sit for awhile do this.
# yum clean all Loading "installonlyn" plugin rpmdb: Lock table is out of available locker entries rpmdb: Unknown locker ID: 912 error: db4 error(22) from db->close: Invalid argument error: cannot open Packages index using db3 - Cannot allocate memory (12) error: cannot open Packages database in /var/lib/rpm
Turn selinux off (not just permissive but disabled), reboot and the problem will go away.
I found that yum/rpm would lock up within a day if I had selinux set to permissive. With it is disabled completely the problem appears gone, even after many days uptime.
-wolfgang
Did you see avc messages in /var/log/audit/audit.log?
I don't see anything that looks like yum or rpm related, my log is much smaller.
For now I am staying with enforcing unless I have to change, I don't mind doing some testing with this email machine.
Note : tried init 1, init 3 but it's not enough
appears to require a reboot.
--- John
Skunk Worx skunkworx@verizon.net writes:
Daniel J Walsh wrote:
Did you see avc messages in /var/log/audit/audit.log?
I don't see anything that looks like yum or rpm related, my log is much smaller.
One thing that stands out (by virtue of having the word "lock" in it ;-)) is the postfix mailbox locking. It might not be anything, since I'm not really sure which needle in this haystack I'm looking for.
# grep -i lock audit.log | grep -v clock_device | tail type=AVC msg=audit(1181331871.161:339): avc: denied { remove_name } for pid=11667 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181331871.161:339): avc: denied { unlink } for pid=11667 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file type=AVC msg=audit(1181333061.829:345): avc: denied { add_name } for pid=11835 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181333064.210:346): avc: denied { remove_name } for pid=11835 comm="local" name="Mailbox.lock" dev=dm-0 ino=71697052 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181333216.630:357): avc: denied { add_name } for pid=11861 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181333217.129:358): avc: denied { remove_name } for pid=11861 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181336644.216:383): avc: denied { add_name } for pid=12469 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181336644.216:383): avc: denied { create } for pid=12469 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file type=AVC msg=audit(1181336644.225:384): avc: denied { remove_name } for pid=12469 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181336644.225:384): avc: denied { unlink } for pid=12469 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file
-wolfgang
Wolfgang S. Rupprecht wrote:
Skunk Worx skunkworx@verizon.net writes:
Daniel J Walsh wrote:
Did you see avc messages in /var/log/audit/audit.log?
I don't see anything that looks like yum or rpm related, my log is much smaller.
One thing that stands out (by virtue of having the word "lock" in it ;-)) is the postfix mailbox locking. It might not be anything, since I'm not really sure which needle in this haystack I'm looking for.
# grep -i lock audit.log | grep -v clock_device | tail type=AVC msg=audit(1181331871.161:339): avc: denied { remove_name } for pid=11667 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181331871.161:339): avc: denied { unlink } for pid=11667 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file type=AVC msg=audit(1181333061.829:345): avc: denied { add_name } for pid=11835 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181333064.210:346): avc: denied { remove_name } for pid=11835 comm="local" name="Mailbox.lock" dev=dm-0 ino=71697052 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181333216.630:357): avc: denied { add_name } for pid=11861 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181333217.129:358): avc: denied { remove_name } for pid=11861 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181336644.216:383): avc: denied { add_name } for pid=12469 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181336644.216:383): avc: denied { create } for pid=12469 comm="local" name="Mailbox.lock" scontext=system_u:system_r:postfix_local_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file type=AVC msg=audit(1181336644.225:384): avc: denied { remove_name } for pid=12469 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=root:object_r:root_t:s0 tclass=dir type=AVC msg=audit(1181336644.225:384): avc: denied { unlink } for pid=12469 comm="local" name="Mailbox.lock" dev=dm-0 ino=70551655 scontext=system_u:system_r:postfix_local_t:s0 tcontext=system_u:object_r:root_t:s0 tclass=file
-wolfgang
This looks like you have postfix attempting to create a file (Mailbox.lock) in the / directory?
Daniel J Walsh dwalsh@redhat.com writes:
This looks like you have postfix attempting to create a file (Mailbox.lock) in the / directory?
I don't see any /Mailbox, but my selinux contexts may be wrong on the user's home directories. They seem to also be "root:object_r:root_t:s0". I copied the home's for f7 from the old fc6 disk using the standard tar pipe. That seems to have mashed things.
tar -c /fc6 -cf - u | tar -C / -xf -
# ll -a --lcontext /u total 120 drwxr-xr-x 14 root:object_r:root_t:s0 root root 4096 2007-06-08 16:37 . drwxr-xr-x 30 system_u:object_r:root_t:s0 root root 4096 2007-06-18 08:59 .. drwxr-xr-x 86 root:object_r:root_t:s0 alison alison 4096 2007-06-08 11:02 alison drwxr-xr-x 2 root:object_r:root_t:s0 root root 4096 2006-11-30 10:51 CVS drwxr-xr-x 35 root:object_r:root_t:s0 cvs staff 4096 2006-11-21 12:52 cvsroot drwxr-xr-x 24 root:object_r:root_t:s0 wolfgang staff 4096 2000-08-29 11:22 gps drwxr-xr-x 3 root:object_r:root_t:s0 laine laine 4096 2006-10-26 15:15 laine drwxr-xr-x 3 root:object_r:root_t:s0 marc marc 4096 2006-10-26 15:15 marc drwxrwxr-x 22 root:object_r:root_t:s0 root wheel 4096 2007-06-15 10:35 src drwxr-xr-x 273 root:object_r:root_t:s0 wolfgang wolfgang 12288 2007-06-18 09:04 wolfgang drwxrwxr-x 17 root:object_r:root_t:s0 root wsrcc 4096 2007-04-04 08:34 www
The /home directory is an automount point that mounts the /u/<username> directory from a main server. This allows the familiar /home for homes.
-wolfgang
Wolfgang S. Rupprecht wrote:
Daniel J Walsh dwalsh@redhat.com writes:
This looks like you have postfix attempting to create a file (Mailbox.lock) in the / directory?
I don't see any /Mailbox, but my selinux contexts may be wrong on the user's home directories. They seem to also be "root:object_r:root_t:s0". I copied the home's for f7 from the old fc6 disk using the standard tar pipe. That seems to have mashed things.
tar -c /fc6 -cf - u | tar -C / -xf -# ll -a --lcontext /u total 120 drwxr-xr-x 14 root:object_r:root_t:s0 root root 4096 2007-06-08 16:37 . drwxr-xr-x 30 system_u:object_r:root_t:s0 root root 4096 2007-06-18 08:59 .. drwxr-xr-x 86 root:object_r:root_t:s0 alison alison 4096 2007-06-08 11:02 alison drwxr-xr-x 2 root:object_r:root_t:s0 root root 4096 2006-11-30 10:51 CVS drwxr-xr-x 35 root:object_r:root_t:s0 cvs staff 4096 2006-11-21 12:52 cvsroot drwxr-xr-x 24 root:object_r:root_t:s0 wolfgang staff 4096 2000-08-29 11:22 gps drwxr-xr-x 3 root:object_r:root_t:s0 laine laine 4096 2006-10-26 15:15 laine drwxr-xr-x 3 root:object_r:root_t:s0 marc marc 4096 2006-10-26 15:15 marc drwxrwxr-x 22 root:object_r:root_t:s0 root wheel 4096 2007-06-15 10:35 src drwxr-xr-x 273 root:object_r:root_t:s0 wolfgang wolfgang 12288 2007-06-18 09:04 wolfgang drwxrwxr-x 17 root:object_r:root_t:s0 root wsrcc 4096 2007-04-04 08:34 www
The /home directory is an automount point that mounts the /u/<username> directory from a main server. This allows the familiar /home for homes.
-wolfgang
restorecon -R -v /root
Daniel J Walsh dwalsh@redhat.com writes:
Wolfgang S. Rupprecht wrote:
# ll -a --lcontext /u total 120 drwxr-xr-x 14 root:object_r:root_t:s0 root root 4096 2007-06-08 16:37 . drwxr-xr-x 30 system_u:object_r:root_t:s0 root root 4096 2007-06-18 08:59 .. drwxr-xr-x 86 root:object_r:root_t:s0 alison alison 4096 2007-06-08 11:02 alison drwxr-xr-x 2 root:object_r:root_t:s0 root root 4096 2006-11-30 10:51 CVS drwxr-xr-x 35 root:object_r:root_t:s0 cvs staff 4096 2006-11-21 12:52 cvsroot drwxr-xr-x 24 root:object_r:root_t:s0 wolfgang staff 4096 2000-08-29 11:22 gps drwxr-xr-x 3 root:object_r:root_t:s0 laine laine 4096 2006-10-26 15:15 laine drwxr-xr-x 3 root:object_r:root_t:s0 marc marc 4096 2006-10-26 15:15 marc drwxrwxr-x 22 root:object_r:root_t:s0 root wheel 4096 2007-06-15 10:35 src drwxr-xr-x 273 root:object_r:root_t:s0 wolfgang wolfgang 12288 2007-06-18 09:04 wolfgang drwxrwxr-x 17 root:object_r:root_t:s0 root wsrcc 4096 2007-04-04 08:34 www
restorecon -R -v /root
Its the user's directories located in /u (instead of /home) that need the restorecon. (The /home directory is automounted.) I suspect I need to copy something in restorecon's database so that it knows that /u contains home directories.
The underlying problem with the rpm db locks failing after a few hours of uptime is what concerns me more. It seemed to be selinux related since turning off selinux fixed the lock leakage.
-wolfgang
Wolfgang S. Rupprecht wrote:
Daniel J Walsh dwalsh@redhat.com writes:
Wolfgang S. Rupprecht wrote:
# ll -a --lcontext /u total 120 drwxr-xr-x 14 root:object_r:root_t:s0 root root 4096 2007-06-08 16:37 . drwxr-xr-x 30 system_u:object_r:root_t:s0 root root 4096 2007-06-18 08:59 .. drwxr-xr-x 86 root:object_r:root_t:s0 alison alison 4096 2007-06-08 11:02 alison drwxr-xr-x 2 root:object_r:root_t:s0 root root 4096 2006-11-30 10:51 CVS drwxr-xr-x 35 root:object_r:root_t:s0 cvs staff 4096 2006-11-21 12:52 cvsroot drwxr-xr-x 24 root:object_r:root_t:s0 wolfgang staff 4096 2000-08-29 11:22 gps drwxr-xr-x 3 root:object_r:root_t:s0 laine laine 4096 2006-10-26 15:15 laine drwxr-xr-x 3 root:object_r:root_t:s0 marc marc 4096 2006-10-26 15:15 marc drwxrwxr-x 22 root:object_r:root_t:s0 root wheel 4096 2007-06-15 10:35 src drwxr-xr-x 273 root:object_r:root_t:s0 wolfgang wolfgang 12288 2007-06-18 09:04 wolfgang drwxrwxr-x 17 root:object_r:root_t:s0 root wsrcc 4096 2007-04-04 08:34 www
restorecon -R -v /root
Its the user's directories located in /u (instead of /home) that need the restorecon. (The /home directory is automounted.) I suspect I need to copy something in restorecon's database so that it knows that /u contains home directories.
The underlying problem with the rpm db locks failing after a few hours of uptime is what concerns me more. It seemed to be selinux related since turning off selinux fixed the lock leakage.
-wolfgang
semanage fcontext -a -t home_root_t /u semanage fcontext -a -t user_home_dir_t -f-d /u/[^/]* semanage fcontext -a -t user_home_t /u/[^/]*/.+
should clean that up
Yes if the rpm problem happens in permissive mode it should be reported as a bug to rpm.
Daniel J Walsh wrote:
Wolfgang S. Rupprecht wrote:
Daniel J Walsh dwalsh@redhat.com writes:
Wolfgang S. Rupprecht wrote:
The underlying problem with the rpm db locks failing after a few hours of uptime is what concerns me more. It seemed to be selinux related since turning off selinux fixed the lock leakage.
-wolfgang
Yes if the rpm problem happens in permissive mode it should be reported as a bug to rpm.
looks like it's bz # 245389
-
Daniel J Walsh -
Skunk Worx -
Wolfgang S. Rupprecht