3:16 p.m.
On F11 beta:
yum update
Loaded plugins: fastestmirror, refresh-packagekit
rpmdb: Thread/process 3935/140267357374192 failed: Thread died in
Berkeley DB library
error: db4 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal
error, run database recovery
error: cannot open Packages index using db3 - (-30974)
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
File "/usr/bin/yum", line 29, in <module>
yummain.user_main(sys.argv[1:], exit_code=True)
File "/usr/share/yum-cli/yummain.py", line 309, in user_main
errcode = main(args)
File "/usr/share/yum-cli/yummain.py", line 157, in main
base.getOptionsConfig(args)
File "/usr/share/yum-cli/cli.py", line 187, in getOptionsConfig
self.conf
File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 649, in
<lambda>
conf = property(fget=lambda self: self._getConfig(),
File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 239, in
_getConfig
self._conf = config.readMainConfig(startupconf)
File "/usr/lib/python2.6/site-packages/yum/config.py", line 794, in
readMainConfig
yumvars['releasever'] = _getsysver(startupconf.installroot,
startupconf.distroverpkg)
File "/usr/lib/python2.6/site-packages/yum/config.py", line 867, in
_getsysver
idx = ts.dbMatch('provides', distroverpkg)
TypeError: rpmdb open failed
And this didn't fix it:
rpm --rebuilddb
rpmdb: Thread/process 3935/140267357374192 failed: Thread died in
Berkeley DB library
error: db4 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal
error, run database recovery
error: cannot open Packages index using db3 - (-30974)
I'll file a bug, but how can I fix this in the meantime?
sean
5:12 p.m.
On Fri, Apr 17, 2009 at 3:16 PM, sean darcy <seandarcy2(a)gmail.com> wrote:
On F11 beta:
yum update
Loaded plugins: fastestmirror, refresh-packagekit
rpmdb: Thread/process 3935/140267357374192 failed: Thread died in Berkeley
DB library
error: db4 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error,
run database recovery
error: cannot open Packages index using db3 - Â (-30974)
error: cannot open Packages database in /var/lib/rpm
Traceback (most recent call last):
 File "/usr/bin/yum", line 29, in <module>
  yummain.user_main(sys.argv[1:], exit_code=True)
 File "/usr/share/yum-cli/yummain.py", line 309, in user_main
  errcode = main(args)
 File "/usr/share/yum-cli/yummain.py", line 157, in main
  base.getOptionsConfig(args)
 File "/usr/share/yum-cli/cli.py", line 187, in getOptionsConfig
  self.conf
 File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 649, in
<lambda>
  conf = property(fget=lambda self: self._getConfig(),
 File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 239, in
_getConfig
  self._conf = config.readMainConfig(startupconf)
 File "/usr/lib/python2.6/site-packages/yum/config.py", line 794, in
readMainConfig
  yumvars['releasever'] = _getsysver(startupconf.installroot,
startupconf.distroverpkg)
 File "/usr/lib/python2.6/site-packages/yum/config.py", line 867, in
_getsysver
  idx = ts.dbMatch('provides', distroverpkg)
TypeError: rpmdb open failed
And this didn't fix it:
rpm --rebuilddb
rpmdb: Thread/process 3935/140267357374192 failed: Thread died in Berkeley
DB library
error: db4 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error,
run database recovery
error: cannot open Packages index using db3 - Â (-30974)
I'll file a bug, but how can I fix this in the meantime?
Reboot? Odd, I know, but I've had to on occasions where updates put
libraries in confused states.
jerry
5:21 p.m.
On 04/18/2009 03:42 AM, Jerry Amundson wrote:
On Fri, Apr 17, 2009 at 3:16 PM, sean darcy
<seandarcy2(a)gmail.com> wrote:
> On F11 beta:
>
> yum update
> Loaded plugins: fastestmirror, refresh-packagekit
> rpmdb: Thread/process 3935/140267357374192 failed: Thread died in Berkeley
> DB library
> error: db4 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error,
> run database recovery
> error: cannot open Packages index using db3 - (-30974)
> error: cannot open Packages database in /var/lib/rpm
> Traceback (most recent call last):
> File "/usr/bin/yum", line 29, in <module>
> yummain.user_main(sys.argv[1:], exit_code=True)
> File "/usr/share/yum-cli/yummain.py", line 309, in user_main
> errcode = main(args)
> File "/usr/share/yum-cli/yummain.py", line 157, in main
> base.getOptionsConfig(args)
> File "/usr/share/yum-cli/cli.py", line 187, in getOptionsConfig
> self.conf
> File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 649, in
> <lambda>
> conf = property(fget=lambda self: self._getConfig(),
> File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 239, in
> _getConfig
> self._conf = config.readMainConfig(startupconf)
> File "/usr/lib/python2.6/site-packages/yum/config.py", line 794, in
> readMainConfig
> yumvars['releasever'] = _getsysver(startupconf.installroot,
> startupconf.distroverpkg)
> File "/usr/lib/python2.6/site-packages/yum/config.py", line 867, in
> _getsysver
> idx = ts.dbMatch('provides', distroverpkg)
> TypeError: rpmdb open failed
>
> And this didn't fix it:
> rpm --rebuilddb
> rpmdb: Thread/process 3935/140267357374192 failed: Thread died in Berkeley
> DB library
> error: db4 error(-30974) from dbenv->failchk: DB_RUNRECOVERY: Fatal error,
> run database recovery
> error: cannot open Packages index using db3 - (-30974)
>
> I'll file a bug, but how can I fix this in the meantime?
Reboot? Odd, I know, but I've had to on occasions where updates put
libraries in confused states.
It might also work because /etc/rc.sysinit has
"rm -f /var/lib/rpm/__db* &> /dev/null"
Rahul
5:20 p.m.
On Fri, Apr 17, 2009 at 5:21 PM, Rahul Sundaram
<sundaram(a)fedoraproject.org> wrote:
On 04/18/2009 03:42 AM, Jerry Amundson wrote:
>> I'll file a bug, but how can I fix this in the meantime?
>
> Reboot? Odd, I know, but I've had to on occasions where updates put
> libraries in confused states.
It might also work because /etc/rc.sysinit has
"rm -f /var/lib/rpm/__db* &> /dev/null"
Heh, that's more likely. I still get get windows-on-the-brain sometimes...
jerry
7:41 p.m.
Jerry Amundson wrote:
On Fri, Apr 17, 2009 at 5:21 PM, Rahul Sundaram
<sundaram(a)fedoraproject.org> wrote:
> On 04/18/2009 03:42 AM, Jerry Amundson wrote:
>>> I'll file a bug, but how can I fix this in the meantime?
>> Reboot? Odd, I know, but I've had to on occasions where updates put
>> libraries in confused states.
> It might also work because /etc/rc.sysinit has
>
> "rm -f /var/lib/rpm/__db* &> /dev/null"
Heh, that's more likely. I still get get windows-on-the-brain sometimes...
jerry
Rebooting worked. I'm using ext4. Some googling says that's the issue.
sean
4:49 a.m.
On Fri, 17 Apr 2009, sean darcy wrote:
Jerry Amundson wrote:
> On Fri, Apr 17, 2009 at 5:21 PM, Rahul Sundaram
> <sundaram(a)fedoraproject.org> wrote:
>> On 04/18/2009 03:42 AM, Jerry Amundson wrote:
>>>> I'll file a bug, but how can I fix this in the meantime?
>>> Reboot? Odd, I know, but I've had to on occasions where updates put
>>> libraries in confused states.
>> It might also work because /etc/rc.sysinit has
>>
>> "rm -f /var/lib/rpm/__db* &> /dev/null"
>
> Heh, that's more likely. I still get get windows-on-the-brain sometimes...
>
> jerry
>
Rebooting worked. I'm using ext4. Some googling says that's the issue.
No, the issue is that rpm died while in middle of Berkeley DB call which
is something that obviously should not happen and this is a situation
which doesn't get automatically cleaned up, so you need manual 'rm -f
/var/lib/rpm/__*' to continue.
What made rpm die in this manner is another question, and typically
pretty much impossible to figure out afterwards. Basically it either
crashed or was kill -9'ed while in middle of operating on rpmdb.
Filesystem bugs can/will certainly make rpm(db) blow up in dramatic ways,
and there was some indication that early ext4 made rpm unhappy but haven't
seen any "evidence" of ext4 being more problematic than ext3 recently.
At some point (but fixed since AFAIK) canceling searches in PackageKit
could trigger this due to using kill -9 on the backend.
You might want to check logs for segfaults from yum, rpm and related
pieces. If there's nothing there... impossible to say.
- Panu -
9:04 a.m.
On Sat, 18 Apr 2009, Panu Matilainen wrote:
No, the issue is that rpm died while in middle of Berkeley DB call
which is
something that obviously should not happen and this is a situation which
doesn't get automatically cleaned up, so you need manual 'rm -f
/var/lib/rpm/__*' to continue.
What made rpm die in this manner is another question, and typically pretty
much impossible to figure out afterwards. Basically it either crashed or was
kill -9'ed while in middle of operating on rpmdb. Filesystem bugs can/will
certainly make rpm(db) blow up in dramatic ways, and there was some
indication that early ext4 made rpm unhappy but haven't seen any "evidence"
of ext4 being more problematic than ext3 recently.
At some point (but fixed since AFAIK) canceling searches in PackageKit could
trigger this due to using kill -9 on the backend.
You might want to check logs for segfaults from yum, rpm and related pieces.
If there's nothing there... impossible to say.
or of PK. It seems like we've been seeing a lot of these when PK is
running.
-sv
6:17 a.m.
On Mon, 2009-04-20 at 10:04 -0400, Seth Vidal wrote:
> You might want to check logs for segfaults from yum, rpm and
related
pieces.
> If there's nothing there... impossible to say.
or of PK. It seems like we've been seeing a lot of these when PK is
running.
Random question. Is it safe to ctrl-c when yum-presto is rebuilding
packages?
Richard.
7:35 a.m.
On Tue, 21 Apr 2009, Richard Hughes wrote:
On Mon, 2009-04-20 at 10:04 -0400, Seth Vidal wrote:
>> You might want to check logs for segfaults from yum, rpm and related
> pieces.
>> If there's nothing there... impossible to say.
>
> or of PK. It seems like we've been seeing a lot of these when PK is
> running.
Random question. Is it safe to ctrl-c when yum-presto is rebuilding
packages?
Safe how? It's likely the package won't be rebuilt properly but it
shouldn't damage anything other than the pkg it is trying to rebuild.
-sv
7:57 a.m.
On Tue, 2009-04-21 at 08:35 -0400, Seth Vidal wrote:
Safe how? It's likely the package won't be rebuilt properly
but it
shouldn't damage anything other than the pkg it is trying to rebuild.
Cool, that's what I wanted to know. PackageKit allows the user to cancel
the transaction when the package is being rebuilt, and I wondered if
this could cause the rpmdb corruption people are seeing. I guess this
isn't the case.
Richard.
8:03 a.m.
On Tue, 21 Apr 2009, Richard Hughes wrote:
On Tue, 2009-04-21 at 08:35 -0400, Seth Vidal wrote:
> Safe how? It's likely the package won't be rebuilt properly but it
> shouldn't damage anything other than the pkg it is trying to rebuild.
Cool, that's what I wanted to know. PackageKit allows the user to cancel
the transaction when the package is being rebuilt, and I wondered if
this could cause the rpmdb corruption people are seeing. I guess this
isn't the case.
Well... that all depends whether the rpmdb is open when the packages are
getting rebuilt or not.
- Panu -
9:35 a.m.
On Tue, 21 Apr 2009, Panu Matilainen wrote:
On Tue, 21 Apr 2009, Richard Hughes wrote:
> On Tue, 2009-04-21 at 08:35 -0400, Seth Vidal wrote:
>> Safe how? It's likely the package won't be rebuilt properly but it
>> shouldn't damage anything other than the pkg it is trying to rebuild.
>
> Cool, that's what I wanted to know. PackageKit allows the user to cancel
> the transaction when the package is being rebuilt, and I wondered if
> this could cause the rpmdb corruption people are seeing. I guess this
> isn't the case.
Well... that all depends whether the rpmdb is open when the packages are
getting rebuilt or not.
Or actually more to the point: sending SIGINT or SIGTERM is fine, rpm
blocks that when the db is open. But sending a SIGKILL (like PK does if
the backend doesn't terminate quickly) is just Russian roulette -
sometimes you get away with it, sometimes you dont.
If SIGKILL happens while rpm is running inside db4 code, with current rpm
+ db4 combination you'll get one of these:
rpmdb: Thread/process 16507/3086059200 failed: Thread died in Berkeley DB
library
error: db4 error(-30975) from dbenv->failchk: DB_RUNRECOVERY: Fatal error,
run database recovery
Previously (eg in F9) you'd just have a dead rpm which hangs when you try
to access it.
- Panu -
10:39 a.m.
On Tue, 21 Apr 2009, Panu Matilainen wrote:
Or actually more to the point: sending SIGINT or SIGTERM is fine, rpm
blocks
that when the db is open. But sending a SIGKILL (like PK does if the backend
doesn't terminate quickly) is just Russian roulette - sometimes you get away
with it, sometimes you dont.
If SIGKILL happens while rpm is running inside db4 code, with current rpm +
db4 combination you'll get one of these:
rpmdb: Thread/process 16507/3086059200 failed: Thread died in Berkeley DB
library
error: db4 error(-30975) from dbenv->failchk: DB_RUNRECOVERY: Fatal error,
run database recovery
Has this been filed, then?
-sv
2:59 a.m.
On Tue, 2009-04-21 at 17:35 +0300, Panu Matilainen wrote:
Or actually more to the point: sending SIGINT or SIGTERM is fine, rpm
blocks that when the db is open. But sending a SIGKILL (like PK does
if the backend doesn't terminate quickly) is just Russian roulette -
sometimes you get away with it, sometimes you dont.
Right. How about I just don't send SIGKILL at all for yum?
Richard.
5:12 a.m.
On Thu, 23 Apr 2009, Richard Hughes wrote:
On Tue, 2009-04-21 at 17:35 +0300, Panu Matilainen wrote:
> Or actually more to the point: sending SIGINT or SIGTERM is fine, rpm
> blocks that when the db is open. But sending a SIGKILL (like PK does
> if the backend doesn't terminate quickly) is just Russian roulette -
> sometimes you get away with it, sometimes you dont.
Right. How about I just don't send SIGKILL at all for yum?
That would be best, yes.
We then probably have a situation where cancel (sometimes?) doesn't seem
to work (within a reasonable time) but non-working rpm is far worse than
non-working PK cancel. And rpm's responsiveness to SIGINT can certainly be
improved a good deal, I'm just looking at how to best deal it and some
other signal related issues which currently are pretty broken.
- Panu -
6:01 a.m.
On Thu, 2009-04-23 at 13:12 +0300, Panu Matilainen wrote:
On Thu, 23 Apr 2009, Richard Hughes wrote:
> Right. How about I just don't send SIGKILL at all for yum?
That would be best, yes.
[hughsie@localhost src]$ git whatchanged
33894239b42dcb6069ab3a99e3cb47667448d499..a782056c337562594155a9405adcd9d5e65be32d
commit a782056c337562594155a9405adcd9d5e65be32d
Author: Richard Hughes <richard(a)hughsie.com>
Date: Thu Apr 23 11:55:59 2009 +0100
yum: opt out of SIGKILL as this can lead to RPM database corruption in some
situations
:100644 100644 05f573c... 87228b1... M backends/yum/pk-backend-yum.c
commit 61295f578f4fcafce512f3f4c29ee28de622a05f
Author: Richard Hughes <richard(a)hughsie.com>
Date: Thu Apr 23 11:55:24 2009 +0100
Allow a spawned backend to opt-out of the SIGKILL cancel process, and add a default in
PackageKit.conf
:100644 100644 36b7e8f... 5ab4cd6... M etc/PackageKit.conf.in
:100644 100644 417f142... 2942584... M src/pk-backend-spawn.c
:100644 100644 894c4a6... b55e7ad... M src/pk-backend-spawn.h
commit d0200f3f2aad881f8aad626c84ed73fe0f840834
Author: Richard Hughes <richard(a)hughsie.com>
Date: Thu Apr 23 11:54:09 2009 +0100
Add a method to disable SIGKILL in PkSpawn
:100644 100644 bd609e4... 5c356a1... M src/pk-spawn.c
:100644 100644 6d81bab... 8901fb0... M src/pk-spawn.h
We then probably have a situation where cancel (sometimes?)
doesn't seem
to work (within a reasonable time) but non-working rpm is far worse than
non-working PK cancel.
Agree.
And rpm's responsiveness to SIGINT can certainly be
improved a good deal, I'm just looking at how to best deal it and some
other signal related issues which currently are pretty broken.
Okay. Now with git master, PackageKit only sends a SIGQUIT, and does not
send SIGKILL at all. It's up to yum and rpm to DTRT in this case.
Richard.
6:48 a.m.
On Thu, 2009-04-23 at 16:47 +0530, Rahul Sundaram wrote:
On 04/23/2009 04:31 PM, Richard Hughes wrote:
> Okay. Now with git master, PackageKit only sends a SIGQUIT, and does not
> send SIGKILL at all. It's up to yum and rpm to DTRT in this case.
Can you backport to Rawhide and request a freeze exception as well?
Apart from PackageKit passing its self checks, it's had zero
testing! :-)
I've just made a F11 build here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=1316602
Can you please test this, and if there are no reported problems I'll
request a freeze exception. Thanks.
Richard.
2:33 a.m.
On Thu, 23 Apr 2009, Richard Hughes wrote:
On Thu, 2009-04-23 at 16:47 +0530, Rahul Sundaram wrote:
> On 04/23/2009 04:31 PM, Richard Hughes wrote:
>
>> Okay. Now with git master, PackageKit only sends a SIGQUIT, and does not
>> send SIGKILL at all. It's up to yum and rpm to DTRT in this case.
Good, thanks. If PK cancels are broken by this, you can pass the blame to
me now ;)
>
> Can you backport to Rawhide and request a freeze exception as well?
Apart from PackageKit passing its self checks, it's had zero
testing! :-)
I've just made a F11 build here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=1316602
Can you please test this, and if there are no reported problems I'll
request a freeze exception. Thanks.
FWIW this is the (presumably) related pile of dupes. It's getting a bit
piled up with the actual issue vs how yum reports it etc but anyway...
https://bugzilla.redhat.com/show_bug.cgi?id=495087
- Panu -
5534
days inactive
5541
days old
18 comments
6 participants
participants (6)
-
Jerry Amundson -
Panu Matilainen -
Rahul Sundaram -
Richard Hughes -
sean darcy -
Seth Vidal