[Fedora-directory-devel] Please review: [Bug 236256] verify-db.pl (db_verify) does not work on a little endian machine

[Noriko Hosoi]

Summary: verify-db.pl (db_verify) does not work on a little endian machine

https://bugzilla.redhat.com/show_bug.cgi?id=236256

Description of problem:
If there is a index key that has more than 255 IDs in a secondary index b-tree,
db_verify reports the b-tree is corrupted on a little endian machine.

------- Additional Comments From nhosoi(a)redhat.com  2007-09-06 17:58 EST -------
Created an attachment (id=189281)
 --> (https://bugzilla.redhat.com/attachment.cgi?id=189281&action=view)
New File: ldapserver/ldap/servers/slapd/back-ldbm/dbverify.c

Implemented dbverify function which calls Berkeley DB's DB->verify on each
database file (.db# file).

It works as a command line mode via dbverify script (being attached next).
I've also implemented and tested the task mode, but DB->verify function does
not like the way the DS opens the DB environment:
[05/Sep/2007:18:41:16 -0700] - libdb: DB->verify may not be used with
transactions, logging, or locking

If we cannot share the DB environment, there is no much advantage to use
the task mechanism, I think.  Thus, I removed the task code for now.

------- Additional Comments From nhosoi(a)redhat.com  2007-09-06 18:09 EST -------
Created an attachment (id=189291)
 --> (https://bugzilla.redhat.com/attachment.cgi?id=189291&action=view)
New File: ldapserver/ldap/admin/src/scripts/template-dbverify.in

This is a template file to be instantiated when a server instance is created.

Usage: dbverify [-n backend_instance] [-V]
Note : if "-n backend_instance" is not passed, verify all DBs.
  -V : verbose

Sample usages:
$ dbverify
DB verify: Passed
$ dbverify -n userRoot
DB verify: Passed

$ dbverify -n bogus
DB verify: Failed

$ dbverify -V
[...] DB verify - /var/lib/dirsrv/slapd-laputa2/db/userRoot/sn.db4: ok
[...] DB verify - /var/lib/dirsrv/slapd-laputa2/db/userRoot/parentid.db4: ok
    [...]
[...] DB verify - /var/lib/dirsrv/slapd-laputa2/db/userRoot/id2entry.db4: ok
DB verify: Passed

$ dbverify -V
[...] DB verify - /var/lib/dirsrv/slapd-laputa3/db/userRoot/parentid.db4: ok
    [...]
[...] - libdb: Page 0: nonsensical bt_minkey value 1 on metadata page
[...] - libdb: Page 0: nonsensical root page 0 on metadata page
[...] DB verify - verify failed(-30976): /var/lib/dirsrv/slapd-laputa3/db/userRoot/cn.db4
    [...]
[...] DB verify - /var/lib/dirsrv/slapd-laputa3/db/userRoot/id2entry.db4: ok
DB verify: Failed

------- Additional Comments From nhosoi(a)redhat.com  2007-09-06 18:17 EST -------
Created an attachment (id=189301)
 --> (https://bugzilla.redhat.com/attachment.cgi?id=189301&action=view)
cvs diffs 

Modified Files:
 Makefile.am
 ldap/admin/src/scripts/template-verify-db.pl.in
 ldap/servers/slapd/main.c
 ldap/servers/slapd/pblock.c
 ldap/servers/slapd/slap.h
 ldap/servers/slapd/slapi-private.h
 ldap/servers/slapd/task.c
 ldap/servers/slapd/back-ldbm/dblayer.c
 ldap/servers/slapd/back-ldbm/init.c
 ldap/servers/slapd/back-ldbm/proto-back-ldbm.h

Descriptions: 
1) adding dbverify (template-dbverify.in)
2) adding dbverify.c
3) updating verify-db.pl calling dbverify instead of db_verify from Berkeley DB
4) updating main.c/pblock.c/slap.h/slapi-private.c to make dbverify mode available
5) fixing minor memory leak (task.c) and mode confusion (dblayer.c)


Sample usages:
$ verify-db.pl
*****************************************************************
verify-db: This tool should only be run if recovery start fails
and the server is down.  If you run this tool while the server is
running, you may get false reports of corrupted files or other
false errors.
*****************************************************************
Verify log files in /var/lib/dirsrv/slapd-laputa2/db ... Good
Verify db files ... Good

$ verify-db.pl
*****************************************************************
verify-db: This tool should only be run if recovery start fails
and the server is down.  If you run this tool while the server is
running, you may get false reports of corrupted files or other
false errors.
*****************************************************************
Verify log files in /var/lib/dirsrv/slapd-laputa3/db ... Good
Verify db files ...
[...] - libdb: Page 0: nonsensical bt_minkey value 1 on metadata page
[...] - libdb: Page 0: nonsensical root page 0 on metadata page
[...] DB verify - verify failed(-30976): /var/lib/dirsrv/slapd-laputa3/db/userRoot/cn.db4
Found the index file(s) was corrupted
Please run db2index on the corrupted index