[389-users] How to tell when database backup has finished?
Rich Megginson
rmeggins at redhat.com
Wed Apr 11 14:30:28 UTC 2012
On 04/11/2012 08:19 AM, MATON Brett wrote:
>
> Thanks for the suggestions.
>
> For the record, status 32 was an error on my part, I hadn’t quoted
> the bind DN so ldapsearch was actually complaining about the bind DN
> not being found, not the backup task.
>
> Noriko, I’ve searched “cn=backup,cn=tasks,cn=config” and can’t find
> nstaskstatus anywhere. I’m assuming that it also gets deleted once
> the task is complete...
>
>
> cn=backup,cn=tasks,cn=config is the parent entry for all backup task
> entries. When you create an entry under this parent, it starts the
> backup task. It is the backup task entry that has the nsTaskStatus
> attribute, not the parent entry "cn=backup,cn=tasks,cn=config". When
> the backup task is complete, and the ttl is expired, the backup task
> entry is automatically removed.
>
>
> Hi Rich, I appreciate your clarification. I did however mean that I
> was searching “cn=backup,cn=tasks,cn=config” with no additional
> filters which should have returned everything, nsTaskStatus included
> for any backup job?
>
> Exactly what search parameters (e.g. ldapsearch command line
> arguments) did you use?
>
> I might just be having a dumb moment but I couldn’t think of an
> efficient way to check the error log for a Backup Finished entry,
> anyone with suggestions?
>
>
> The best way is to do the ldapsearch as described above.
>
>
> I’m still having issues, $taskDN for example would be something like
> cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config
>
>
> COUNT=1
>
> while [ ${COUNT} -ne 0 ]; do
>
> #
>
> # OOPS: -b ${taskDN}
>
> # -b "cn=backup,cn=tasks,cn=config" "cn=<task>"
>
> #
>
> COUNT=$(ldapsearch -x -H ${HOST} -D "${BINDAS}" -y ${PWFILE}
> -b "cn=backup,cn=tasks,cn=config" "(${taskDN})" dn | grep -c '^dn')
>
> so this would expand to something like
> COUNT=$(ldapsearch -x -H ldap.example.com -D "cn=directory manager" -y
> pwfile.txt -b "cn=backup,cn=tasks,cn=config"
> "(cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config)" dn |
> grep -c '^dn')
>
> This is incorrect in the following ways:
> 1) the argument to -H must be a valid LDAP URL - so -H
> ldap://ldap.example.com $HOST is just a variable name, it’s actually
> the URL
> 2) "(cn=backup_2012_4_5_13_36_23, cn=backup, cn=tasks, cn=config)" is
> not a valid LDAP search filter - if you want to search for the backup
> entry, do one of the following
> a) ldapsearch ... -s base -b "cn=backup_2012_4_5_13_36_23, cn=backup,
> cn=tasks, cn=config" ...
> or
> b) ldapsearch ... -b "cn=backup, cn=tasks, cn=config"
> "(cn=backup_2012_4_5_13_36_23)" ...
>
> OK, search terms corrected.
>
> The command expands to and returns this:
>
> ldapsearch -x -H ldaps://<host fqdn>:636 -D "cn=Directory Manager" -y
> /root/.389ds-pw -s base -b "cn=backup_2012_4_11_7_28_40, cn=backup,
> cn=tasks, cn=config"
>
> # extended LDIF
>
> #
>
> # LDAPv3
>
> # base <cn=backup_2012_4_11_7_28_40, cn=backup, cn=tasks, cn=config>
> with scope baseObject
>
> # filter: (objectclass=*)
>
> # requesting: ALL
>
> #
>
> # search result
>
> search: 2
>
> result: 32 No such object
>
> # numResponses: 1
>
>
> I tweaked the db2bak.pl script to include ttl: 600 as I wanted some
> time to examine the backup task entry and search results:
>
> ...
>
> $dn = "dn: cn=$taskname, cn=backup, cn=tasks, cn=config\n";
>
> $misc = "changetype: add\nobjectclass: top\nobjectclass:
> extensibleObject\n*ttl: 600\n*";
>
> $cn = "cn: $taskname\n";
> ...
>
> The entry gets added, but still disappears near instantaneously
> (db2bak.pl called standalone for output):
>
> Back up directory:
> /var/lib/dirsrv/slapd-<host>/bak/<host>-2012_4_11_7_42_4
>
> ldap_initialize( <DEFAULT> )
>
> add objectclass:
>
> top
>
> extensibleObject
>
> add ttl:
>
> 600
>
> add cn:
>
> backup_2012_4_11_7_42_4
>
> add nsArchiveDir:
>
> /var/lib/dirsrv/slapd-<host>/bak/<host>-2012_4_11_7_42_4
>
> add nsDatabaseType:
>
> ldbm database
>
> adding new entry "cn=backup_2012_4_11_7_42_4, cn=backup, cn=tasks,
> cn=config"
>
> modify complete
>
> As you can see from the ls output below, tar is still being called
> before the task has completed:
>
> [root@<host> backup]# ls -ltr
>
> -rw-r--r--. 1 root root 6788 Apr 11 07:28 20120411072840-<host>.tgz
>
> drwx------. 5 nobody nobody 4096 Apr 11 07:28 20120411072840-<host>
>
> Thanks for your time....
>
> ok - I'd like you to try two more things:
> 1) alter your search parameters - do this instead:
> ldapsearch -x -H ldaps://<host fqdn>:636 -D "cn=Directory Manager" -y
> /root/.389ds-pw -b "cn=backup, cn=tasks, cn=config"
> do you see anything?
>
> Ok this is a bit freaky:
>
> Executing db2pk.pl manually from the command line I get this in
> /var/log/dirsrv/<instance>/error:
>
> [11/Apr/2012:15:33:11 +0200] - Beginning backup of 'ldbm database'
>
> [11/Apr/2012:15:33:12 +0200] - Backing up file 35
> (/var/lib/dirsrv/slapd-<instance>/bak/<instance>-2012_4_11_15_33_12/.repl_changelog_backup/changelogdb/6b85db82-5df911e1-ab138798-28541438_4f45faa60000004b0000.db4)
>
> [11/Apr/2012:15:33:12 +0200] - Backing up file 36
> (/var/lib/dirsrv/slapd-<instance>/bak/<instance>-2012_4_11_15_33_12/.repl_changelog_backup/changelogdb/DBVERSION)
>
> [11/Apr/2012:15:33:12 +0200] - Backup finished.
>
> ldapsearch -x -H ldaps://<instance>.nrb.be:636 -D "cn=Directory
> Manager" -y /root/.389ds-pw -b
> "cn=backup_2012_4_11_15_33_12,cn=backup,cn=tasks,cn=config"
> # extended LDIF
>
> #
>
> # LDAPv3
>
> # base <cn=backup_2012_4_11_15_33_12,cn=backup,cn=tasks,cn=config>
> with scope subtree
>
> # filter: (objectclass=*)
>
> # requesting: ALL
>
> #
>
> # backup_2012_4_11_15_33_12, backup, tasks, config
>
> dn: cn=backup_2012_4_11_15_33_12,cn=backup,cn=tasks,cn=config
>
> objectClass: top
>
> objectClass: extensibleObject
>
> ttl: 600
>
> cn: backup_2012_4_11_15_33_12
>
> nsarchivedir:
> /var/lib/dirsrv/slapd-<instance>/bak/<instance>-2012_4_11_15_33_12
>
> nsdatabasetype: ldbm database
>
> nstaskcurrentitem: 0
>
> nstasktotalitems: 1
>
> nstasklog:: <snipped>
>
> nstaskstatus: Backup finished.
>
> nstaskexitcode: 0
>
> # search result
>
> search: 2
>
> result: 0 Success
>
> # numResponses: 2
>
> # numEntries: 1
>
> Scripted version:
>
> /var/log/dirsrv/<instance>/error:
>
> [11/Apr/2012:15:35:06 +0200] - Beginning backup of 'ldbm database'
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 1
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/telephoneNumber.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 2
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/ntUserDomainId.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 3
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/numsubordinates.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 4
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/objectclass.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 5
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/mail.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 6
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/ntUniqueId.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 7
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/parentid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 8
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/aci.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 9
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/cn.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 10
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/seeAlso.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 11
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/entryrdn.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 12
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/uid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 13
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/ancestorid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 14
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/givenName.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 15
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/sn.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 16
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/uniquemember.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 17
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/nsuniqueid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 18
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/nscpEntryDN.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 19
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/id2entry.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 20
> (/usr/backup/20120411153507-nrbnrl0075/userRoot/DBVERSION)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 21
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/numsubordinates.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 22
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/objectclass.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 23
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/parentid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 24
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/aci.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 25
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/cn.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 26
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/entryrdn.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 27
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/uid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 28
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/ancestorid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 29
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/givenName.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 30
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/sn.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 31
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/uniquemember.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 32
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/nsuniqueid.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 33
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/id2entry.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 34
> (/usr/backup/20120411153507-nrbnrl0075/NetscapeRoot/DBVERSION)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 35
> (/usr/backup/20120411153507-nrbnrl0075/.repl_changelog_backup/changelogdb/6b85db82-5df911e1-ab138798-28541438_4f45faa60000004b0000.db4)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 36
> (/usr/backup/20120411153507-nrbnrl0075/.repl_changelog_backup/changelogdb/DBVERSION)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 37
> (/usr/backup/20120411153507-nrbnrl0075/log.0000000018)
>
> [11/Apr/2012:15:35:07 +0200] - Backing up file 38
> (/usr/backup/20120411153507-nrbnrl0075/DBVERSION)
>
> [11/Apr/2012:15:35:07 +0200] - Backup finished.
>
> Search result:
>
> ldapsearch -x -H ldaps://<host fqdn>:636-D "cn=Directory Manager" -y
> /root/.389ds-pw -s base -b "cn=backup_2012_4_11_15_35_7, cn=backup,
> cn=tasks, cn=config"
>
> # extended LDIF
>
> #
>
> # LDAPv3
>
> # base <cn=backup_2012_4_11_15_35_7, cn=backup, cn=tasks, cn=config>
> with scope baseObject
>
> # filter: (objectclass=*)
>
> # requesting: ALL
>
> #
>
> # search result
>
> search: 2
>
> result: 32 No such object
>
> # numResponses: 1
>
> Search without taskid:
>
> ldapsearch -x -H ldaps://<host fqdn>:636-D "cn=Directory Manager" -y
> /root/.389ds-pw -s base -b "cn=backup,cn=tasks,cn=config"
>
> # extended LDIF
>
> #
>
> # LDAPv3
>
> # base <cn=backup,cn=tasks,cn=config> with scope baseObject
>
> # filter: (objectclass=*)
>
> # requesting: ALL
>
> #
>
> # backup, tasks, config
>
> dn: cn=backup,cn=tasks,cn=config
>
> objectClass: top
>
> objectClass: extensibleObject
>
> cn: backup
>
> # search result
>
> search: 2
>
> result: 0 Success
>
> # numResponses: 2
>
> # numEntries: 1
>
> The only difference I can see is that the scripted version specifies a
> backup path with the –a option
>
> Another test including the –a option from the command line:
>
> [11/Apr/2012:16:10:31 +0200] - Beginning backup of 'ldbm database'
>
> [11/Apr/2012:16:10:31 +0200] - Backing up file 35
> (/tmp/test/.repl_changelog_backup/changelogdb/6b85db82-5df911e1-ab138798-28541438_4f45faa60000004b0000.db4)
>
> [11/Apr/2012:16:10:31 +0200] - Backing up file 36
> (/tmp/test/.repl_changelog_backup/changelogdb/DBVERSION)
>
> [11/Apr/2012:16:10:31 +0200] - Backup finished.
>
> ldapsearch does return the backup task, although the error log is only
> listing backup files 35 / 36 I created tgz’s of the backups and both
> command line / scripted version were identical in size.
> Even more confused J
>
As am I. I'm just not sure what's going on, nor do I know why adding -a
would make any difference here. If you think there is a bug here,
please file a ticket at https://fedorahosted.org/389
>
> 2) can you confirm using the errors log (manually, not automated with
> a script) that your backup task was started successfully and completes
> successfully? Just to be sure that it is indeed being started and
> finishing correctly
>
> Error logs above.
>
> if [ ${COUNT} -ne 0 ]; then
>
> sleep 5s
>
> fi
>
> done
>
> tar czf ${BACKUPPATH}${BKTSTAMP}-${INSTANCE}.tgz -C / ${DSBACKUPDIR#/*}
>
> However the tar file is incomplete, because (assumption) the backup
> has not finished writing to disk before tar starts....
>
> For the time being I’ve settled on two scripts, one to start backups
> and another to archive yesterdays (or older) backups. Not ideal but
> workable
>
> Thanks again,
>
> Brett
>
> *From:*389-users-bounces at lists.fedoraproject.org
> <mailto:389-users-bounces at lists.fedoraproject.org>
> [mailto:389-users-bounces at lists.fedoraproject.org] *On Behalf Of
> *Noriko Hosoi
> *Sent:* 05 April 2012 18:40
> *To:* General discussion list for the 389 Directory server project.
> *Subject:* Re: [389-users] How to tell when database backup has finished?
>
> You could search the task.with nstaskstatus attribute.
>
> $ ldapsearch -LLLx -h localhost -p <port> -D 'cn=directory manager' -w
> <pw> -b "cn=backup,cn=tasks,cn=config" "(cn=backup_*)" nstaskstatus
>
> When the task is done, you'll see the "Backup finished." status:
> dn: cn=backup_2012_4_5_9_35_35,cn=backup,cn=tasks,cn=config
> nstaskstatus: Backup finished.
>
> Another way would be checking the errors log, which logs the end of
> the back up.
>
>
>
> [05/Apr/2012:09:32:09 -0700] - Backing up file 31
> (/var/lib/dirsrv/slapd-totoro/bak/totoro-2012_4_5_9_32_8/DBVERSION)
> [05/Apr/2012:09:32:09 -0700] - Backup finished.
>
> Thanks,
> --noriko
> MATON Brett wrote:
>
> Hi guys,
>
> I want to tar up the backup set once a db2bak.pl backup job has finished.
>
> however because it writes an entry to the LDAP and doesn’t wait for
> the backup to finish I was wondering what the best way to determine
> when the backup has actually finished?
>
> I was thinking that I could
>
> Ldapsearch at intervals for the backup cn entry, when it’s not there
> anymore backup has finished
>
> Ro maybe watch the audit log for a delete action on the backup cn entry..
>
> Any thoughts appreciated.
>
> Cheers,
>
> Brett
>
> -------------------------------------------------------------------
>
> *GreeNRB
> */NRB considers its environmental responsibility and goes for green IT./
> /May we ask you to consider yours before printing this e-mail? /**
>
> *NRB, daring to commit
> */This e-mail and any attachments, which may contain information that
> is confidential and/or protected by intellectual property rights, are
> intended for the exclusive use of the above-mentioned addressee(s).
> Any use (including reproduction, disclosure and whole or partial
> distribution in any form whatsoever) of their content is prohibited
> without prior authorization of NRB. If you have received this message
> by error, please contact the sender promptly by resending this e-mail
> back to him (her), or by calling the above number. Thank you for
> subsequently deleting this e-mail and any files attached thereto./
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/389-users/attachments/20120411/63b5de7c/attachment.html>
More information about the 389-users
mailing list