[389-users] MMR Dead-Lock

[Joel Levin]

Hi List:

We have a multi-master set-up: 1 Primary Master, 1 Cold Master, 3 Consumers.

All usually humming well - however today, there were a number of deadlocks
- like below - 2 of which brought the 1 Primary Master (example below from
'error' logs resulted in master going offline).

Any ideas on where to look on what could have caused the deadlock and the
subsequent taking offline of the Primary Master?

Thanks.



[06/Aug/2015:14:17:22 -0700] eldapubcpostop_mod -
20150806141722[06/Aug/2015:14:17:22 -0700] NSMMReplicationPlugin -
agmt="cn=eldap2" (eldap2:636): Consumer failed to replay cha
nge (uniqueid 67c49201-3c6411e5-97f8dfeb-4acc1d05, CSN
55c3cee4000000010000): Protocol error (2). Will retry later.
[06/Aug/2015:14:17:22 -0700] NSMMReplicationPlugin - agmt="cn=eldap3"
(eldap3:636): Consumer failed to replay change (uniqueid
67c49201-3c6411e5-97f8dfeb-4acc1d05, CSN 55c3cee40
00000010000): Protocol error (2). Will retry later.
[06/Aug/2015:14:17:23 -0700] eldapubcpostop_mod - Opened database
successfully
[06/Aug/2015:14:17:23 -0700] eldapubcpostop_mod -
20150806141723[06/Aug/2015:14:17:23 -0700] eldapubcpostop_mod - Opened
database successfully
[06/Aug/2015:14:17:23 -0700] eldapubcpostop_mod -
20150806141723[06/Aug/2015:14:20:17 -0700] NSMMReplicationPlugin -
changelog program - _cl5WriteOperationTxn: retry (49) the tr
ansaction (csn=55c3cf8e000000010000) failed (rc=-30994 (DB_LOCK_DEADLOCK:
Locker killed to resolve a deadlock))
[06/Aug/2015:14:20:17 -0700] NSMMReplicationPlugin - changelog program -
_cl5WriteOperationTxn: failed to write entry with csn
(55c3cf8e000000010000); db error - -30994 DB_LOCK_
DEADLOCK: Locker killed to resolve a deadlock
[06/Aug/2015:14:20:17 -0700] NSMMReplicationPlugin -
write_changelog_and_ruv: can't add a change for
uid=foobar,ou=org,dc=example,dc=com (uniqid: e62f2d01-3c8011e5-
a838dfeb-4acc1d05, optype: 16) to changelog csn 55c3cf8e000000010000
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.fedoraproject.org/pipermail/389-users/attachments/20150806/edb47a09/attachment.html>