read_state: failed to get generator's state

[Jim Tyrrell]

Hi,

I have a setup with 3 389 Directory servers in a master-master setup 
which has been working fine until now.  One of the servers died this 
morning due to a memory issue and now will not restart:


[18/May/2011:12:34:32 +0100] memory allocator - malloc of 1538 bytes 
failed; OS error 12 (Cannot allocate memory)
The server has probably allocated all available virtual memory. To solve
this problem, make more virtual memory available to your server, or reduce
one or more of the following server configuration settings:
   nsslapd-cachesize        (Database Settings - Maximum entries in cache)
   nsslapd-cachememsize     (Database Settings - Memory available for cache)
   nsslapd-dbcachesize      (LDBM Plug-in Settings - Maximum cache size)
   nsslapd-import-cachesize (LDBM Plug-in Settings - Import cache size).
Can't recover; calling exit(1).


I'm not unable to restart the server, the error log reports the following:

[18/May/2011:12:58:38 +0100] - 389-Directory/1.2.2 B2009.237.206 starting up
[18/May/2011:12:58:38 +0100] uuid - read_state: failed to get 
generator's state
[18/May/2011:12:58:38 +0100] uuid - uuid_init: failed to get generator's 
state
[18/May/2011:12:58:38 +0100] uniqueid generator - uniqueIDGenInit: 
generator initialization failed
[18/May/2011:12:58:38 +0100] - Fatal Error---Failed to initialize 
uniqueid generator; error = 13. Exiting now.
         389-Directory/1.2.2 B2009.237.206


Any ideas what the issue is and how to fix?  Where is it trying to get 
the generators state from?

Thanks.

Jim.

[Rich Megginson]

On 05/18/2011 06:36 AM, Jim Tyrrell wrote:
>  Hi,
>
>  I have a setup with 3 389 Directory servers in a master-master setup
>  which has been working fine until now.  One of the servers died this
>  morning due to a memory issue and now will not restart:
>
>
>  [18/May/2011:12:34:32 +0100] memory allocator - malloc of 1538 bytes
>  failed; OS error 12 (Cannot allocate memory)
>  The server has probably allocated all available virtual memory. To solve
>  this problem, make more virtual memory available to your server, or reduce
>  one or more of the following server configuration settings:
>      nsslapd-cachesize        (Database Settings - Maximum entries in cache)
>      nsslapd-cachememsize     (Database Settings - Memory available for cache)
>      nsslapd-dbcachesize      (LDBM Plug-in Settings - Maximum cache size)
>      nsslapd-import-cachesize (LDBM Plug-in Settings - Import cache size).
>  Can't recover; calling exit(1).
>
>
>  I'm not unable to restart the server, the error log reports the following:
>
>  [18/May/2011:12:58:38 +0100] - 389-Directory/1.2.2 B2009.237.206 starting up
>  [18/May/2011:12:58:38 +0100] uuid - read_state: failed to get
>  generator's state
>  [18/May/2011:12:58:38 +0100] uuid - uuid_init: failed to get generator's
>  state
>  [18/May/2011:12:58:38 +0100] uniqueid generator - uniqueIDGenInit:
>  generator initialization failed
>  [18/May/2011:12:58:38 +0100] - Fatal Error---Failed to initialize
>  uniqueid generator; error = 13. Exiting now.
>            389-Directory/1.2.2 B2009.237.206
>
>
>  Any ideas what the issue is and how to fix?  Where is it trying to get
>  the generators state from? 
It's trying to get the generator state from
cn=uniqueid 
generator,cn=config.  It must have been corrupted by the OOM shutdown.
Just remove this entry from dse.ldif.  The server will create a new one.
Also, I strongly encourage you to move to 1.2.8.2 or 1.2.8.3.
>  Thanks.
>
>  Jim.
>  --
>  389 users mailing list
>  389-users(a)lists.fedoraproject.org
>  https://admin.fedoraproject.org/mailman/listinfo/389-users

[Jeremy A. Mates]

If Linux is involved, you may have a version of selinux that leaks
memory (disable selinux and reboot). Also, 389-ds 1.2.8 something may
leak memory if the /var/lib*/dirsrv/*/db/userRoot/id2entry.db4 file
exceeds the nsslapd-cachememsize setting.

https://bugzilla.redhat.com/show_bug.cgi?id=695440

https://bugzilla.redhat.com/show_bug.cgi?id=697701

http://docs.redhat.com/docs/en-US/Red_Hat_Directory_Server/8.2/html-singl...

Also consider periodic preemptive graceful restarts at as appropriate
times as possible to avoid hard crashes, if the leaks cannot be worked
around.

Jeremy