[389-users] Lots of abandoned connections from sssd

[Rich Megginson]

On 11/10/2014 03:32 PM, Orion Poplawski wrote:
> On 11/06/2014 03:14 AM, Rich Megginson wrote:
>> On 11/06/2014 04:16 AM, Orion Poplawski wrote:
>>> Just recently we're seeing some very strange behavior on our system.
>>> Periodically we will see a sssd process start to have an ever greater number
>>> of connections to our ldap server until the server runs out of file
>>> descriptors.  This seems to be happening with a particular user, who is
>>> having trouble logging in at times, particularly with email (dovecot).  We
>>> see entries like the following on our sever:
>>>
>>> [05/Nov/2014:17:14:51 -0700] conn=1786153 op=0 EXT
>>> oid="1.3.6.1.4.1.1466.20037" name="startTLS"
>>> [05/Nov/2014:17:14:51 -0700] conn=1786153 op=0 RESULT err=0 tag=120
>>> nentries=0 etime=0
>>> [05/Nov/2014:17:14:51 -0700] conn=1786153 SSL 128-bit AES
>>> [05/Nov/2014:17:14:51 -0700] conn=1786153 op=1 BIND
>>> dn="uid=user,ou=People,dc=domain,dc=com" method=128 version=3
>>> [05/Nov/2014:17:14:56 -0700] conn=1786153 op=2 ABANDON targetop=NOTFOUND
>>> msgid=2
>>> [05/Nov/2014:17:14:56 -0700] conn=1786153 op=3 UNBIND
>>> [05/Nov/2014:17:14:56 -0700] conn=1786153 op=3 fd=1022 closed - U1
>>>
>>> I don't yet have debug info from the sssd process.  Any ideas from the above?
>>>
>>> Restarting the sssd process seems to clear things up for a while.
>>>
>>> - Orion
>>>
>> Try to reproduce the problem while using gdb to capture stack traces every few
>> seconds as in http://www.port389.org/docs/389ds/FAQ/faq.html#debugging-hangs
>> Ideally, we can get some stack traces of the server during the time between
>> the BIND and the ABANDON
> If I catch the problem early enough I can still get a stack trace.  A series
> of them are in http://www.cora.nwra.com/~orion/ns-slapd-trace.tar.gz.
> Anything useful there?
>
These traces show the server is almost entirely idle - not working on 
client operations, not deadlocked or otherwise waiting on locks, not 
waiting on I/O to complete.  So either we're still not catching dirsrv 
in the act, or dirsrv is not the problem.