[389-users] Re: FileDescriptors exhausted

[Paul Robert Marino]

It sounds like you may have stale client connections. There are a lot
of bad LDAP clients that never close their connections properly.
the way to work around this is to set a TCP keepalive to a shorter
than default. here is a good article on the subject
https://www.looklinux.com/how-to-configure-tcp-keepalive-setting-in-linux/
i recommend that heartbeats be initiated after 60 seconds after the
connection becomes idle and only a few failures before they are
dropped.
Also it's important to know all 3 settings must be set for this to
work. If you don't set all 3 related settings or if any of them are
set to 0 it will not do anything.

an other thing is the open file limits they must be set in two places
first is explained here
https://access.redhat.com/solutions/1257953#:~:text=service.,descriptors%....
The other is in the dse.ldif file
https://docs.kolab.org/administrator-guide/tweaking-389ds.html#:~:text=A%....

On Fri, Nov 11, 2022 at 11:37 PM Howard Chu <hyc(a)symas.com&gt; wrote:
>
> > Date: Fri, 11 Nov 2022 14:25:57 +0000
> > From: Tobias Ernstberger <tobias.ernstberger(a)de.ibm.com&gt;
>
> > Hello,
> >
> > we're observing the following error message:
> > "ERR - accept_and_configure - PR_Accept() failed, Netscape Portable Runtime
error -5971 (Process open FD table is full.)"
> > Looks like the file descriptors are exhausted, probably mainly used by incoming
TCP Connections (based on our investigation regarding open FDs).
> > We've set (and checked using the runtime information in /proc/PID/limits)
the ulimits and the nsslapd-maxdescriptors to many thousands (while having about 1000 open
connection regularly)
> >
> > We are investigating in multiple directions here, and have some questions - any
input is appreciated:
>
> Have you checked with e.g. lsof to see where all the FDs are being used? If you think
there's only 1000
> incoming connections, probably there are other things going on e.g. DNS reverse
lookups on client
> addresses, miscellaneous other files being opened, etc.
> >
> > 1) We acknowledge that exhausted FDs prevent additional connections to be
opened. But we also see, that existing connections are getting unusable, too. Is this a
known behaviour? Can this be avoided?
> > 2) Is there any chance to limit the number of open connections (lower than the
max FDs)? (trying to achieve that existing connections still work)
> > 3) What are best practice to prevent the ldap server from getting completely
useless (until restart) if a client opens many connections?
> > 4) Any additional remarks to prevent this situation?
>
> Fyi, OpenLDAP has no issue with multiple thousands of FDs being served
concurrently...
> >
> >
> > Kind regards
> >
> > Tobias Ernstberger
> > IBM Security
> >
> > IBM Deutschland GmbH
> > Vorsitzender des Aufsichtsrats: Sebastian Krause
> > Geschäftsführung: Gregor Pillen (Vorsitzender), Nicole Reimer, Gabriele
Schwarenthorer, Christine Rupp, Frank Theisen
> > Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, HRB
14562 / WEEE-Reg.-Nr. DE 99369940
> > https://www.ibm.com/privacy/us/en/
>
> --
>   -- Howard Chu
>   CTO, Symas Corp.           http://www.symas.com
>   Director, Highland Sun     http://highlandsun.com/hyc/
>   Chief Architect, OpenLDAP  http://www.openldap.org/project/
> _______________________________________________
> 389-users mailing list -- 389-users(a)lists.fedoraproject.org
> To unsubscribe send an email to 389-users-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/[e-mail]...
> Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue