[Fedora-directory-users] temporary resource unavailable problem with fedora directory server
Low Kian Seong
ipvx.low at gmail.com
Tue Feb 26 04:06:06 UTC 2008
This is what I found :
Access Log Analyzer 6.0
Command : logconv.pl -V /opt/fedora-ds/slapd-ldap1/logs/access
Processing 1 Access Log(s)...
Filename Total Lines Lines processed
---------------------------------------------------------------
/opt/fedora-ds/slapd-ldap1/logs/access 175820 175820
----------- Access Log Output ------------
Restarts: 0
Total Connections: 19774
Peak Concurrent Connections: 340
Total Operations: 65300
Total Results: 65320
Overall Performance: 100.0%
Searches: 38913
Modifications: 13
Adds: 2
Deletes: 0
Mod RDNs: 0
6.x Stats
Persistent Searches: 0
Internal Operations: 0
Entry Operations: 0
Extended Operations: 0
Abandoned Requests: 0
Smart Referrals Received: 0
VLV Operations: 0
VLV Unindexed Searches: 0
SORT Operations: 0
SSL Connections: 0
Entire Search Base Queries: 0
Unindexed Searches: 0
FDs Taken: 19774
FDs Returned: 19546
Highest FD Taken: 3348
Broken Pipes: 0
Connections Reset By Peer: 0
Resource Unavailable: 2187
- 2187 (T1) Idle Timeout Exceeded
Binds: 26372
Unbinds: 5877
LDAP v2 Binds: 659
LDAP v3 Binds: 25713
SSL Client Binds: 0
Failed SSL Client Binds: 0
SASL Binds: 0
Directory Manager Binds: 12909
Anonymous Binds: 2998
Other Binds: 10465
----- Connection Latency Details -----
(in seconds) <=1 2 3 4-5 6-10 11-15 >15
--------------------------------------------------------------------------
(# of connections) 9320 255 59 257 1082 129 8444
----- Current Open Connection IDs -----
268631
268632
268846
268850
268854
268894
268906
268946
268947
268948
268949
268950
268954
268956
268957
268959
268962
268963
268968
268973
268974
268977
268978
268979
268980
268981
268988
268990
268992
268993
268994
268999
269002
269004
269007
269012
269013
269014
269016
269029
269030
269031
269033
269036
269050
269055
269056
269057
269058
269062
269066
269067
269069
269071
269077
269079
269080
269082
269085
269086
269091
269093
269094
269095
269096
269098
269103
269104
269106
269107
269108
269110
269112
269116
269117
269119
269120
269122
269123
269124
269126
269153
269154
269332
269333
269435
269517
269793
270157
270193
270228
270281
270294
270565
270658
270726
270888
270940
270944
271201
271206
271210
271225
271235
271538
271552
272005
272054
272086
272127
272240
272252
272285
272494
272686
272728
272848
273004
273183
273201
273221
273222
273223
273268
273269
273326
273331
273340
273342
273353
273361
273405
273509
273725
273915
273916
273940
273953
274018
274027
274028
274034
274591
274677
274679
274721
274835
274836
274962
274991
275058
275120
275214
275489
275493
275609
275614
275651
275660
275692
275706
275809
275889
275920
275959
276144
276282
276348
276440
277346
277392
277661
277662
277665
277690
277695
277703
277704
277729
277809
277908
277922
278102
278197
278309
278316
278317
278366
278436
278517
278524
278632
278640
278651
278691
278806
278807
278872
279088
279089
279243
279371
279394
279601
279602
279849
279871
279873
279897
280004
280095
280304
280451
280452
280614
280733
280744
280824
280977
281069
281122
281216
281253
281254
281262
281367
281395
281452
281598
281672
281682
281804
281824
281895
281903
281913
281923
281958
281988
282000
282001
282016
282022
282210
282521
282525
282535
282569
282573
282574
282577
282580
282581
282611
282693
282989
283029
283030
283076
283232
283325
283371
283450
283493
283536
283595
283626
283631
283757
283770
283776
283793
283794
283796
283914
283916
283917
284096
284132
284160
284267
284280
284583
284742
285026
285027
285221
285233
285236
285250
285261
285436
285523
285544
285550
285585
285587
285591
285812
285845
285892
285894
286057
286060
286064
286065
286066
286067
286068
286512
286573
286724
286799
286810
286855
286901
287044
287049
287051
287134
287156
287263
287308
287496
287511
287587
287681
287682
287734
287741
287742
287747
287773
287778
287802
287805
287850
287939
288024
288058
288060
288062
288064
288068
288069
288071
288123
288127
288142
288195
288231
288236
288281
288307
288409
288410
288438
288598
288601
288628
288633
288650
288651
288653
288695
288702
288728
288744
288775
288799
288812
288813
288815
288825
288832
288833
288835
288837
288838
288841
288852
288886
288887
288890
288891
288892
288894
288895
288896
288899
----- Errors -----
err=0 64083 Successful Operations
err=32 1152 No Such Object
err=49 85 Invalid Credentials (Bad Password)
----- Top 20 Failed Logins ------
23 uid=salewati.damit at dss.com.bn,ou=people,ou=email,ou=dss.com.bn
,dc=simpur
9 uid=titi.yusop at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
9 uid=hanafadziliah.halidin at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
9 uid=salenawati,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
6 uid=noorulafiza.ishak at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
3 uid=mashadi.phtengah at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
3 uid=hashimah.mudjono at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
3 uid=pgsuriati.hidup at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
3 uid=ipvx, ou=people, ou=email, ou=simpur.net.bn,dc=simpur
3 uid=dennis.kong at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
3 uid=eng.noc at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
3 uid=hamdilah.kamaluddin at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
1 uid=koala,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
1 uid=teresa,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
1 uid=dorauhudt,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
1 uid=support, ou=people, ou=email, ou=simpur.net.bn,dc=simpur
1 uid=webmaster,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
1 uid=superman,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
1 uid=miles,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
1 uid=mark,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
>From the IP address(s) :
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.2
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.2
202.152.94.2
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.130
202.152.94.130
202.152.94.130
202.152.94.1
202.152.94.130
202.152.94.130
202.152.94.130
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.129
202.152.94.1
----- Total Connection Codes -----
B1 11480 Bad Ber Tag Encountered
U1 5877 Cleanly Closed Connections
T1 2187 Idle Timeout Exceeded
----- Top 20 Clients -----
Number of Clients: 8
13800 202.152.94.129
8254 - B1 Bad Ber Tag Encountered
3608 - U1 Cleanly Closed Connections
1864 - T1 Idle Timeout Exceeded
4300 202.152.94.130
3178 - B1 Bad Ber Tag Encountered
743 - U1 Cleanly Closed Connections
145 - T1 Idle Timeout Exceeded
678 202.152.94.1
535 - U1 Cleanly Closed Connections
142 - T1 Idle Timeout Exceeded
536 202.152.94.2
536 - U1 Cleanly Closed Connections
214 127.0.0.1
214 - U1 Cleanly Closed Connections
214 202.152.94.33
214 - U1 Cleanly Closed Connections
32 192.168.100.61
30 - T1 Idle Timeout Exceeded
* Unknown Host
48 - B1 Bad Ber Tag Encountered
27 - U1 Cleanly Closed Connections
6 - T1 Idle Timeout Exceeded
----- Top 20 Bind DN's -----
Number of Unique Bind DN's: 270
12909 cn=directory manager
2998 Anonymous Binds
1111 cn=admin
1056 uid=dst_noc,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
511 uid=nagios,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
308 uid=delai,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
220
uid=farihana.kayan at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
213 uid=choo,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
206 uid=domainadmin at dst-group.com,ou=dst-group.com,dc=simpur
196 uid=chua at dss.com.bn,ou=people,ou=email,ou=dss.com.bn ,dc=simpur
192
uid=dygmasyanti.mashhor at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
191 uid=nagios, ou=dial up, ou=simpur.net.bn, dc=simpur
140 uid=dstmm.content,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
140 uid=khai,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
132
uid=meleendawaty.saini at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
128
uid=dkhjhnorhartati.phismail at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
110
uid=rosniah.karia at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
103 uid=bismi,ou=people,ou=email,ou=simpur.net.bn,dc=simpur
100 uid=fizah at dss.com.bn,ou=people,ou=email,ou=dss.com.bn ,dc=simpur
100
uid=mashadi.phtengah at dst-group.com,ou=people,ou=email,ou=dst-group.com,dc=simpur
----- Top 20 Search Bases -----
Number of Unique Search Bases: 4
38131 dc=simpur
408 ou=dst-group.com,dc=simpur
354 ou=dial up, ou=simpur.net.bn, dc=simpur
20 root dse
----- Top 20 Search Filters -----
Number of Unique Search Filters: 1629
4904 (&(objectclass=shadowaccount)(uid=vmail))
2471 (&(objectclass=posixgroup)(memberuid=avs-smtp))
2471 (&(objectclass=posixaccount)(uid=avs-smtp))
2132 (&(objectclass=posixgroup)(memberuid=root))
2132 (&(objectclass=posixaccount)(uid=root))
2081 (&(objectclass=posixaccount)(uid=postfix))
2081 (&(objectclass=posixgroup)(memberuid=postfix))
1699 (uid=root)
1325 (objectclass=*)
1150
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=dst_noc))
530
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=nagios))
397
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=noc))
360
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=delai))
231
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=farihana.kayan at dst-group.com))
223
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=choo))
211
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=chua at dss.com.bn))
208 (&(objectclass=posixaccount)(uid=smsgw.smsgw))
208 (objectclass=posixaccount)
203
(&(objectclass=mailinetorgperson)(accountstatus=active)(uid=dstmm.content))
203 (uid=domainadmin at dst-group.com)
----- Top 20 Most Frequent etimes -----
65102 etime=0
158 etime=2
60 etime=1
----- Top 20 Longest etimes -----
etime=2 158
etime=1 60
etime=0 65102
----- Top 20 Largest nentries -----
nentries=1281 45
nentries=1279 152
nentries=502 26
nentries=501 3
nentries=500 1
nentries=61 7
nentries=60 1
nentries=56 2
nentries=52 2
nentries=50 2
nentries=39 2
nentries=31 1
nentries=29 2
nentries=28 1
nentries=27 2
nentries=20 208
nentries=17 3
nentries=16 1
nentries=12 3
nentries=11 2
----- Top 20 Most returned nentries -----
21625 nentries=0
16760 nentries=1
208 nentries=20
152 nentries=1279
45 nentries=1281
32 nentries=2
26 nentries=502
23 nentries=3
9 nentries=5
7 nentries=61
5 nentries=7
5 nentries=6
5 nentries=8
3 nentries=17
3 nentries=9
3 nentries=12
3 nentries=501
2 nentries=11
2 nentries=29
2 nentries=27
----- Top 20 Most Requested Attributes -----
31924 mailMessageStore
21300 userPassword
21097 uid
16193 cn
15982 mailQuotaSize
15962 enableSpamFolder
15962 smsAlertPkg
10348 All Attributes
6944 gidNumber
4904 shadowMax
4904 shadowWarning
4904 shadowInactive
4904 shadowLastChange
4904 shadowFlag
4904 shadowMin
4904 shadowExpire
354 radiusProfile
231 uidNumber
231 gecos
231 homeDirectory
----- Recommendations -----
1. You have some connections that are are being closed by the
idletimeout setting. You may want to increase the idletimeout if it is
set low.
2. You have a significant difference between binds and unbinds. You
may want to investigate this difference.
3. You have a high number of Directory Manager binds. The Directory
Manager account should only be used under certain circumstances.
Avoid using this account for client applications.
4. You have more abnormal connection codes than cleanly closed
connections. You may want to investigate this difference.
Any ideas ?
On Tue, Feb 26, 2008 at 11:30 AM, Rich Megginson <rmeggins at redhat.com> wrote:
> Low Kian Seong wrote:
> > In order to solve this problem I am looking at upgrading my version of
> > fedora-ds which is :
> >
> > fedora-ds-1.0.4-1.RHEL4
> >
> > to the latest version which is 1.1.
> >
> > Is there anything I need to look out for in this kind of upgrade ?
> >
> I don't think upgrading will solve your problem.
>
> The large number of 2743 (T1) Idle Timeout Exceeded errors is really
> strange. I think
> logconv.pl -V /opt/fedora-ds/slapd-ldap1/logs/access
> may provide more information about those connections. I'd like to see
> what the etime is for those operations. If those errors really are
> caused by some application just hanging attempting to read from the
> directory server, you might be able to increase the ioblocktimeout
> setting in the server, but that will work only if those operations are
> hanging.
>
>
> > Thank you in advance.
> >
> >
> > On Wed, Feb 20, 2008 at 12:20 PM, Low Kian Seong <ipvx.low at gmail.com> wrote:
> >
> >> this is what i got
> >>
> >>
> >> Access Log Analyzer 6.0
> >>
> >> Command : logconv.pl /opt/fedora-ds/slapd-ldap1/logs/access
> >>
> >> Processing 1 Access Log(s)...
> >>
> >> Filename Total Lines Lines processed
> >> ---------------------------------------------------------------
> >> /opt/fedora-ds/slapd-ldap1/logs/access 162024 162024
> >>
> >>
> >> ----------- Access Log Output ------------
> >>
> >> Restarts: 0
> >>
> >> Total Connections: 20511
> >> Peak Concurrent Connections: 278
> >> Total Operations: 57545
> >> Total Results: 57600
> >> Overall Performance: 100.0%
> >>
> >> Searches: 33737
> >> Modifications: 2
> >> Adds: 0
> >> Deletes: 0
> >> Mod RDNs: 0
> >>
> >> 6.x Stats
> >> Persistent Searches: 0
> >> Internal Operations: 0
> >> Entry Operations: 0
> >> Extended Operations: 0
> >> Abandoned Requests: 0
> >> Smart Referrals Received: 0
> >>
> >> VLV Operations: 3
> >> VLV Unindexed Searches: 1
> >> SORT Operations: 41
> >> SSL Connections: 0
> >>
> >> Entire Search Base Queries: 20
> >> Unindexed Searches: 43
> >>
> >> FDs Taken: 20511
> >> FDs Returned: 20277
> >> Highest FD Taken: 1404
> >>
> >> Broken Pipes: 0
> >> Connections Reset By Peer: 0
> >> Resource Unavailable: 2743
> >> - 2743 (T1) Idle Timeout Exceeded
> >>
> >> Binds: 23806
> >> Unbinds: 6044
> >>
> >> LDAP v2 Binds: 711
> >> LDAP v3 Binds: 23095
> >> SSL Client Binds: 0
> >> Failed SSL Client Binds: 0
> >> SASL Binds: 0
> >>
> >> Directory Manager Binds: 13372
> >> Anonymous Binds: 2670
> >> Other Binds: 7764
> >>
> >>
> >>
> >>
> >>
> >> On Feb 20, 2008 2:20 AM, Rich Megginson <rmeggins at redhat.com> wrote:
> >> > Low Kian Seong wrote:
> >> > > This is running on a rhel4 and during this time it doesn't respond to
> >> > > ldap queries.
> >> > >
> >> > run /opt/fedora-ds/bin/slapd/admin/bin/logconf.pl
> >> > /opt/fedora-ds/slapd-yourinstance/logs/access
> >> >
> >> > > On Feb 18, 2008 12:08 PM, Satish Chetty <satish at suburbia.org.au> wrote:
> >> > >
> >> > >> Low,
> >> > >> What is the load on the system? Also, when you see this error, does the
> >> > >> LDAP respond to any ldap queries (getent or ladpsearch)?
> >> > >>
> >> > >> -Satish.
> >> > >>
> >> > >>
> >> > >> Low Kian Seong wrote:
> >> > >>
> >> > >>> Dear all,
> >> > >>>
> >> > >>> I have installed fedora directory server version :
> >> > >>> fedora-ds-1.0.4-1.RHEL4. This ldap server integrates with postfix and
> >> > >>> our radius server. My problem is when I check the access log I see
> >> > >>> this error
> >> > >>>
> >> > >>> .[18/Feb/2008:11:04:51 +0800] conn=72887 op=-1 fd=593 closed error 11
> >> > >>> (Resource temporarily unavailable) - T1
> >> > >>> [18/Feb/2008:11:04:54 +0800] conn=72898 op=-1 fd=666 closed error 11
> >> > >>> (Resource temporarily unavailable) - T1
> >> > >>> [18/Feb/2008:11:05:22 +0800] conn=72895 op=-1 fd=605 closed error 11
> >> > >>> (Resource temporarily unavailable) - T1
> >> > >>>
> >> > >>> occuring again and again very frequently. I have already tuned the
> >> > >>> server according to the tuning guide on fedora directory server site.
> >> > >>> This is my sysctl.conf :
> >> > >>>
> >> > >>>
> >> > >>> # Kernel sysctl configuration file for Red Hat Linux
> >> > >>> #
> >> > >>> # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and
> >> > >>> # sysctl.conf(5) for more details.
> >> > >>>
> >> > >>> # Controls IP packet forwarding
> >> > >>> net.ipv4.ip_forward = 0
> >> > >>>
> >> > >>> # Controls source route verification
> >> > >>> net.ipv4.conf.default.rp_filter = 1
> >> > >>>
> >> > >>> # Do not accept source routing
> >> > >>> net.ipv4.conf.default.accept_source_route = 0
> >> > >>>
> >> > >>> # Controls the System Request debugging functionality of the kernel
> >> > >>> kernel.sysrq = 0
> >> > >>>
> >> > >>> # Controls whether core dumps will append the PID to the core filename.
> >> > >>> # Useful for debugging multi-threaded applications.
> >> > >>> kernel.core_uses_pid = 1
> >> > >>> net.ipv4.ip_local_port_range = 1024 65000
> >> > >>> fs.file-max = 128000
> >> > >>> net.ipv4.tcp_keepalive_time = 300
> >> > >>>
> >> > >>> Am I missing something that I haven't done ?
> >> > >>>
> >> > >>> --
> >> > >>> Fedora-directory-users mailing list
> >> > >>> Fedora-directory-users at redhat.com
> >> > >>> https://www.redhat.com/mailman/listinfo/fedora-directory-users
> >> > >>>
> >> > >>>
> >> > >> --
> >> > >> Fedora-directory-users mailing list
> >> > >> Fedora-directory-users at redhat.com
> >> > >> https://www.redhat.com/mailman/listinfo/fedora-directory-users
> >> > >>
> >> > >>
> >> > >
> >> > > --
> >> > > Fedora-directory-users mailing list
> >> > > Fedora-directory-users at redhat.com
> >> > > https://www.redhat.com/mailman/listinfo/fedora-directory-users
> >> > >
> >> >
> >> >
> >> > --
> >> > Fedora-directory-users mailing list
> >> > Fedora-directory-users at redhat.com
> >> > https://www.redhat.com/mailman/listinfo/fedora-directory-users
> >> >
> >> >
> >>
> >>
> >
> > --
> > Fedora-directory-users mailing list
> > Fedora-directory-users at redhat.com
> > https://www.redhat.com/mailman/listinfo/fedora-directory-users
> >
>
>
> --
> Fedora-directory-users mailing list
> Fedora-directory-users at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-directory-users
>
>
More information about the 389-users
mailing list