10:58 a.m.
We are trying to reindex RHDS 9.1 after importing an updated index. Services were
stopped. Started the /usr/lib64/dirsrv/slapd-instance/db2index.
Reindex starts but then consistently reports:
Processed 300,000 entries ( pass 1)
Processed 300,000 entries ( pass 2)
and keeps repeating that sequence with same amount of entries. How long does it usually
take to complete the process and is there a limit or issue with this feature? This
happened once before and system was left overnight performing indexing only to still
processing with new passes. We ended up breaking the process and just reinitializing the
server.
Thank you,
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
10:58 a.m.
On 01/30/2014 09:58 AM, Paul Whitney wrote:
We are trying to reindex RHDS 9.1 after importing an updated index.
Services were stopped. Started the
/usr/lib64/dirsrv/slapd-instance/db2index.
Reindex starts but then consistently reports:
Processed 300,000 entries ( pass 1)
Processed 300,000 entries ( pass 2)
and keeps repeating that sequence with same amount of entries. How
long does it usually take to complete the process
Not this long.
and is there a limit or issue with this feature?
No.
This happened once before and system was left overnight performing
indexing only to still processing with new passes. We ended up
breaking the process and just reinitializing the server.
rpm -q 389-ds-base
Any messages in the errors log?
Thank you,
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
11:17 a.m.
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate
15283456.5/sec, recent rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass 11,
sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or actually
performing work ?
Could you note and post here the machine load in the sulking period --
CPU, I/O stats, e.g the output from "iostat -x 1" ?
Also, if you could grab some process thread stack captures (use "pstack"
, or gdb) from the indexing process and post the highlights here, that
might give some insight into what's going on.
The output from "strace -p xxx" run on the indexing process could also
be useful, but probably less useful than the information mentioned above.
11:18 a.m.
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate 15283456.5/sec, recent
rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass 11, sweeps files for
merging later, then restarts with pass 12.
There do not appear to be any glaring errors, just a constant processing and increment in
pass number.
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
Cell: 410.493.9448
On Jan 30, 2014, at 11:57 AM, Rich Megginson <rmeggins(a)redhat.com> wrote:
On 01/30/2014 09:58 AM, Paul Whitney wrote:
We are trying to reindex RHDS 9.1 after importing an updated index. Services were
stopped. Started the /usr/lib64/dirsrv/slapd-instance/db2index.
Reindex starts but then consistently reports:
Processed 300,000 entries ( pass 1)
Processed 300,000 entries ( pass 2)
and keeps repeating that sequence with same amount of entries. How long does it usually
take to complete the process
Not this long.
and is there a limit or issue with this feature?
No.
This happened once before and system was left overnight performing indexing only to still
processing with new passes. We ended up breaking the process and just reinitializing the
server.
rpm -q 389-ds-base
Any messages in the errors log?
Thank you,
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
11:31 a.m.
On 01/30/2014 10:17 AM, David Boreham wrote:
On 1/30/2014 10:18 AM, Paul Whitney wrote:
> rpm -q 389-ds-base
>
> 389-ds-base-1.2.11.15-30.el6_5.x86_64
>
> No errors, just a status:
>
> reindex userRoot: Processed 315000 entries (pass 11) -- avg rate
> 15283456.5/sec, recent rate 0.0/sec. hit ration 0%
>
> Then errors log states threshold has dropped belo 50%, ending pass
> 11, sweeps files for merging later, then restarts with pass 12.
>
First thing to determine is : is it "stuck" for some reason, or
actually performing work ?
Could you note and post here the machine load in the sulking period --
CPU, I/O stats, e.g the output from "iostat -x 1" ?
Also, if you could grab some process thread stack captures (use
"pstack" , or gdb) from the indexing process and post the highlights
here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions:
http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also
be useful, but probably less useful than the information mentioned above.
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
12:33 p.m.
On 01/30/2014 11:33 AM, Paul Whitney wrote:
Guys I appreciate you help in this issue. I unfortunately am hosting
on a disconnected network and cannot post any of the information you
are requesting without in essence "re-typing" it all in. In the
interest of saving time, I am just reinitializing the directory
server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2.
But on both RHDS 9.0 and 9.1, it appears to get stuck. Even after
killing the process, the server attmepts to resume the process but
makes no progress.
If we cannot get large logs or debug traces, how do you suggest going
about solving this problem?
Can you give us your databases and index configuration?
I know for a fact that db2index works on RHDS 9.1, with very large
databases. So the problem them becomes - what is it about your data
and/or configuration that is causing problems?
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson <rmeggins(a)redhat.com> wrote:
> On 01/30/2014 10:17 AM, David Boreham wrote:
>> On 1/30/2014 10:18 AM, Paul Whitney wrote:
>>> rpm -q 389-ds-base
>>> 389-ds-base-1.2.11.15-30.el6_5.x86_64
>>> No errors, just a status:
>>> reindex userRoot: Processed 315000 entries (pass 11) -- avg rate
>>> 15283456.5/sec, recent rate 0.0/sec. hit ration 0%
>>> Then errors log states threshold has dropped belo 50%, ending pass
>>> 11, sweeps files for merging later, then restarts with pass 12.
>> First thing to determine is : is it "stuck" for some reason, or
>> actually performing work ?
>> Could you note and post here the machine load in the sulking period --
>> CPU, I/O stats, e.g the output from "iostat -x 1" ?
>> Also, if you could grab some process thread stack captures (use
>> "pstack" , or gdb) from the indexing process and post the highlights
>> here, that might give some insight into what's going on.
>
> Yes, using the Debugging_Hangs instructions:
> http://port389.org/wiki/FAQ#Debugging_Hangs
>
>> The output from "strace -p xxx" run on the indexing process could also
>> be useful, but probably less useful than the information mentioned
>> above.
>> --
>> 389 users mailing list
>> 389-users(a)lists.fedoraproject.org
>> <mailto:389-users@lists.fedoraproject.org>
>> https://admin.fedoraproject.org/mailman/listinfo/389-users
>
> --
> 389 users mailing list
> 389-users(a)lists.fedoraproject.org
> <mailto:389-users@lists.fedoraproject.org>
> https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
12:33 p.m.
Guys I appreciate you help in this issue. I unfortunately am hosting on a disconnected
network and cannot post any of the information you are requesting without in essence
"re-typing" it all in. In the interest of saving time, I am just reinitializing
the directory server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2. But on both RHDS
9.0 and 9.1, it appears to get stuck. Even after killing the process, the server attmepts
to resume the process but makes no progress.
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson <rmeggins(a)redhat.com> wrote:
On 01/30/2014 10:17 AM, David Boreham wrote:
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate
15283456.5/sec, recent rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass
11, sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or
actually performing work ?
Could you note and post here the machine load in the sulking period --
CPU, I/O stats, e.g the output from "iostat -x 1" ?
Also, if you could grab some process thread stack captures (use
"pstack" , or gdb) from the indexing process and post the highlights
here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions:
http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also
be useful, but probably less useful than the information mentioned above.
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
12:57 p.m.
That is a very good question. No cannot provide any of those items. I was just hoping
maybe there would be some "known undocumented trick" to kickstart this server.
Perhaps the next time Kent L. comes out here he can take a look and see if there is
anything that stands out.
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
On Jan 30, 2014, at 01:32 PM, Rich Megginson <rmeggins(a)redhat.com> wrote:
On 01/30/2014 11:33 AM, Paul Whitney wrote:
Guys I appreciate you help in this issue. I unfortunately am hosting on a disconnected
network and cannot post any of the information you are requesting without in essence
"re-typing" it all in. In the interest of saving time, I am just reinitializing
the directory server. In the past this has worked.
For the record, this db2index function does work fine on the RHDS 8.2. But on both RHDS
9.0 and 9.1, it appears to get stuck. Even after killing the process, the server attmepts
to resume the process but makes no progress.
If we cannot get large logs or debug traces, how do you suggest going about solving this
problem?
Can you give us your databases and index configuration?
I know for a fact that db2index works on RHDS 9.1, with very large databases. So the
problem them becomes - what is it about your data and/or configuration that is causing
problems?
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
On Jan 30, 2014, at 12:48 PM, Rich Megginson <rmeggins(a)redhat.com> wrote:
On 01/30/2014 10:17 AM, David Boreham wrote:
On 1/30/2014 10:18 AM, Paul Whitney wrote:
rpm -q 389-ds-base
389-ds-base-1.2.11.15-30.el6_5.x86_64
No errors, just a status:
reindex userRoot: Processed 315000 entries (pass 11) -- avg rate
15283456.5/sec, recent rate 0.0/sec. hit ration 0%
Then errors log states threshold has dropped belo 50%, ending pass
11, sweeps files for merging later, then restarts with pass 12.
First thing to determine is : is it "stuck" for some reason, or
actually performing work ?
Could you note and post here the machine load in the sulking period --
CPU, I/O stats, e.g the output from "iostat -x 1" ?
Also, if you could grab some process thread stack captures (use
"pstack" , or gdb) from the indexing process and post the highlights
here, that might give some insight into what's going on.
Yes, using the Debugging_Hangs instructions:
http://port389.org/wiki/FAQ#Debugging_Hangs
The output from "strace -p xxx" run on the indexing process could also
be useful, but probably less useful than the information mentioned above.
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
4:34 a.m.
Hi,
I remeber s similar problem (with a similar DS), where import was
looping on the same entry and just starting a new pass again. It was a
problem in the import queue regulation, but to closer debug this we
would neeed more data. What you could do is to try to change the cache
sizes, which can have an effect on buffer size and hit ratio.
Regards,
Ludwig
On 01/30/2014 07:57 PM, Paul Whitney wrote:
That is a very good question. No cannot provide any of those items.
I was just hoping maybe there would be some "known undocumented
trick" to kickstart this server. Perhaps the next time Kent L. comes
out here he can take a look and see if there is anything that stands out.
Paul M. Whitney
E-mail: paul.whitney(a)mac.com
On Jan 30, 2014, at 01:32 PM, Rich Megginson <rmeggins(a)redhat.com> wrote:
> On 01/30/2014 11:33 AM, Paul Whitney wrote:
>> Guys I appreciate you help in this issue. I unfortunately am
>> hosting on a disconnected network and cannot post any of the
>> information you are requesting without in essence "re-typing" it all
>> in. In the interest of saving time, I am just reinitializing the
>> directory server. In the past this has worked.
>>
>> For the record, this db2index function does work fine on the RHDS
>> 8.2. But on both RHDS 9.0 and 9.1, it appears to get stuck. Even
>> after killing the process, the server attmepts to resume the process
>> but makes no progress.
>
> If we cannot get large logs or debug traces, how do you suggest going
> about solving this problem?
> Can you give us your databases and index configuration?
>
> I know for a fact that db2index works on RHDS 9.1, with very large
> databases. So the problem them becomes - what is it about your data
> and/or configuration that is causing problems?
>
>>
>> Paul M. Whitney
>> E-mail:paul.whitney@mac.com
>>
>>
>>
>>
>>
>> On Jan 30, 2014, at 12:48 PM, Rich Megginson <rmeggins(a)redhat.com>
>> wrote:
>>
>>> On 01/30/2014 10:17 AM, David Boreham wrote:
>>>> On 1/30/2014 10:18 AM, Paul Whitney wrote:
>>>>> rpm -q 389-ds-base
>>>>> 389-ds-base-1.2.11.15-30.el6_5.x86_64
>>>>> No errors, just a status:
>>>>> reindex userRoot: Processed 315000 entries (pass 11) -- avg rate
>>>>> 15283456.5/sec, recent rate 0.0/sec. hit ration 0%
>>>>> Then errors log states threshold has dropped belo 50%, ending pass
>>>>> 11, sweeps files for merging later, then restarts with pass 12.
>>>> First thing to determine is : is it "stuck" for some reason,
or
>>>> actually performing work ?
>>>> Could you note and post here the machine load in the sulking period --
>>>> CPU, I/O stats, e.g the output from "iostat -x 1" ?
>>>> Also, if you could grab some process thread stack captures (use
>>>> "pstack" , or gdb) from the indexing process and post the
highlights
>>>> here, that might give some insight into what's going on.
>>>
>>> Yes, using the Debugging_Hangs instructions:
>>> http://port389.org/wiki/FAQ#Debugging_Hangs
>>>
>>>> The output from "strace -p xxx" run on the indexing process
could also
>>>> be useful, but probably less useful than the information mentioned
>>>> above.
>>>> --
>>>> 389 users mailing list
>>>> 389-users(a)lists.fedoraproject.org
>>>> <mailto:389-users@lists.fedoraproject.org>
>>>> https://admin.fedoraproject.org/mailman/listinfo/389-users
>>>
>>> --
>>> 389 users mailing list
>>> 389-users(a)lists.fedoraproject.org
>>> <mailto:389-users@lists.fedoraproject.org>
>>> https://admin.fedoraproject.org/mailman/listinfo/389-users
>>
>>
>> --
>> 389 users mailing list
>> 389-users(a)lists.fedoraproject.org
>> https://admin.fedoraproject.org/mailman/listinfo/389-users
>
> --
> 389 users mailing list
> 389-users(a)lists.fedoraproject.org
> <mailto:389-users@lists.fedoraproject.org>
> https://admin.fedoraproject.org/mailman/listinfo/389-users
--
389 users mailing list
389-users(a)lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
3738
days inactive
3739
days old
389-users@lists.fedoraproject.org
8 comments
4 participants
participants (4)
-
David Boreham -
Ludwig Krispenz -
Paul Whitney -
Rich Megginson