389-commits October 2019

389-commits@lists.fedoraproject.org

1 participants
28 discussions

[389-ds-base] 02/02: Issue 50634 - Clean up CLI errors output - Fix wrong exception
by pagure＠pagure.io 21 Oct '19

21 Oct '19

This is an automated email from the git hooks/post-receive script. spichugi pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. commit ea5eb49e70eb92b7cbf0a3a7452aafff295dc375 Author: Simon Pichugin <spichugi(a)redhat.com> AuthorDate: Mon Oct 21 18:25:20 2019 +0200 Issue 50634 - Clean up CLI errors output - Fix wrong exception Description: The previous commit takes care only about ValueError evaluation. But it is possible that other exceptions will be raised which will result in a wrong error output. Make the exception object more general. https://pagure.io/389-ds-base/issue/50634 Reviewed by: ? --- src/lib389/lib389/cli_base/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/lib389/lib389/cli_base/__init__.py b/src/lib389/lib389/cli_base/__init__.py index 2783cb2..245dc8f 100644 --- a/src/lib389/lib389/cli_base/__init__.py +++ b/src/lib389/lib389/cli_base/__init__.py @@ -415,6 +415,6 @@ def format_error_to_dict(exception): errmsg = str(exception) try: msg = ast.literal_eval(errmsg) - except ValueError: + except Exception: msg = {'desc': errmsg} return msg -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] 01/02: Issue 50634 - Clean up CLI errors output
by pagure＠pagure.io 21 Oct '19

21 Oct '19

This is an automated email from the git hooks/post-receive script. spichugi pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. commit 3798137c4b408debfc088f02da1df000e1790cc5 Author: Simon Pichugin <spichugi(a)redhat.com> AuthorDate: Thu Oct 17 13:46:56 2019 +0200 Issue 50634 - Clean up CLI errors output Description: CLI tools should print human easy readable messages if something went wrong. As discussed here: https://pagure.io/389-ds-base/pull-request/50624 Change the CLI error processing so the dict type is always transformed. https://pagure.io/389-ds-base/issue/50634 Reviewed by: mreynolds, mhonek (Thanks!) --- src/lib389/cli/dsconf | 12 +++++------- src/lib389/cli/dscreate | 4 +++- src/lib389/cli/dsctl | 11 +++++++---- src/lib389/cli/dsidm | 4 +++- src/lib389/lib389/cli_base/__init__.py | 20 ++++++++++++++++++++ 5 files changed, 38 insertions(+), 13 deletions(-) diff --git a/src/lib389/cli/dsconf b/src/lib389/cli/dsconf index 22635ca..7486dbe 100755 --- a/src/lib389/cli/dsconf +++ b/src/lib389/cli/dsconf @@ -15,7 +15,7 @@ import ldap import sys import signal import json -import ast +from lib389._constants import DSRC_HOME from lib389.cli_conf import config as cli_config from lib389.cli_conf import backend as cli_backend from lib389.cli_conf import directory_manager as cli_directory_manager @@ -33,6 +33,7 @@ from lib389.cli_conf import security as cli_security from lib389.cli_base import disconnect_instance, connect_instance from lib389.cli_base.dsrc import dsrc_to_ldap, dsrc_arg_concat from lib389.cli_base import setup_script_logger +from lib389.cli_base import format_error_to_dict parser = argparse.ArgumentParser(allow_abbrev=True) @@ -137,15 +138,12 @@ if __name__ == '__main__': log.info("Command successful.") except Exception as e: log.debug(e, exc_info=True) - errmsg = str(e) + msg = format_error_to_dict(e) + if args and args.json: - try: - msg = ast.literal_eval(errmsg) - except: - msg = {'desc': errmsg} sys.stderr.write(json.dumps(msg)) else: - log.error("Error: %s" % errmsg) + log.error("Error: %s" % " - ".join(msg.values())) result = False disconnect_instance(inst) diff --git a/src/lib389/cli/dscreate b/src/lib389/cli/dscreate index 4af852b..b516eb9 100755 --- a/src/lib389/cli/dscreate +++ b/src/lib389/cli/dscreate @@ -17,6 +17,7 @@ import signal from lib389 import DirSrv from lib389.cli_ctl import instance as cli_instance from lib389.cli_base import setup_script_logger +from lib389.cli_base import format_error_to_dict parser = argparse.ArgumentParser() parser.add_argument('-v', '--verbose', @@ -74,7 +75,8 @@ if __name__ == '__main__': result = args.func(inst, log, args) except Exception as e: log.debug(e, exc_info=True) - log.error("Error: %s" % str(e)) + msg = format_error_to_dict(e) + log.error("Error: %s" % " - ".join(msg.values())) result = False # Done! diff --git a/src/lib389/cli/dsctl b/src/lib389/cli/dsctl index ed8fc8f..04c83a8 100755 --- a/src/lib389/cli/dsctl +++ b/src/lib389/cli/dsctl @@ -21,6 +21,7 @@ from lib389 import DirSrv from lib389.cli_ctl import instance as cli_instance from lib389.cli_ctl import dbtasks as cli_dbtasks from lib389.cli_base import disconnect_instance, setup_script_logger +from lib389.cli_base import format_error_to_dict from lib389.cli_ctl.instance import instance_remove_all parser = argparse.ArgumentParser() @@ -105,10 +106,12 @@ if __name__ == '__main__': insts = inst.list(serverid=args.instance) except (PermissionError, IOError) as e: log.error("Unable to access instance information. Are you running as the correct user? (usually dirsrv or root)") - log.error("Error: %s" % str(e)) + msg = format_error_to_dict(e) + log.error("Error: %s" % " - ".join(msg.values())) sys.exit(1) except Exception as e: - log.error("Error: %s" % str(e)) + msg = format_error_to_dict(e) + log.error("Error: %s" % " - ".join(msg.values())) sys.exit(1) if len(insts) != 1: log.error("No such instance '%s'" % args.instance) @@ -122,11 +125,11 @@ if __name__ == '__main__': result = args.func(inst, log, args) except Exception as e: log.debug(e, exc_info=True) - log.error("Error: %s" % str(e)) + msg = format_error_to_dict(e) + log.error("Error: %s" % " - ".join(msg.values())) result = False disconnect_instance(inst) # Done! if result is False: sys.exit(1) - diff --git a/src/lib389/cli/dsidm b/src/lib389/cli/dsidm index 9e5b998..51ffb56 100755 --- a/src/lib389/cli/dsidm +++ b/src/lib389/cli/dsidm @@ -25,6 +25,7 @@ from lib389.cli_idm import user as cli_user from lib389.cli_idm import client_config as cli_client_config from lib389.cli_base import connect_instance, disconnect_instance, setup_script_logger from lib389.cli_base.dsrc import dsrc_to_ldap, dsrc_arg_concat +from lib389.cli_base import format_error_to_dict parser = argparse.ArgumentParser(allow_abbrev=True) @@ -129,7 +130,8 @@ if __name__ == '__main__': log.info("Command successful.") except Exception as e: log.debug(e, exc_info=True) - log.error("Error: %s" % str(e)) + msg = format_error_to_dict(e) + log.error("Error: %s" % " - ".join(msg.values())) result = False disconnect_instance(inst) diff --git a/src/lib389/lib389/cli_base/__init__.py b/src/lib389/lib389/cli_base/__init__.py index d139040..2783cb2 100644 --- a/src/lib389/lib389/cli_base/__init__.py +++ b/src/lib389/lib389/cli_base/__init__.py @@ -7,6 +7,7 @@ # See LICENSE for details. # --- END COPYRIGHT BLOCK --- +import ast import logging import sys import json @@ -398,3 +399,22 @@ def setup_script_logger(name, verbose=False): root.addHandler(log_handler) return log + + +def format_error_to_dict(exception): + """python-ldap str(exception) processing is not consistent. + This function makes sure that the result is dict + + :param exception: Exception you need to print + :type exception: ldap.LDAPError + :returns: dict + """ + + # The issue was raise on python-ldap repo: https://github.com/python-ldap/python-ldap/issues/304 + # We should fix the code here after the issue is fixed + errmsg = str(exception) + try: + msg = ast.literal_eval(errmsg) + except ValueError: + msg = {'desc': errmsg} + return msg -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.3.10 updated: Issue 50636 - Crash during sasl bind
by pagure＠pagure.io 17 Oct '19

17 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.3.10 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.3.10 by this push: new 66cfa2a Issue 50636 - Crash during sasl bind 66cfa2a is described below commit 66cfa2a47cd430f1cacc8363a091681edbf2602a Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Thu Oct 17 09:42:02 2019 -0400 Issue 50636 - Crash during sasl bind Bug Description: Sasl bind registers IO layers (sasl_IoMethods) that will be pushed (and called) by the next incoming operation. So the next incoming operation should synchronize itself with the sasl bind. Fix Description: The call to connection_call_io_layer_callbacks, that pushes registered methods, must hold c_mutex so that it let a pending sasl bind to fully register the methods. https://pagure.io/389-ds-base/issue/50636 Reviewed by: Ludwig Krispenz, Mark Reynolds --- ldap/servers/slapd/connection.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/ldap/servers/slapd/connection.c b/ldap/servers/slapd/connection.c index 845a670..9abd546 100644 --- a/ldap/servers/slapd/connection.c +++ b/ldap/servers/slapd/connection.c @@ -1585,12 +1585,14 @@ connection_threadmain() */ pb_conn->c_anonlimits_set = 1; } - PR_ExitMonitor(pb_conn->c_mutex); - + /* must hold c_mutex so that it synchronizes the IO layer push + * with a potential pending sasl bind that is registering the IO layer + */ if (connection_call_io_layer_callbacks(pb_conn)) { slapi_log_err(SLAPI_LOG_ERR, "connection_threadmain", "Could not add/remove IO layers from connection\n"); } + PR_ExitMonitor(pb_conn->c_mutex); break; default: break; -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.3.10 updated: Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.3.10 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.3.10 by this push: new 3be2a20 Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries 3be2a20 is described below commit 3be2a20eac317ee71945ea872db56e9bb0aaa32e Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 20:27:30 2019 -0400 Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries Bug Description: The logs from an LDIF import indicated that gathering non-leaf IDs for creating the ancestorid index took an enormous amount of time, over 10hrs. The root cause is that the parentid index btree ordering is lexical, but the IDList being built up from it is sorted numerically. In the existing code, the IDList is maintained in constantly sorted order by idl_insert(). Fix Description: ldbm_get_nonleaf_ids() switches to idl_append_extend() instead idl_insert() for building up the IDList and then sorts the result only once, using qsort with idl_sort_cmp, after the entire list has been gathered. The improvement on identical hardware is for the operation to take 10 seconds rather than 10 hours Patch Author: Thomas Lackey <telackey(a)bozemanpass.com> Thanks for the great contribution!!! relates: https://pagure.io/389-ds-base/issue/49850 Reviewed by: mreynolds, tbordaz, and firstyear --- ldap/servers/slapd/back-ldbm/ancestorid.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/back-ldbm/ancestorid.c b/ldap/servers/slapd/back-ldbm/ancestorid.c index 2464292..254a3aa 100644 --- a/ldap/servers/slapd/back-ldbm/ancestorid.c +++ b/ldap/servers/slapd/back-ldbm/ancestorid.c @@ -82,7 +82,14 @@ ldbm_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) ret = dbc->c_get(dbc, &key, &data, DB_NEXT_NODUP); if ((ret == 0) && (*(char *)key.data == EQ_PREFIX)) { id = (ID)strtoul((char *)key.data + 1, NULL, 10); - idl_insert(&nodes, id); + /* + * TEL 20180711 - switch to idl_append instead of idl_insert because there is no + * no need to keep the list constantly sorted, which can be very expensive with + * large databases (exacerbated by the fact that the parentid btree ordering is + * lexical, but the idl_insert ordering is numeric). It is enough to gather them + * all together and sort them once at the end. + */ + idl_append_extend(&nodes, id); } key_count++; if (!(key_count % PROGRESS_INTERVAL)) { @@ -107,6 +114,17 @@ ldbm_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) if (ret != 0) ldbm_nasty("ldbm_get_nonleaf_ids", sourcefile, 13030, ret); + if (ret == 0) { + /* now sort it */ + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Starting sort of ancestorid non-leaf IDs..."); + + qsort((void *)&nodes->b_ids[0], nodes->b_nids, (size_t)sizeof(ID), idl_sort_cmp); + + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Finished sort of ancestorid non-leaf IDs."); + } + out: /* Close the cursor */ if (dbc != NULL) { -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.0 by this push: new 9eba2ce Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries 9eba2ce is described below commit 9eba2ce6f94d8c406c18e5791430c6d7da131261 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 20:27:30 2019 -0400 Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries Bug Description: The logs from an LDIF import indicated that gathering non-leaf IDs for creating the ancestorid index took an enormous amount of time, over 10hrs. The root cause is that the parentid index btree ordering is lexical, but the IDList being built up from it is sorted numerically. In the existing code, the IDList is maintained in constantly sorted order by idl_insert(). Fix Description: ldbm_get_nonleaf_ids() switches to idl_append_extend() instead idl_insert() for building up the IDList and then sorts the result only once, using qsort with idl_sort_cmp, after the entire list has been gathered. The improvement on identical hardware is for the operation to take 10 seconds rather than 10 hours Patch Author: Thomas Lackey <telackey(a)bozemanpass.com> Thanks for the great contribution!!! relates: https://pagure.io/389-ds-base/issue/49850 Reviewed by: mreynolds, tbordaz, and firstyear --- ldap/servers/slapd/back-ldbm/ancestorid.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/back-ldbm/ancestorid.c b/ldap/servers/slapd/back-ldbm/ancestorid.c index 2464292..254a3aa 100644 --- a/ldap/servers/slapd/back-ldbm/ancestorid.c +++ b/ldap/servers/slapd/back-ldbm/ancestorid.c @@ -82,7 +82,14 @@ ldbm_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) ret = dbc->c_get(dbc, &key, &data, DB_NEXT_NODUP); if ((ret == 0) && (*(char *)key.data == EQ_PREFIX)) { id = (ID)strtoul((char *)key.data + 1, NULL, 10); - idl_insert(&nodes, id); + /* + * TEL 20180711 - switch to idl_append instead of idl_insert because there is no + * no need to keep the list constantly sorted, which can be very expensive with + * large databases (exacerbated by the fact that the parentid btree ordering is + * lexical, but the idl_insert ordering is numeric). It is enough to gather them + * all together and sort them once at the end. + */ + idl_append_extend(&nodes, id); } key_count++; if (!(key_count % PROGRESS_INTERVAL)) { @@ -107,6 +114,17 @@ ldbm_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) if (ret != 0) ldbm_nasty("ldbm_get_nonleaf_ids", sourcefile, 13030, ret); + if (ret == 0) { + /* now sort it */ + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Starting sort of ancestorid non-leaf IDs..."); + + qsort((void *)&nodes->b_ids[0], nodes->b_nids, (size_t)sizeof(ID), idl_sort_cmp); + + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Finished sort of ancestorid non-leaf IDs."); + } + out: /* Close the cursor */ if (dbc != NULL) { -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.1 updated: Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.1 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.1 by this push: new fb4ea1d Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries fb4ea1d is described below commit fb4ea1da83eb0138af5e4ee62d34dcc6ed5539b0 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 20:27:30 2019 -0400 Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries Bug Description: The logs from an LDIF import indicated that gathering non-leaf IDs for creating the ancestorid index took an enormous amount of time, over 10hrs. The root cause is that the parentid index btree ordering is lexical, but the IDList being built up from it is sorted numerically. In the existing code, the IDList is maintained in constantly sorted order by idl_insert(). Fix Description: ldbm_get_nonleaf_ids() switches to idl_append_extend() instead idl_insert() for building up the IDList and then sorts the result only once, using qsort with idl_sort_cmp, after the entire list has been gathered. The improvement on identical hardware is for the operation to take 10 seconds rather than 10 hours Patch Author: Thomas Lackey <telackey(a)bozemanpass.com> Thanks for the great contribution!!! relates: https://pagure.io/389-ds-base/issue/49850 Reviewed by: mreynolds, tbordaz, and firstyear --- ldap/servers/slapd/back-ldbm/ancestorid.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/back-ldbm/ancestorid.c b/ldap/servers/slapd/back-ldbm/ancestorid.c index 2464292..254a3aa 100644 --- a/ldap/servers/slapd/back-ldbm/ancestorid.c +++ b/ldap/servers/slapd/back-ldbm/ancestorid.c @@ -82,7 +82,14 @@ ldbm_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) ret = dbc->c_get(dbc, &key, &data, DB_NEXT_NODUP); if ((ret == 0) && (*(char *)key.data == EQ_PREFIX)) { id = (ID)strtoul((char *)key.data + 1, NULL, 10); - idl_insert(&nodes, id); + /* + * TEL 20180711 - switch to idl_append instead of idl_insert because there is no + * no need to keep the list constantly sorted, which can be very expensive with + * large databases (exacerbated by the fact that the parentid btree ordering is + * lexical, but the idl_insert ordering is numeric). It is enough to gather them + * all together and sort them once at the end. + */ + idl_append_extend(&nodes, id); } key_count++; if (!(key_count % PROGRESS_INTERVAL)) { @@ -107,6 +114,17 @@ ldbm_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) if (ret != 0) ldbm_nasty("ldbm_get_nonleaf_ids", sourcefile, 13030, ret); + if (ret == 0) { + /* now sort it */ + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Starting sort of ancestorid non-leaf IDs..."); + + qsort((void *)&nodes->b_ids[0], nodes->b_nids, (size_t)sizeof(ID), idl_sort_cmp); + + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Finished sort of ancestorid non-leaf IDs."); + } + out: /* Close the cursor */ if (dbc != NULL) { -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch master updated: Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch master in repository 389-ds-base. The following commit(s) were added to refs/heads/master by this push: new fc47620 Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries fc47620 is described below commit fc47620855e21c631e8346f734221ad08601b5a1 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 19:50:04 2019 -0400 Issue 49850 - ldbm_get_nonleaf_ids() slow for databases with many non-leaf entries Bug Description: The logs from an LDIF import indicated that gathering non-leaf IDs for creating the ancestorid index took an enormous amount of time, over 10hrs. The root cause is that the parentid index btree ordering is lexical, but the IDList being built up from it is sorted numerically. In the existing code, the IDList is maintained in constantly sorted order by idl_insert(). Fix Description: ldbm_get_nonleaf_ids() switches to idl_append_extend() instead idl_insert() for building up the IDList and then sorts the result only once, using qsort with idl_sort_cmp, after the entire list has been gathered. The improvement on identical hardware is for the operation to take 10 seconds rather than 10 hours Patch Author: Thomas Lackey <telackey(a)bozemanpass.com> Thanks for the great contribution!!! relates: https://pagure.io/389-ds-base/issue/49850 Reviewed by: mreynolds, tbordaz, and firstyear (Thanks!) --- ldap/servers/slapd/back-ldbm/db-bdb/bdb_import.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_import.c b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_import.c index 2fd7524..79c9c7c 100644 --- a/ldap/servers/slapd/back-ldbm/db-bdb/bdb_import.c +++ b/ldap/servers/slapd/back-ldbm/db-bdb/bdb_import.c @@ -457,7 +457,14 @@ bdb_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) ret = dbc->c_get(dbc, &key, &data, DB_NEXT_NODUP); if ((ret == 0) && (*(char *)key.data == EQ_PREFIX)) { id = (ID)strtoul((char *)key.data + 1, NULL, 10); - idl_insert(&nodes, id); + /* + * TEL 20180711 - switch to idl_append instead of idl_insert because there is no + * no need to keep the list constantly sorted, which can be very expensive with + * large databases (exacerbated by the fact that the parentid btree ordering is + * lexical, but the idl_insert ordering is numeric). It is enough to gather them + * all together and sort them once at the end. + */ + idl_append_extend(&nodes, id); } key_count++; if (!(key_count % PROGRESS_INTERVAL)) { @@ -482,6 +489,17 @@ bdb_get_nonleaf_ids(backend *be, DB_TXN *txn, IDList **idl, ImportJob *job) if (ret != 0) ldbm_nasty("bdb_get_nonleaf_ids", sourcefile, 13030, ret); + if (ret == 0) { + /* now sort it */ + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Starting sort of ancestorid non-leaf IDs..."); + + qsort((void *)&nodes->b_ids[0], nodes->b_nids, (size_t)sizeof(ID), idl_sort_cmp); + + import_log_notice(job, SLAPI_LOG_INFO, "ldbm_get_nonleaf_ids", + "Finished sort of ancestorid non-leaf IDs."); + } + out: /* Close the cursor */ if (dbc != NULL) { -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.3.10 updated: Issue 50655 - access log etime is not properly formatted
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.3.10 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.3.10 by this push: new 3fbff08 Issue 50655 - access log etime is not properly formatted 3fbff08 is described below commit 3fbff0870ae1bac549c12e40df6d2d0da2e1e7e2 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 16:52:59 2019 -0400 Issue 50655 - access log etime is not properly formatted Description: The wrong printf format was used for displaying the nanosecond etime in the access log. relates: https://pagure.io/389-ds-base/issue/50655 Reviewed by: firstyear(Thanks!) --- ldap/servers/slapd/result.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/result.c b/ldap/servers/slapd/result.c index 34ddd85..61e7a70 100644 --- a/ldap/servers/slapd/result.c +++ b/ldap/servers/slapd/result.c @@ -1925,7 +1925,8 @@ log_result(Slapi_PBlock *pb, Operation *op, int err, ber_tag_t tag, int nentries struct timespec o_hr_time_end; slapi_operation_time_elapsed(op, &o_hr_time_end); - snprintf(etime, ETIME_BUFSIZ, "%" PRId64 ".%010" PRId64 "", (int64_t)o_hr_time_end.tv_sec, (int64_t)o_hr_time_end.tv_nsec); + + snprintf(etime, ETIME_BUFSIZ, "%" PRId64 ".%.09" PRId64 "", (int64_t)o_hr_time_end.tv_sec, (int64_t)o_hr_time_end.tv_nsec); slapi_pblock_get(pb, SLAPI_OPERATION_NOTES, &operation_notes); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.0 updated: Issue 50655 - access log etime is not properly formatted
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.0 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.0 by this push: new 13340ad Issue 50655 - access log etime is not properly formatted 13340ad is described below commit 13340add262ad65ecc28e94c22ed277c622a2ff2 Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 16:52:59 2019 -0400 Issue 50655 - access log etime is not properly formatted Description: The wrong printf format was used for displaying the nanosecond etime in the access log. relates: https://pagure.io/389-ds-base/issue/50655 Reviewed by: firstyear(Thanks!) --- ldap/servers/slapd/result.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/result.c b/ldap/servers/slapd/result.c index 4f9bda0..0e4850e 100644 --- a/ldap/servers/slapd/result.c +++ b/ldap/servers/slapd/result.c @@ -1931,7 +1931,8 @@ log_result(Slapi_PBlock *pb, Operation *op, int err, ber_tag_t tag, int nentries struct timespec o_hr_time_end; slapi_operation_time_elapsed(op, &o_hr_time_end); - snprintf(etime, ETIME_BUFSIZ, "%" PRId64 ".%010" PRId64 "", (int64_t)o_hr_time_end.tv_sec, (int64_t)o_hr_time_end.tv_nsec); + + snprintf(etime, ETIME_BUFSIZ, "%" PRId64 ".%.09" PRId64 "", (int64_t)o_hr_time_end.tv_sec, (int64_t)o_hr_time_end.tv_nsec); slapi_pblock_get(pb, SLAPI_OPERATION_NOTES, &operation_notes); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

[389-ds-base] branch 389-ds-base-1.4.1 updated: Issue 50655 - access log etime is not properly formatted
by pagure＠pagure.io 16 Oct '19

16 Oct '19

This is an automated email from the git hooks/post-receive script. mreynolds pushed a commit to branch 389-ds-base-1.4.1 in repository 389-ds-base. The following commit(s) were added to refs/heads/389-ds-base-1.4.1 by this push: new 3b47e3f Issue 50655 - access log etime is not properly formatted 3b47e3f is described below commit 3b47e3f904f9e993cf962a262cfb9a2760ca635c Author: Mark Reynolds <mreynolds(a)redhat.com> AuthorDate: Wed Oct 16 16:52:59 2019 -0400 Issue 50655 - access log etime is not properly formatted Description: The wrong printf format was used for displaying the nanosecond etime in the access log. relates: https://pagure.io/389-ds-base/issue/50655 Reviewed by: firstyear(Thanks!) --- ldap/servers/slapd/result.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/ldap/servers/slapd/result.c b/ldap/servers/slapd/result.c index 22522d7..bda44a8 100644 --- a/ldap/servers/slapd/result.c +++ b/ldap/servers/slapd/result.c @@ -1984,7 +1984,8 @@ log_result(Slapi_PBlock *pb, Operation *op, int err, ber_tag_t tag, int nentries struct timespec o_hr_time_end; slapi_operation_time_elapsed(op, &o_hr_time_end); - snprintf(etime, ETIME_BUFSIZ, "%" PRId64 ".%010" PRId64 "", (int64_t)o_hr_time_end.tv_sec, (int64_t)o_hr_time_end.tv_nsec); + + snprintf(etime, ETIME_BUFSIZ, "%" PRId64 ".%.09" PRId64 "", (int64_t)o_hr_time_end.tv_sec, (int64_t)o_hr_time_end.tv_nsec); operation_notes = slapi_pblock_get_operation_notes(pb); -- To stop receiving notification emails like this one, please contact the administrator of this repository.

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-commits October 2019