-
Bug
-
Resolution: Done-Errata
-
Minor
-
rhel-8.3.0, rhel-9.3.0
-
389-ds-base-2.5.2-1.el9
-
None
-
Low
-
rhel-sst-idm-ds
-
ssg_idm
-
25
-
0
-
QE ack
-
False
-
-
None
-
None
-
-
Pass
-
Automated
-
Bug Fix
-
-
Done
-
-
Unspecified
-
None
Description of problem:
When run from the test framework, ldif2db is very slow (average : 200 entries/sec for an ldif file containing 12000 entries).
Stangely, ldif2db run from the command line with the same ldif file is much faster (more than 2000 entries/sec)
Version-Release number of selected component (if applicable):
389-ds-base-1.4.3.8-2.module+el8.3.0+6591+ebfc9766.x86_64
How reproducible:
always
Steps to Reproduce:
1.Run automated test dirsrvtests/tests/suites/import/regression_test.py::test_large_ldif2db_ancestorid_index_creation, (changing num_users to 10000 to have faster results) with DEBUGGING=True to keep at the end the generated ldif file
(This test is currently under review as pr/51099)
2.Run ldif2db from the command line, using the ldif file created in the test above :
- ldif2db -n test -i '/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif'
Actual results:
For step 1. (ldif2db started from the test framework) traces in the errors log show a very slow import rate:
[20/May/2020:06:00:25.820924564 -0400] - INFO - bdb_import_main - import test: Import complete. Processed 12047 entries in 62 seconds. (194.31 entries/sec)
For step 2. (ldif2db started from the cli) traces in the errors log show a much faster import rate for the same ldif file :
[20/May/2020:06:03:04.990715437 -0400] - INFO - bdb_import_main - import test: Import complete. Processed 12047 entries in 5 seconds. (2409.40 entries/sec)
Expected results:
Same behavior observed for ldif2db launched from the testframework and from the cli.
Performances shown using ldif2db in the test framework improved to get closer to the ones obtained with the cli
Additional info:
In the 1st case (from test framework), traces in the errors log for the ldif file processing :
[20/May/2020:05:59:24.030003867 -0400] - INFO - import_producer - import test: Processing file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif"
[20/May/2020:05:59:44.215667926 -0400] - INFO - import_monitor_threads - import test: Processed 4119 entries – average rate 196.1/sec, recent rate 196.1/sec, hit ratio 0%
[20/May/2020:06:00:04.687489324 -0400] - INFO - import_monitor_threads - import test: Processed 8118 entries – average rate 198.0/sec, recent rate 198.0/sec, hit ratio 100%
[20/May/2020:06:00:24.511326169 -0400] - INFO - import_producer - import test: Finished scanning file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif" (12047 entries)
Traces in the 2nd case (from cli)
[20/May/2020:06:02:59.413316634 -0400] - INFO - import_producer - import test: Processing file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif"
[20/May/2020:06:03:03.122115784 -0400] - INFO - import_producer - import test: Finished scanning file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif" (12047 entries)
- external trackers
- links to
-
RHSA-2024:132195 389-ds-base bug fix and enhancement update