Loading...

XML

Word

Printable

Type: Bug
Resolution: Done-Errata
Priority: Minor
Fix Version/s: rhel-9.5
Affects Version/s: rhel-8.3.0, rhel-9.3.0
Component/s: 389-ds-base
Labels:
- MigratedToJIRA
- Triaged

Fixed in Build:
389-ds-base-2.5.2-1.el9
Regression:
None
Severity:
Low

Pool Team:

sst_idm_ds
Sub-System Group:

ssg_idm

Internal Target Milestone:
25
Story Points:
0
ACKs Check:

QE ack
Target Version:

rhel-9.3.0
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Product Documentation Required:
None
Sprint:
None

Acceptance Criteria:

Hide

Run referenced test case and change num_users to 10000 to have faster results. Use DEBUGGING=True to keep at the end the generated ldif file. Run ldif2db manually as described in Steps to Reproduce. Compare performance.

Show
Run referenced test case and change num_users to 10000 to have faster results. Use DEBUGGING=True to keep at the end the generated ldif file. Run ldif2db manually as described in Steps to Reproduce. Compare performance.
Preliminary Testing:
Pass
Test Link:
https://polarion.engineering.redhat.com/polarion/redirect/project/RHDS/workitem?id=RHDS-14798
Errata Link:
https://errata.engineering.redhat.com/advisory/132195
Test Coverage:

Automated

Release Note Type:
Bug Fix
Release Note Text:

Hide
.Offline import of LDIF files now runs correctly

Previously, before an offline import the cache autotuning operation was not triggered. As a result, the import operation was slow when performed by the `ldif2db` script. With this update, Directory Server triggers the cache autotuning before the `ldif2db` operation increasing the import performance.

Show
.Offline import of LDIF files now runs correctly Previously, before an offline import the cache autotuning operation was not triggered. As a result, the import operation was slow when performed by the `ldif2db` script. With this update, Directory Server triggers the cache autotuning before the `ldif2db` operation increasing the import performance.
Release Note Status:
Done

Experience:
Architecture:

Unspecified
Bugzilla Bug:
RHBZ: 1837982

PX Impact Score:
PX Priority Data:
PX Review Complete:
SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Planning:
None

Description of problem:

When run from the test framework, ldif2db is very slow (average : 200 entries/sec for an ldif file containing 12000 entries).
Stangely, ldif2db run from the command line with the same ldif file is much faster (more than 2000 entries/sec)

Version-Release number of selected component (if applicable):
389-ds-base-1.4.3.8-2.module+el8.3.0+6591+ebfc9766.x86_64

How reproducible:
always

Steps to Reproduce:
1.Run automated test dirsrvtests/tests/suites/import/regression_test.py::test_large_ldif2db_ancestorid_index_creation, (changing num_users to 10000 to have faster results) with DEBUGGING=True to keep at the end the generated ldif file

(This test is currently under review as pr/51099)

2.Run ldif2db from the command line, using the ldif file created in the test above :

ldif2db -n test -i '/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif'

Actual results:
For step 1. (ldif2db started from the test framework) traces in the errors log show a very slow import rate:

[20/May/2020:06:00:25.820924564 -0400] - INFO - bdb_import_main - import test: Import complete. Processed 12047 entries in 62 seconds. (194.31 entries/sec)

For step 2. (ldif2db started from the cli) traces in the errors log show a much faster import rate for the same ldif file :
[20/May/2020:06:03:04.990715437 -0400] - INFO - bdb_import_main - import test: Import complete. Processed 12047 entries in 5 seconds. (2409.40 entries/sec)

Expected results:

Same behavior observed for ldif2db launched from the testframework and from the cli.
Performances shown using ldif2db in the test framework improved to get closer to the ones obtained with the cli

Additional info:
In the 1st case (from test framework), traces in the errors log for the ldif file processing :

[20/May/2020:05:59:24.030003867 -0400] - INFO - import_producer - import test: Processing file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif"
[20/May/2020:05:59:44.215667926 -0400] - INFO - import_monitor_threads - import test: Processed 4119 entries – average rate 196.1/sec, recent rate 196.1/sec, hit ratio 0%
[20/May/2020:06:00:04.687489324 -0400] - INFO - import_monitor_threads - import test: Processed 8118 entries – average rate 198.0/sec, recent rate 198.0/sec, hit ratio 100%
[20/May/2020:06:00:24.511326169 -0400] - INFO - import_producer - import test: Finished scanning file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif" (12047 entries)

Traces in the 2nd case (from cli)
[20/May/2020:06:02:59.413316634 -0400] - INFO - import_producer - import test: Processing file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif"
[20/May/2020:06:03:03.122115784 -0400] - INFO - import_producer - import test: Finished scanning file "/var/lib/dirsrv/slapd-standalone1/ldif/large_nested.ldif" (12047 entries)