Uploaded image for project: 'Red Hat Directory Server'
  1. Red Hat Directory Server
  2. DIRSRV-75

RHDS-11.7.6 memory leak investigation

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • None
    • dirsrv-11.5
    • 389-ds-base
    • None
    • None
    • rhel-sst-idm-ds
    • 0
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None

      Description of problem:

      "dirsrv service getting killed by oom-killer along with multiple restarts on RHDS 11.7.6"

      ASAN reports with small findings, the top one is related to secure connections with:

      ==2688323==ERROR: LeakSanitizer: detected memory leaks

      Direct leak of 4097 byte(s) in 1 object(s) allocated from:
      #0 0x7fe81c7a5bb8 in __interceptor_malloc (/usr/lib64/libasan.so.5+0xefbb8)
      #1 0x7fe81c6fa5f8 (/usr/lib64/libasan.so.5+0x445f8)
      #2 0x7fe808640fbd (<unknown module>)
      #3 0x7fe8085c1064 (<unknown module>)
      #4 0x7fe8085b4064 (<unknown module>)
      #5 0x7fe8085b464f (<unknown module>)
      #6 0x7fe8085b5516 (<unknown module>)
      #7 0x7fe8085b9784 (<unknown module>)
      #8 0x7fe81964fa76 (/lib64/libnss3.so+0x4da76)
      #9 0x7fe81965db8c in SECMOD_LoadModule (/lib64/libnss3.so+0x5bb8c)
      #10 0x7fe81965dcd7 in SECMOD_LoadModule (/lib64/libnss3.so+0x5bcd7)
      #11 0x7fe81962112b (/lib64/libnss3.so+0x1f12b)
      #12 0x7fe8196215f5 in NSS_Initialize (/lib64/libnss3.so+0x1f5f5)
      #13 0x7fe81c3d1a3d in slapd_nss_init (/usr/lib64/dirsrv/libslapd.so.0+0x115a3d)
      #14 0x562f4934ea27 in slapd_do_all_nss_ssl_init ldap/servers/slapd/main.c:2921
      #15 0x562f493499b4 in detach ldap/servers/slapd/detach.c:178
      #16 0x562f49339ac0 (/usr/sbin/ns-slapd+0x12ac0)
      #17 0x7fe81aa00d84 in __libc_start_main (/lib64/libc.so.6+0x3ad84)
      #18 0x562f4933bb9d (/usr/sbin/ns-slapd+0x14b9d)
      ...

      Version-Release number of selected component (if applicable):

      389-ds-base-1.4.3.34-3.module+el8dsrv+21391+b62d2223.x86_64 Tue Mar 26 09:36:05 2024
      nspr-4.35.0-1.el8_8.x86_64 Wed Mar 13 19:19:15 2024
      nss-3.90.0-4.el8_8.x86_64 Wed Mar 13 19:20:42 2024
      redhat-release-8.8-0.8.el8.x86_64 Wed Mar 13 19:19:13 2024
      redhat-release-eula-8.8-0.8.el8.x86_64 Wed Mar 13 19:19:13 2024
      sssd-2.9.1-2.el8.x86_64 Wed Mar 13 19:21:04 2024

      How reproducible:
      N/A
      situation happens once to twice a day in customer's environment

      Steps to Reproduce:
      1. N/A
      2.
      3.

      Actual results:

      Mar 21 19:30:29 editedlab807vm5 kernel: Out of memory: Killed process 1439999 (ns-slapd) total-vm:11498036kB, anon-rss:4745504kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:17024kB oom_score_adj:0
      Mar 22 15:57:44 editedlab807vm5 kernel: Out of memory: Killed process 2814619 (ns-slapd) total-vm:11497124kB, anon-rss:5048624kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:16608kB oom_score_adj:0
      Mar 22 15:57:44 editedlab807vm5 kernel: oom_reaper: reaped process 2814619 (ns-slapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
      Mar 23 04:05:24 editedlab807vm5 kernel: Out of memory: Killed process 1869667 (ns-slapd) total-vm:11700644kB, anon-rss:4930256kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:16788kB oom_score_adj:0
      Mar 23 04:05:24 editedlab807vm5 kernel: oom_reaper: reaped process 1869667 (ns-slapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:4kB
      Mar 26 22:46:35 editedlab807vm5 kernel: Out of memory: Killed process 3342693 (ns-slapd) total-vm:11566236kB, anon-rss:4941532kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:17036kB oom_score_adj:0
      Mar 26 22:46:36 editedlab807vm5 kernel: oom_reaper: reaped process 3342693 (ns-slapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:8kB
      Mar 27 10:30:21 editedlab807vm5 kernel: Out of memory: Killed process 2340 (ns-slapd) total-vm:10116992kB, anon-rss:4562208kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:17296kB oom_score_adj:0
      Mar 27 10:30:21 editedlab807vm5 kernel: oom_reaper: reaped process 2340 (ns-slapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:8kB
      Mar 27 19:30:10 editedlab807vm5 kernel: Out of memory: Killed process 2376 (ns-slapd) total-vm:10160380kB, anon-rss:4883536kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:17556kB oom_score_adj:0
      Mar 28 03:30:05 editedlab807vm5 kernel: Out of memory: Killed process 1378433 (ns-slapd) total-vm:11627936kB, anon-rss:4783960kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:16908kB oom_score_adj:0
      Mar 29 01:20:34 editedlab807vm5 kernel: Out of memory: Killed process 3850620 (ns-slapd) total-vm:11502548kB, anon-rss:4432688kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:16640kB oom_score_adj:0
      Mar 29 01:20:34 editedlab807vm5 kernel: oom_reaper: reaped process 3850620 (ns-slapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:8kB
      Mar 29 10:10:51 editedlab807vm5 kernel: Out of memory: Killed process 1301798 (ns-slapd) total-vm:9706016kB, anon-rss:4721632kB, file-rss:0kB, shmem-rss:8kB, UID:410 pgtables:16456kB oom_score_adj:0
      Mar 29 10:10:52 editedlab807vm5 kernel: oom_reaper: reaped process 1301798 (ns-slapd), now anon-rss:0kB, file-rss:0kB, shmem-rss:8kB

      Expected results:
      yes

      Additional info:

      anomaly: THP appears to till be enabled despite efforts to disable this feature, has:

      sys/kernel/mm/transparent_hugepage/enabled
      always madvise [never]

      /sys/kernel/mm/transparent_hugepage/defrag
      always defer defer+madvise madvise [never]

      but sosreport shows:

      proc/meminfo
      AnonHugePages: 28672 kB

      proc/vmstat
      nr_anon_transparent_hugepages 14

      anomaly: no trace of redhat-ds module in the collected sosreport, missing file:
      /etc/dnf/modules.d/redhat-ds.module
      [redhat-ds]
      name=redhat-ds
      stream=11
      profiles=default
      state=enabled
      (END)

              jira-bugzilla-migration RH Bugzilla Integration
              rhn-support-msauton Marc Sauton
              IdM DS QE IdM DS QE
              Evgenia Martyniuk Evgenia Martyniuk
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: