Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-12876

named crashed on assertion when destroying a view shortly after a reload occurred

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • rhel-7.9.z
    • bind
    • None
    • None
    • Moderate
    • rhel-sst-cs-net-perf-services
    • ssg_core_services
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • None
    • None
    • None

      What were you trying to do that didn't work?

      A customer reported a crash of named a few seconds after a reload happened:

      Oct 08 03:39:02 xxx named[38862]: reloading configuration succeeded
      Oct 08 03:39:02 xxx named[38862]: reloading zones succeeded
      Oct 08 03:39:02 xxx systemd[1]: Reloaded Berkeley Internet Name Domain (DNS).
       :
      Oct 08 03:39:02 xxx named[38862]: all zones loaded
      Oct 08 03:39:02 xxx named[38862]: running
       :
      Oct 08 03:39:09 xxx named[38862]: resolver priming query complete
      Oct 08 03:39:09 xxx named[38862]: ../../../lib/isc/mem.c:1080: INSIST(ctx->stats[i].gets == 0U) failed, back trace 

      The coredump shows the following backtrace and slot 362 of stats array not being 0, causing the assertion to fail:

      Core was generated by `/usr/sbin/named -u named -c /etc/named.conf'.
      Program terminated with signal 6, Aborted.
      #0  0x00007f59c48c5387 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
      55      return INLINE_SYSCALL (tgkill, 3, pid, selftid, sig);
      (gdb) bt
      #0  0x00007f59c48c5387 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:55
      #1  0x00007f59c48c6a78 in __GI_abort () at abort.c:90
      #2  0x00005609b901a36e in assertion_failed (file=<optimized out>, line=<optimized out>, type=<optimized out>, 
          cond=<optimized out>) at ../../../bin/named/main.c:232
      #3  0x00007f59c61b02ea in isc_assertion_failed (file=file@entry=0x7f59c61f8b74 "../../../lib/isc/mem.c", 
          line=line@entry=1080, type=type@entry=isc_assertiontype_insist, 
          cond=cond@entry=0x7f59c61f8d09 "ctx->stats[i].gets == 0U") at ../../../lib/isc/assertions.c:51
      #4  0x00007f59c61c27b8 in destroy (ctx=ctx@entry=0x7f593480f5f0) at ../../../lib/isc/mem.c:1080
      #5  0x00007f59c61c2adc in isc__mem_detach (ctxp=ctxp@entry=0x7f59340c98f8) at ../../../lib/isc/mem.c:1135
      #6  0x00007f59c61c58d8 in isc_mem_detach (mctxp=mctxp@entry=0x7f59340c98f8) at ../../../lib/isc/mem.c:2746
      #7  0x00007f59c75026d6 in destroy (res=0x7f59a8c794d8) at ../../../lib/dns/resolver.c:8912
      #8  dns_resolver_detach (resp=resp@entry=0x7f59340b9a68) at ../../../lib/dns/resolver.c:9504
      #9  0x00007f59c753d244 in destroy (view=0x7f59340b9a40) at ../../../lib/dns/view.c:396
      #10 0x00007f59c753e15c in resolver_shutdown (task=<optimized out>, event=0x0) at ../../../lib/dns/view.c:692
      #11 0x00007f59c61d4deb in dispatch (manager=0x7f59c7be2010) at ../../../lib/isc/task.c:1141
      #12 run (uap=0x7f59c7be2010) at ../../../lib/isc/task.c:1313
      #13 0x00007f59c591aea5 in start_thread (arg=0x7f59bb789700) at pthread_create.c:307
      #14 0x00007f59c498db0d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
      
      (gdb) f 4
      #4  0x00007f59c61c27b8 in destroy (ctx=ctx@entry=0x7f593480f5f0) at ../../../lib/isc/mem.c:1080
      1080    ../../../lib/isc/mem.c: No such file or directory.
      
      (gdb) p ctx->stats
      $2 = (struct stats *) 0x7f593480f760
      (gdb) p ctx->stats[0]
      $3 = {gets = 0, totalgets = 0, blocks = 0, freefrags = 0}
      (gdb) p ctx->stats[1]
      $4 = {gets = 0, totalgets = 0, blocks = 0, freefrags = 0}
      (gdb) p ctx->stats[2]
      $5 = {gets = 0, totalgets = 0, blocks = 0, freefrags = 0}
      (gdb) p i
      $6 = 362
      (gdb) p/x ctx->stats[362]
      $7 = {gets = 0x400000000, totalgets = 0x0, blocks = 0x0, freefrags = 0x0}
      (gdb) p ctx->stats[361]
      $8 = {gets = 0, totalgets = 0, blocks = 0, freefrags = 0}
      (gdb) p ctx->stats[363]
      $9 = {gets = 0, totalgets = 0, blocks = 0, freefrags = 0}

      Here above we crashed in destroy() due to having ctx->stats[362].gets == 0x400000000:

      1029 static void
      1030 destroy(isc__mem_t *ctx) {
       :
      1069         if (ctx->checkfree) {
      1070                 for (i = 0; i <= ctx->max_size; i++) {
      1071                         if (ctx->stats[i].gets != 0U) {
      1072                                 fprintf(stderr,
      1073                                         "Failing assertion due to probable "
      1074                                         "leaked memory in context %p (\"%s\") "
      1075                                         "(stats[%u].gets == %lu).\n",
      1076                                         ctx, ctx->name, i, ctx->stats[i].gets);
      1077 #if ISC_MEM_TRACKLINES
      1078                                 print_active(ctx, stderr);
      1079 #endif
      1080  HERE>>>>>                      INSIST(ctx->stats[i].gets == 0U);
      1081                         }
      1082                 }
      1083         }
       : 

      Please provide the package NVR for which bug is seen:

      bind-9.11.4-26.P2.el7_9.13.x86_64

      How reproducible:

      Happened only once so far

       

              pemensik@redhat.com Petr Mensik
              rhn-support-rmetrich Renaud Métrich
              Petr Mensik Petr Mensik
              rhel-cs-infra-services-qe rhel-cs-infra-services-qe rhel-cs-infra-services-qe rhel-cs-infra-services-qe
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: