Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-25168

Need public clarification about qsort out of bounds memory access in RHEL glibc

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • rhel-9.5
    • glibc
    • None
    • None
    • 1
    • rhel-sst-pt-libraries
    • ssg_platform_tools
    • 1
    • False
    • Hide

      None

      Show
      None
    • No
    • Red Hat Enterprise Linux
    • SST PT Libraries Sprint 4
    • None
    • None
    • All
    • None

        The article Out-of-bounds read & write in the glibc's qsort() should not apply to any RHEL glibc.

        The clarification should be to describe that the main problem was https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=709fbd3ec3595f2d1076b4fec09a739327459288

      that added:

      -  if (total_elems > MAX_THRESH)
      +  if (total_size < sizeof buf)
      +    buf = tmp;
      +  else

      and fixed in
      https://sourceware.org/git/gitweb.cgi?p=glibc.git;a=commitdiff;h=dfa3394a605c8f6f25e4f827789bc89eca1d206c

       

        The openwall article is confusing, and at one point tells it is required to add the patch:

      --- a/stdlib/qsort.c
      +++ b/stdlib/qsort.c
      @@ -238,7 +238,7 @@ insertion_sort_qsort_partitions (void *const pbase, size_t total_elems,
         while ((run_ptr += size) <= end_ptr)
           {
             tmp_ptr = run_ptr - size;
      -      while (run_ptr != tmp_ptr && cmp (run_ptr, tmp_ptr, arg) < 0)
      +      while (tmp_ptr != base_ptr && cmp (run_ptr, tmp_ptr, arg) < 0)
               tmp_ptr -= size;
      {{}}
             tmp_ptr += size;

      what is not correct either. RHEL8 and RHEL9 glibc do not have the pointer comparison, and if anything, could add a paranoia check of:

      +      while (tmp_ptr >= base_ptr && cmp (run_ptr, tmp_ptr, arg) < 0)

      (and/or also add check to avoid the case of run_ptr == tmp_ptr for the sake of also avoiding a self comparison) but such test of tmp_ptr not being out bounds probably would only hold true in very special cases if the comparison function did return random values. As long as it is stable for the same input, it should not happen.

        The article is also somewhat confusing when it describes, for example:

      $ while true; do n=$((RANDOM*64+RANDOM+1)); prlimit --as=$((n*4/2*3)) ./qsort $n; done
      Aborted (core dumped)
      Aborted (core dumped)
      Aborted (core dumped)
      ...

      as with small RANDOM values it will fail to exec or map shared libraries.

      As expected, using the input in the backtrace in the article does not cause any problem in RHEL glibc.

      $ prlimit -as=8104854 ./qsort 1350809

      We might need to ask openwall to rewrite that link to clarify it was a temporary issye in glibc-2.36.

      Otherwise, we can just extend Does RHEL glibc have the refactor to qsort() to solve the memory corruption issue?

              fweimer@redhat.com Florian Weimer
              rhn-support-pandrade Paulo Andrade
              Florian Weimer Florian Weimer
              qe-baseos-tools-bugs@redhat.com qe-baseos-tools-bugs@redhat.com qe-baseos-tools-bugs@redhat.com qe-baseos-tools-bugs@redhat.com
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: