Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-33883

HAProxy consuming high cpu usage (release 4.15)

XMLWordPrintable

    • Important
    • No
    • 2
    • Sprint 253, Sprint 254
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, in HAProxy 2.6 deployments on OpenShift, shutting down HAproxy could result in a race condition. The main thread `(tid=0)` would wait for other threads to complete, but some threads would enter an
      infinite loop, consuming 100% CPU. With this release, the variable
      controlling the loop's termination is now properly reset, preventing non-main threads from looping indefinitely. This ensures that the thread's poll loop can terminate correctly. (link:https://issues.redhat.com/browse/OCPBUGS-33883[*OCPBUGS-33883*])
      ______________
      Previous Behaviour:

      In HAProxy 2.6 deployments on OpenShift, shutting down haproxy
      could result in a race condition. The main thread (tid=0) would wait
      for other threads to complete, but some threads would enter an
      infinite loop, consuming 100% CPU. This loop involved continuous
      syscalls such as epoll_wait and clock_gettime.

      Fixed Behaviour:

      The issue has been addressed in the upstream issue
      https://github.com/haproxy/haproxy/issues/2537 and we carry the fix in
      the following RPM version: haproxy-2.6.13-3.rhaos4.15.el8.

      The fix ensures that the thread's poll loop can terminate correctly. The variable
      controlling the loop's termination is now properly reset, preventing non-main threads from looping indefinitely. This change ensures that the "stopping" part of the run_poll_loop() function is reached,
      allowing all threads to exit cleanly during shutdown, except for the
      main thread (tid=0), which handles signals.
      Show
      * Previously, in HAProxy 2.6 deployments on OpenShift, shutting down HAproxy could result in a race condition. The main thread `(tid=0)` would wait for other threads to complete, but some threads would enter an infinite loop, consuming 100% CPU. With this release, the variable controlling the loop's termination is now properly reset, preventing non-main threads from looping indefinitely. This ensures that the thread's poll loop can terminate correctly. (link: https://issues.redhat.com/browse/OCPBUGS-33883 [* OCPBUGS-33883 *]) ______________ Previous Behaviour: In HAProxy 2.6 deployments on OpenShift, shutting down haproxy could result in a race condition. The main thread (tid=0) would wait for other threads to complete, but some threads would enter an infinite loop, consuming 100% CPU. This loop involved continuous syscalls such as epoll_wait and clock_gettime. Fixed Behaviour: The issue has been addressed in the upstream issue https://github.com/haproxy/haproxy/issues/2537 and we carry the fix in the following RPM version: haproxy-2.6.13-3.rhaos4.15.el8. The fix ensures that the thread's poll loop can terminate correctly. The variable controlling the loop's termination is now properly reset, preventing non-main threads from looping indefinitely. This change ensures that the "stopping" part of the run_poll_loop() function is reached, allowing all threads to exit cleanly during shutdown, except for the main thread (tid=0), which handles signals.
    • Bug Fix
    • Done

      Description of problem:

      This was originally observed on a haproxy 2.6 deployment in OpenShift.
      When shutting down, we observe a race condition where the thread with tid=0 waits for other threads to complete, while some remaining threads will loop and take 100% CPU doing syscalls epoll_wait and clock_gettime
      
      The customer has raised an github issue for haproxy behaviour.
      
      https://github.com/haproxy/haproxy/issues/2537

      Version-Release number of selected component (if applicable):

          

      How reproducible:

          See upstream issue (ch)

      Steps to Reproduce:

          1.
          2.
          3.
          

      Actual results:

          

      Expected results:

          

      Additional info:

          

            amcdermo@redhat.com Andrew McDermott
            rhn-support-bmehra Bobby Mehra
            Shudi Li Shudi Li
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: