Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-6683

ReadWriteLock::unlock CPU contention

XMLWordPrintable

    • False
    • None
    • False
    • ARTEMIS-3848
    • Hide

      Here is a way to reproduce this issue:

      clone my github sandbox project (I just created it).

      with the old version of the broker, create a broker:

      no need to auto-tune, the test will not send messages
      ./artemis create my-broker --no-autotune
      start the broker:

      cd my-broker/bin
      ./artemis run

      clone a project I just created for this:

      git clone https://github.com/clebertsuconic/sandbox
      cd sandbox/session-stress
      ./build.sh
      java -jar target/session-stress-0.2.SNAPSHOT-jar-with-dependencies.jar

      find the broker process with jps:

      jmap -dump:dump.hprof <process-id>

      and inspect the generated hprof using Eclipse Memory Analyzer tool (MAT).

      Do the following OQL:

      select * from org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1

      and watch the retained HEAPs. for the large objects, inspect the threadLocal variable on each one of these threads.

      with the fix in place you should have just a handful of elements.

      Show
      Here is a way to reproduce this issue: clone my github sandbox project (I just created it). with the old version of the broker, create a broker: no need to auto-tune, the test will not send messages ./artemis create my-broker --no-autotune start the broker: cd my-broker/bin ./artemis run clone a project I just created for this: git clone https://github.com/clebertsuconic/sandbox cd sandbox/session-stress ./build.sh java -jar target/session-stress-0.2.SNAPSHOT-jar-with-dependencies.jar find the broker process with jps: jmap -dump:dump.hprof <process-id> and inspect the generated hprof using Eclipse Memory Analyzer tool (MAT). Do the following OQL: select * from org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1 and watch the retained HEAPs. for the large objects, inspect the threadLocal variable on each one of these threads. with the fix in place you should have just a handful of elements.

      There is a contention sometimes observed on a broker with a very high volume of messages.

      This ticket is to investigate why this happens. We are suspicious of two possible root causes:

      1) There is a bug or leak on a readWriteLock

      2) This happens because of the high number of cores and CPU they have

       

              csuconic@redhat.com Clebert Suconic
              rhn-support-anarvaez Alfredo Narvaez
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: