Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-9936

High CPU usage in OutboundStore.generateMqttId

XMLWordPrintable

      A customer is having symptoms of http://issues.apache.org/jira/browse/ARTEMIS-5467. As this bug is fixed on Artemis 2.42 and AMQ Broker 7.13.x is on 2.40. Could you please include it in the next A-MQ 7.13 minor release?

      Following the relevant log messages and relevant threads from the customer, which match the upstream bug

      2025-08-16 16:44:03,564 INFO  [org.apache.activemq.artemis.core.server] AMQ224107: The Critical Analyzer detected slow paths on the broker.  It is recommended that you enable trace logs on org.apache.activemq.artemis.utils.critical while you troubleshoot this issue. You should disable the trace logs when you have finished troubleshooting.
      2025-08-16 16:44:03,564 ERROR [org.apache.activemq.artemis.core.server] AMQ224079: The process for the virtual machine will be killed, as component QueueImpl[name=EWC63_f26925c4-5af2-4745-a529-9e86969b4477.tlms.PROD.central, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::name=ProdPrimary], temp=false]@5afa1b1 is not responsive
      2025-08-16 16:44:05,517 WARN  [org.apache.activemq.artemis.core.server] AMQ222199: Thread dump: *******************************************************************************
      ...
      "Thread-21 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@7a55f148)" Id=93 BLOCKED on java.lang.Object@2b912809 owned by "Thread-64 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@7a55f148)" Id=136
      	at org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionState$OutboundStore.generateMqttId(MQTTSessionState.java:459)
      	-  blocked on java.lang.Object@2b912809
      	at org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:167)
      	at org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionCallback.sendMessage(MQTTSessionCallback.java:50)
      	at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.deliverStandardMessage(ServerConsumerImpl.java:1223)
      	at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:528)
      	at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:4071)
      	at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3324)
      	at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4439)
      	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
      	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
      	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
      	at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$469/0x00007f200c349da0.run(Unknown Source)
      	at java.base@17.0.15/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      	at java.base@17.0.15/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      	at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
      	Number of locked synchronizers = 2
      	- java.util.concurrent.locks.ReentrantLock$NonfairSync@2ca8ee44
      	- java.util.concurrent.ThreadPoolExecutor$Worker@4db60246
      
      ...
      "Thread-64 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@7a55f148)" Id=136 RUNNABLE
      	at java.base@17.0.15/java.lang.Integer.valueOf(Integer.java:1081)
      	at org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionState$OutboundStore.generateMqttId(MQTTSessionState.java:468)
      	-  locked java.lang.Object@2b912809
      	at org.apache.activemq.artemis.core.protocol.mqtt.MQTTPublishManager.sendMessage(MQTTPublishManager.java:167)
      	at org.apache.activemq.artemis.core.protocol.mqtt.MQTTSessionCallback.sendMessage(MQTTSessionCallback.java:50)
      	at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.deliverStandardMessage(ServerConsumerImpl.java:1223)
      	at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.proceedDeliver(ServerConsumerImpl.java:528)
      	at org.apache.activemq.artemis.core.server.impl.QueueImpl.proceedDeliver(QueueImpl.java:4071)
      	at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:3324)
      	at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:4439)
      	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:57)
      	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:32)
      	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:68)
      	at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$469/0x00007f200c349da0.run(Unknown Source)
      	at java.base@17.0.15/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      	at java.base@17.0.15/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      	at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)
      	Number of locked synchronizers = 2
      	- java.util.concurrent.locks.ReentrantLock$NonfairSync@31b071f4
      	- java.util.concurrent.ThreadPoolExecutor$Worker@361abd01

              rhn-support-jbertram Justin Bertram
              rhn-support-anarvaez Alfredo Narvaez
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: