-
Bug
-
Resolution: Done
-
Major
-
AMQ 7.7.0.GA
-
+
-
Previously, if a queue was destroyed (for example when a non-durable consumer closed its connection) and depaging was concurrently happening, this might result in a deadlock condition. This issue is now resolved.
-
Documented as Resolved Issue
-
Verified in a release
I've opened this JIRA since a customer reports an issue described by ARTEMIS-2592 that was supposed to be fixed in AMQ 7.6. But they claim to be able to see the same issue with AMQ 7.7 HF 1.
The broker JVM gets killed
2020-08-10 15:21:06,210 ERROR [org.apache.activemq.artemis.core.server] AMQ224079: The process for the virtual machine will be killed, as component QueueImpl[name=nonDurable.IM_lmprdrds010_baileym_2020-08-10T14:54:32:660.order_history, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=e98abd58-6b0a-11ea-9b95-7cd30ad7cc58], temp=true]@5a41e1a1 is not responsive
And after that broker log file shows deadlock that seems to be almost identical to ARTEMIS-2592.
Deadlock detected! "Thread-1820 (ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$7@1de76cc7)" Id=5501 BLOCKED on org.apache.activemq.artemis.core.server.impl.QueueImpl@5a41e1a1 owned by "Thread-25 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@7bd4937b)" Id=178 at org.apache.activemq.artemis.core.server.impl.QueueImpl.depage(QueueImpl.java:3078) - blocked on org.apache.activemq.artemis.core.server.impl.QueueImpl@5a41e1a1 at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2600(QueueImpl.java:126) at org.apache.activemq.artemis.core.server.impl.QueueImpl$DepageRunner.run(QueueImpl.java:4079) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$29/910599202.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) Number of locked synchronizers = 2 - java.util.concurrent.ThreadPoolExecutor$Worker@5798e685 - java.util.concurrent.locks.ReentrantLock$NonfairSync@2029c643 "Thread-25 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$6@7bd4937b)" Id=178 WAITING on java.util.concurrent.locks.ReentrantLock$NonfairSync@2029c643 owned by "Thread-1820 (ActiveMQ-IO-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$7@1de76cc7)" Id=5501 at sun.misc.Unsafe.park(Native Method) - waiting on java.util.concurrent.locks.ReentrantLock$NonfairSync@2029c643 at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199) at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209) at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285) at org.apache.activemq.artemis.core.server.impl.QueueImpl.iterQueue(QueueImpl.java:2073) at org.apache.activemq.artemis.core.server.impl.QueueImpl.deleteMatchingReferences(QueueImpl.java:2040) at org.apache.activemq.artemis.core.server.Queue.deleteMatchingReferences(Queue.java:296) at org.apache.activemq.artemis.core.server.impl.QueueImpl.deleteAllReferences(QueueImpl.java:2030) at org.apache.activemq.artemis.core.server.impl.QueueImpl.deleteAllReferences(QueueImpl.java:2025) at org.apache.activemq.artemis.core.server.impl.QueueImpl.deleteQueue(QueueImpl.java:2228) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.destroyQueue(ActiveMQServerImpl.java:2213) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.destroyQueue(ActiveMQServerImpl.java:2161) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.destroyQueue(ActiveMQServerImpl.java:2152) at org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl.destroyQueue(ActiveMQServerImpl.java:2132) at org.apache.activemq.artemis.core.server.impl.TransientQueueManagerImpl.doIt(TransientQueueManagerImpl.java:43) at org.apache.activemq.artemis.core.server.impl.TransientQueueManagerImpl$$Lambda$201/1675316695.run(Unknown Source) at org.apache.activemq.artemis.utils.ReferenceCounterUtil.execute(ReferenceCounterUtil.java:81) at org.apache.activemq.artemis.utils.ReferenceCounterUtil.decrement(ReferenceCounterUtil.java:71) at org.apache.activemq.artemis.core.server.impl.QueueImpl.removeConsumer(QueueImpl.java:1425) - locked org.apache.activemq.artemis.core.server.impl.QueueImpl@5a41e1a1 at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.removeItself(ServerConsumerImpl.java:633) at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.close(ServerConsumerImpl.java:566) - locked org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl@115ee963 at org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl.close(ServerConsumerImpl.java:540) at org.apache.activemq.artemis.core.server.impl.ServerSessionImpl.closeConsumer(ServerSessionImpl.java:1707) at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.slowPacketHandler(ServerSessionPacketHandler.java:597) at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler.onMessagePacket(ServerSessionPacketHandler.java:292) at org.apache.activemq.artemis.core.protocol.core.ServerSessionPacketHandler$$Lambda$107/1704846988.onMessage(Unknown Source) at org.apache.activemq.artemis.utils.actors.Actor.doTask(Actor.java:33) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$29/910599202.run(Unknown Source) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:65) at org.apache.activemq.artemis.utils.actors.ProcessorBase$$Lambda$29/910599202.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) Number of locked synchronizers = 1 - java.util.concurrent.ThreadPoolExecutor$Worker@6123280d
- clones
-
ENTMQBR-3812 Potential deadlock when destroying a queue and depaging concurrently
- Closed