Details
-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
AMQ 7.10.2.GA
-
None
-
False
-
None
-
False
-
Important
Description
This problem was originally raised for EAP clients, but it reproduces with Spring Boot clients based on Fuse 7.11. All components used in the test case are in the Fuse BOM, which includes an entry for the ActiveMQ 5.11 client runtime (a Red Hat built). So the problem reproduces with the latest Red Hat tested components that still support ActiveMQ.
The application consumes messages from AMQ 7, and updates a database. The customer's real application uses Oracle, but the test case uses H2. For every 100 messages that are delivered to the queue from which the application consumer, 85-100 end up fully committed. The rest end up in the DLQ.
There is nothing in the application that shows any sign of rejecting messages. In fact, there is no code in place to do so. The application logs nothing and raises no exceptions. It seems to be completely unaware of any problem.
In the broker log, however, we see many messages that indicate that the client closed a connection unexpectedly:
2023-01-27 09:54:00,661 WARN [org.apache.activemq.artemis.core.server] AMQ222061: Client connection failed, clearing up resources for session ID:fedora-36671-1674813233871-1:45:1 2023-01-27 09:54:00,664 WARN [org.apache.activemq.artemis.core.server] AMQ222107: Cleared up resources for session ID:fedora-36671-1674813233871-1:45:1
After some number of these, we see the message sent to the DLQ:
2023-01-27 09:54:00,682 WARN [org.apache.activemq.artemis.core.server] AMQ222149: Message Reference[167]:RELIABLE:CoreMessage[messageID=167,durable=true,userID=843d6609-9e28-11ed-a6af-2016b900fb90,priority=4, timestamp=Fri Jan 27 09:53:57 GMT 2023,expiration=0, durable=true, address=quickstart-messages,size=1584,properties=TypedProperties[__HDR_dlqDeliveryFailureCause=java.lang.Throwable: Dispatch[8] to ID:fedora-36671-1674813233871-3:44:1:1 exceeds redelivery policy limit:RedeliveryPolicy {destination = null, collisionAvoidanceFactor = 0.15, maximumRedeliveries = 6, maximumRedeliveryDelay = -1, initialRedeliveryDelay = 1000, useCollisionAvoidance = false, useExponentialBackOff = false, backOffMultiplier = 5.0, redeliveryDelay = 1000, preDispatchCheck = true},__AMQ_CID=ID:fedora-36671-1674813233871-2:2,_AMQ_GROUP_SEQUENCE=0,__HDR_BROKER_IN_TIME=1674813237285,_AMQ_ROUTING_TYPE=1,__HDR_ARRIVAL=0,__HDR_COMMAND_ID=240,id=79,__HDR_PRODUCER_ID=ID:fedora-36671-1674813233871-3:2:1:79,__HDR_MESSAGE_ID=ID:fedora-36671-1674813233871-3:2:1:79:1,__HDR_DROPPABLE=false]]@1645649488 has reached maximum delivery attempts, sending it to Dead Letter Address DLQ from quickstart-messages
The problem does not occur with the Artemis client runtime, which requires no changes in the code – just a different dependency in the POM.
The problem does not occur with the OpenWire client when exactly the same test case is run against AMQ 6.3.
The problem does not occur with the OpenWire client when the pre-fetch value is reduced to zero – but this causes other problems.
