Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-3922

Temporary Queue Leak With OpenWire Request-Reply Clients

XMLWordPrintable

    • False
    • False
    • Undefined
    • Verified in a release
    • Hide

      Reproducer attached:

      1. Configure an AMQ 7.7.0 broker with default settings
      2. Build and run the response-consumer (this is the consumer for the producer that emits response messages) - configure application.properties for the correct broker-url. Run using mvn spring-boot:run.
      3. Build and run the replyto-producer (emits the original messages and creates the reply consumer) - configure application.properties for the correct broker-url. Run using mvn spring-boot:run.
      4. Allow the applications to run for 10 or so minutes
      5. Restart the replyto-producer
      6. If no issue is observed, let the apps run for another 10 or so minutes - repeat until the WARN messages are observed in the response-consumer log
      7. Take a heap dump of the broker for examination:

      SELECT toString(tq.physicalName) FROM org.apache.activemq.command.ActiveMQTempQueue tq

      Show
      Reproducer attached: 1. Configure an AMQ 7.7.0 broker with default settings 2. Build and run the response-consumer (this is the consumer for the producer that emits response messages) - configure application.properties for the correct broker-url. Run using mvn spring-boot:run . 3. Build and run the replyto-producer (emits the original messages and creates the reply consumer) - configure application.properties for the correct broker-url. Run using mvn spring-boot:run . 4. Allow the applications to run for 10 or so minutes 5. Restart the replyto-producer 6. If no issue is observed, let the apps run for another 10 or so minutes - repeat until the WARN messages are observed in the response-consumer log 7. Take a heap dump of the broker for examination: SELECT toString(tq.physicalName) FROM org.apache.activemq.command.ActiveMQTempQueue tq

      When using the camel-activiemq component with an InOut exchange pattern and the default settings (use a temporary queue for replyTo address), a build-up of temporary reply queues with the same name occurs.

      When the InOut producers is restarted, I can see orphan temporary queues corresponding to the original producer client ID remain stranded in the broker heap left behind.

      Sometimes when this occurs the consumer trying to respond to the producer begins emitting WARN messages like the following that do not stop until the broker or the replying consumer are restarted:

      18:32:34.409 [Camel (camel-1) thread #2 - JmsConsumer[replyto-producer-v1]] WARN  o.a.c.c.jms.EndpointMessageListener - Execution of JMS message listener failed. Caused by: [org.apache.camel.RuntimeCamelException - org.springframework.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:test.remote.csb-43698-1600554750891-1:1:1; nested exception is javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:test.remote.csb-43698-1600554750891-1:1:1]
      org.apache.camel.RuntimeCamelException: org.springframework.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:test.remote.csb-43698-1600554750891-1:1:1; nested exception is javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:test.remote.csb-43698-1600554750891-1:1:1
      	at org.apache.camel.util.ObjectHelper.wrapRuntimeCamelException(ObjectHelper.java:1826)
      	at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:134)
      	at org.springframework.jms.listener.AbstractMessageListenerContainer.doInvokeListener(AbstractMessageListenerContainer.java:736)
      	at org.springframework.jms.listener.AbstractMessageListenerContainer.invokeListener(AbstractMessageListenerContainer.java:696)
      	at org.springframework.jms.listener.AbstractMessageListenerContainer.doExecuteListener(AbstractMessageListenerContainer.java:674)
      	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.doReceiveAndExecute(AbstractPollingMessageListenerContainer.java:318)
      	at org.springframework.jms.listener.AbstractPollingMessageListenerContainer.receiveAndExecute(AbstractPollingMessageListenerContainer.java:257)
      	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.invokeListener(DefaultMessageListenerContainer.java:1189)
      	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.executeOngoingLoop(DefaultMessageListenerContainer.java:1179)
      	at org.springframework.jms.listener.DefaultMessageListenerContainer$AsyncMessageListenerInvoker.run(DefaultMessageListenerContainer.java:1076)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: org.springframework.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:test.remote.csb-43698-1600554750891-1:1:1; nested exception is javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:testremote.csb-43698-1600554750891-1:1:1
      	at org.springframework.jms.support.JmsUtils.convertJmsAccessException(JmsUtils.java:280)
      	at org.springframework.jms.support.JmsAccessor.convertJmsAccessException(JmsAccessor.java:185)
      	at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:507)
      	at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.send(JmsConfiguration.java:556)
      	at org.apache.camel.component.jms.EndpointMessageListener.sendReply(EndpointMessageListener.java:370)
      	at org.apache.camel.component.jms.EndpointMessageListener$EndpointMessageListenerAsyncCallback.done(EndpointMessageListener.java:223)
      	at org.apache.camel.component.jms.EndpointMessageListener.onMessage(EndpointMessageListener.java:117)
      	... 11 common frames omitted
      Caused by: javax.jms.InvalidDestinationException: Cannot publish to a deleted Destination: temp-queue://ID:test.remote.csb-43698-1600554750891-1:1:1
      	at org.apache.activemq.ActiveMQSession.send(ActiveMQSession.java:1911)
      	at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:288)
      	at org.apache.activemq.ActiveMQMessageProducer.send(ActiveMQMessageProducer.java:223)
      	at org.apache.activemq.ActiveMQMessageProducerSupport.send(ActiveMQMessageProducerSupport.java:241)
      	at org.springframework.jms.core.JmsTemplate.doSend(JmsTemplate.java:634)
      	at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.doSend(JmsConfiguration.java:634)
      	at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.doSendToDestination(JmsConfiguration.java:573)
      	at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate.access$100(JmsConfiguration.java:515)
      	at org.apache.camel.component.jms.JmsConfiguration$CamelJmsTemplate$4.doInJms(JmsConfiguration.java:558)
      	at org.springframework.jms.core.JmsTemplate.execute(JmsTemplate.java:504)
      	... 15 common frames omitted
      

              rhn-support-jbertram Justin Bertram
              rhn-support-dhawkins Duane Hawkins
              Dominik Lenosi Dominik Lenosi (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: