Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-2767

[AMQ7, purge message, OutOfMemoryException] with a large queue size, removeAllMessages() takes a long time and eventually results in an OOM exception (if enough messages on the queue)

XMLWordPrintable

    • Release Notes
    • +
    • Hide
      Previously, for a queue with a large number (that is, in the order of millions) of messages, the JMX operation `removeAllMessages` could take up to several minutes to complete. In addition, the operation sometimes failed to complete, returning an out-of-memory (OOM) exception instead. This issue is now resolved.
      Show
      Previously, for a queue with a large number (that is, in the order of millions) of messages, the JMX operation `removeAllMessages` could take up to several minutes to complete. In addition, the operation sometimes failed to complete, returning an out-of-memory (OOM) exception instead. This issue is now resolved.
    • Documented as Resolved Issue
    • Verified in a release
    • Hide

      1) generate a broker instance using the default options.

      2) use defaults for "./artemis producer" to generate 4 million messages on the "TEST" queue.

      3) invoke JMX operation "removeAllMessages()" on the "TEST" queue.

      RESULT:

      • takes excessively long time to complete (on my system more than 4 minutes).
      • eventually returns with an OOM exception.
      Show
      1) generate a broker instance using the default options. 2) use defaults for "./artemis producer" to generate 4 million messages on the "TEST" queue. 3) invoke JMX operation "removeAllMessages()" on the "TEST" queue. RESULT: takes excessively long time to complete (on my system more than 4 minutes). eventually returns with an OOM exception.

      On a queue that contains 4,000, 000 messages the JMX operation "removeAllMessages()" eventually results in an OOM exception on the broker (below).

      The messages were generated using the defaults for sample producer

      ./artemis producer

      For my test I was using the default generated broker configuration for AMQ 7.4.0 . Before "removeAllMessages()" was invoked the destination was PAGING and had more than 50 pages.

      From the users perspective there are 2 closely related usability issues:

      1) Out of Memory exception should not occur
      2) The operation took more than 4 minutes to complete. 

      2019-08-22 17:37:03,315 WARN  [org.apache.activemq.artemis.journal] AMQ142024: Error completing callback: java.lang.OutOfMemoryError: Java heap space
	at java.util.LinkedList.listIterator(LinkedList.java:868) [rt.jar:1.8.0_212]
	at java.util.AbstractList.listIterator(AbstractList.java:299) [rt.jar:1.8.0_212]
	at java.util.AbstractSequentialList.iterator(AbstractSequentialList.java:239) [rt.jar:1.8.0_212]
	at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.checkTasks(OperationContextImpl.java:216) [artemis-server-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl.done(OperationContextImpl.java:197) [artemis-server-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.journal.impl.TransactionCallback.done(TransactionCallback.java:51) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.io.IOCallback.lambda$done$0(IOCallback.java:43) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.io.IOCallback$$Lambda$62/631008546.accept(Unknown Source)
	at java.util.ArrayList.forEach(ArrayList.java:1257) [rt.jar:1.8.0_212]
	at org.apache.activemq.artemis.core.io.IOCallback.done(IOCallback.java:41) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.io.DelegateCallback.done(DelegateCallback.java:41) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory$AIOSequentialCallback.sequentialDone(AIOSequentialFileFactory.java:400) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.io.aio.AIOSequentialFile.done(AIOSequentialFile.java:302) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory$AIOSequentialCallback.done(AIOSequentialFileFactory.java:384) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]
	at org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.done(LibaioContext.java:362) [activemq-artemis-native-1.0.0.redhat-00001.jar:1.0.0.redhat-00001]
	at org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.blockedPoll(Native Method) [activemq-artemis-native-1.0.0.redhat-00001.jar:1.0.0.redhat-00001]
	at org.apache.activemq.artemis.nativo.jlibaio.LibaioContext.poll(LibaioContext.java:354) [activemq-artemis-native-1.0.0.redhat-00001.jar:1.0.0.redhat-00001]
	at org.apache.activemq.artemis.core.io.aio.AIOSequentialFileFactory$PollerThread.run(AIOSequentialFileFactory.java:422) [artemis-journal-2.9.0.redhat-00002.jar:2.9.0.redhat-00002]

              dbruscin Domenico Francesco Bruscino
              rhn-support-pfox Patrick Fox (Inactive)
              Oleg Sushchenko Oleg Sushchenko
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: