Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-2539

Warn exception after migration of high number of messages

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • AMQ 7.2.4.GA, AMQ 7.4.1.GA
    • container image
    • None
    • Hide

      1. Deploy clustered AMQ 7 of N replicas
      2. Produce a higher number of messages
      e.g. in the last broker pod `artemis producer --url tcp://broker-1:61616 --destination queue/internal --message-count 10000`
      3. Scale down N-1

      Show
      1. Deploy clustered AMQ 7 of N replicas 2. Produce a higher number of messages e.g. in the last broker pod `artemis producer --url tcp://broker-1:61616 --destination queue/internal --message-count 10000` 3. Scale down N-1
    • AMQ Broker 2619, AMQ Broker 2919

      I've tested higher volume of messages to be migrated, specifically 10K, 50K and 100K in one queue and in multiple queues.

      After reported successful migration the warning exception related to journal cleanup started to appear repeatedly. I've confirmed that all the messages were migrated and reachable on the new broker.

      I just want to make sure that the exceptions are as harmless as it appears at first glance.

      Full log included.

      2018-10-11 14:50:06,046 INFO [org.apache.activemq.artemis.core.server] AMQ221029: stopped bridge $.artemis.internal.sf.my-cluster.b42c35ef-cd64-11e8-8c7e-0a580a8200d9
      2018-10-11 14:50:22,138 INFO [org.apache.activemq.artemis.core.server] AMQ221078: Scaled down 10,000 messages total.
      2018-10-11 14:50:22,235 WARN [org.apache.activemq.artemis.core.server] AMQ222153: Unable to remove message id = 6,937 please remove manually: ActiveMQShutdownException[errorType=SHUTDOWN_ERROR message=Journal must be in state=LOADED, was [STOPPED]]
      at org.apache.activemq.artemis.core.journal.impl.JournalImpl.checkJournalIsLoaded(JournalImpl.java:1051) [artemis-journal-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.journal.impl.JournalImpl.appendDeleteRecord(JournalImpl.java:918) [artemis-journal-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.persistence.impl.journal.AbstractJournalStorageManager.deleteMessage(AbstractJournalStorageManager.java:450) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.server.impl.QueueImpl.postAcknowledge(QueueImpl.java:3062) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.server.impl.RefsOperation.afterCommit(RefsOperation.java:157) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.afterCommit(TransactionImpl.java:546) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl.access$100(TransactionImpl.java:38) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.transaction.impl.TransactionImpl$2.done(TransactionImpl.java:304) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.core.persistence.impl.journal.OperationContextImpl$1.run(OperationContextImpl.java:236) [artemis-server-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_181]
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_181]
      at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.6.3.redhat-00004.jar:2.6.3.redhat-00004]

        1. broker-draining-3.log
          6.07 MB

              rhn-support-rkieley Roderick Kieley
              dsimansk@redhat.com David Simansky
              Roman Vais Roman Vais (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: