Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-8496

Hanging slow subscriber with JDBC journal store

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • None
    • None
    • None
    • None
    • False
    • None
    • False
    • Hide

      Steps to reproduce:

      git clone -b master git@gitlab.mw.lab.eng.bos.redhat.com:mnovak/messaging-testsuite.git messaging-testsuite
cd messaging-testsuite/scripts/

groovy -DEAP_ZIP_URL=<path-to>/jboss-eap.zip PrepareServers7.groovy
export WORKSPACE=$PWD
export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap
export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap
export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap
export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap

cd ../jboss-hornetq-testsuite/
mvn clean install -B -Dartemis.version=2.21.0 -Deap7.org.jboss.qa.hornetq.apps.clients.version=8.1696614115-SNAPSHOT -DreuseForks=false -Dmaven.test.failure.ignore=true -Deap7.clients.version=8.1696614115-SNAPSHOT -Dsurefire.failIfNoSpecifiedTests=false -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true -Dtest=LargeMessageFileDescriptorsTestCase#openFileDescriptorsOnTopicTest | tee log
      Show
      Steps to reproduce: git clone -b master git@gitlab.mw.lab.eng.bos.redhat.com:mnovak/messaging-testsuite.git messaging-testsuite cd messaging-testsuite/scripts/ groovy -DEAP_ZIP_URL=<path-to>/jboss-eap.zip PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean install -B -Dartemis.version=2.21.0 -Deap7.org.jboss.qa.hornetq.apps.clients.version=8.1696614115-SNAPSHOT -DreuseForks= false -Dmaven.test.failure.ignore= true -Deap7.clients.version=8.1696614115-SNAPSHOT -Dsurefire.failIfNoSpecifiedTests= false -Dmaven.wagon.http.ssl.insecure= true -Dmaven.wagon.http.ssl.allowall= true -Dtest=LargeMessageFileDescriptorsTestCase#openFileDescriptorsOnTopicTest | tee log

      Test Scenario:

      • Start single server with deployed topic (JDBC store journal on DB2)
      • Send 1000 Large Messages to topic.
      • Slow (there is 500ms delay between receiving messages) and fast subscriber start consume messages from topic (receive timeout is 10 seconds)

      Expected result: Check that all messages were received.
      Actual result: Slow consumer does not finish and test timeouts after 1 hour

      Suspected warning in server log:

      14:24:01 12:24:01,642 WARN  [org.apache.activemq.artemis.utils.actors.OrderedExecutor] (Thread-2 (ActiveMQ-PageExecutor-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$9@2219d905)) null: java.lang.AssertionError
14:24:01 	at org.apache.activemq.artemis@2.21.0.redhat-00044//org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl$CoreLargeMessageDeliverer.deliver(ServerConsumerImpl.java:1361)
14:24:01 	at org.apache.activemq.artemis@2.21.0.redhat-00044//org.apache.activemq.artemis.core.server.impl.ServerConsumerImpl$2.run(ServerConsumerImpl.java:1224)
14:24:01 	at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:56)
14:24:01 	at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31)
14:24:01 	at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:67)
14:24:01 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
14:24:01 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
14:24:01 	at org.apache.activemq.artemis.journal//org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118)

      Attaching client-thread-dump.txt from maven surefire process and server.log

      This issue was not hit with File based journal.

        1. client-thread-dump.txt
          47 kB
          Emmanuel Hugonnet
        2. server.log
          82 kB
          Emmanuel Hugonnet

              Unassigned Unassigned
              ehugonne1@redhat.com Emmanuel Hugonnet
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: