Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-9685

AMQ119027: Could not find reference on consumer ID=0

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • 7.1.0.ER3
    • 7.1.0.DR14, 7.1.0.DR15, 7.1.0.DR16, 7.1.0.DR18, 7.1.0.DR19, 7.1.0.ER1
    • ActiveMQ
    • Regression
    • Hide 
      git clone git://git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git
cd eap-tests-hornetq/scripts/
git checkout 20283f4076ee2245570f9781c4dbc47e322792c5
groovy -DEAP_VERSION=7.1.0.ER1 PrepareServers7.groovy
export WORKSPACE=$PWD
export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap
export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap
export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap
export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap

cd ../jboss-hornetq-testsuite/

mvn clean test -Dtest=DedicatedFailoverTestCase#testMultipleFailoverReceiver -DfailIfNoTests=false -Deap=7x | tee log
      Show
      git clone git: //git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ git checkout 20283f4076ee2245570f9781c4dbc47e322792c5 groovy -DEAP_VERSION=7.1.0.ER1 PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=DedicatedFailoverTestCase#testMultipleFailoverReceiver -DfailIfNoTests= false -Deap=7x | tee log
    • AMQ Sprint 3

      We have test which does 10 times failover/failback in shared store HA scenario to be 100% sure that HA is robust. The test at first send 50 000 messages. During sending there is no crash of server. After that there is sequence of 10 kills and restarts of server 1 which is doing during receiving of messages.

      Since server 1 has backup, receiver should receive all messages even server 1 crashes. However sometimes happens that receiver starts getting error AMQ119027: Could not find reference on consumer ID=0 on all attempts to commit messages. After several attempts to again receive bulk of messages and commit them, receive method returns NULL and receiver finishes before time.

      Customer impact: In some certain cases receiver is not able to do failover/failback and it returns NULL even there are messages to consume.

      This is regression against EAP 7.0.z.

      03:19:10,910 Thread-704 ERROR [org.jboss.qa.hornetq.apps.clients.Receiver11:148] RETRY receive for host: 127.0.0.1, Trying to receive message with count: 19728
javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0, messageId = 77,698 queue = jms.queue.testQueue0
        at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:407)
        at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:318)
        at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:487)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:746)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1105)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.acknowledge(ClientConsumerImpl.java:764)
        at org.apache.activemq.artemis.core.client.impl.ClientMessageImpl.acknowledge(ClientMessageImpl.java:96)
        at org.apache.activemq.artemis.core.client.impl.ClientMessageImpl.acknowledge(ClientMessageImpl.java:38)
        at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.getMessage(ActiveMQMessageConsumer.java:241)
        at org.apache.activemq.artemis.jms.client.ActiveMQMessageConsumer.receive(ActiveMQMessageConsumer.java:132)
        at org.jboss.qa.hornetq.apps.clients.Receiver11.receiveMessage(Receiver11.java:140)
        at org.jboss.qa.hornetq.apps.clients.ReceiverTransAck.run(ReceiverTransAck.java:83)
Caused by: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE message=AMQ119027: Could not find reference on consumer ID=0, messageId = 77,698 queue = jms.queue.testQueue0]
        ... 12 more
03:19:10,930 Thread-704 ERROR [org.jboss.qa.hornetq.apps.clients.Receiver11:55]  Receiver - COMMIT FAILED - JMSException thrown during commit: AMQ119027: Could not find reference on consumer ID=0, messageId = 77
,670 queue = jms.queue.testQueue0. Receiver for node: 127.0.0.1. Received message - count: 19728, retrying receive
javax.jms.IllegalStateException: AMQ119027: Could not find reference on consumer ID=0, messageId = 77,670 queue = jms.queue.testQueue0
        at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:407)
        at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:318)
        at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.sendACK(ActiveMQSessionContext.java:487)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.acknowledge(ClientSessionImpl.java:746)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.doAck(ClientConsumerImpl.java:1105)
        at org.apache.activemq.artemis.core.client.impl.ClientConsumerImpl.flushAcks(ClientConsumerImpl.java:789)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.flushAcks(ClientSessionImpl.java:1646)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:564)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollback(ClientSessionImpl.java:538)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.rollbackOnFailover(ClientSessionImpl.java:472)
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:493)
        at org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:218)
        at org.jboss.qa.hornetq.apps.clients.Receiver11.commitSession(Receiver11.java:39)
        at org.jboss.qa.hornetq.apps.clients.ReceiverTransAck.run(ReceiverTransAck.java:103)
Caused by: ActiveMQIllegalStateException[errorType=ILLEGAL_STATE message=AMQ119027: Could not find reference on consumer ID=0, messageId = 77,670 queue = jms.queue.testQueue0]
        ... 14 more

        1. threaddumps.zip
          2.66 MB
        2. test.log.zip
          2.97 MB
        3. server-1.log.7z
          15 kB
        4. server-1-trace.log.7z
          10.73 MB
        5. server-1-trace.log.1.7z
          11.71 MB
        6. server-1-trace.log.2.7z
          15.10 MB
        7. server-2-logs.7z
          1.83 MB
        8. test-trace.log.7z
          5.12 MB
        9. journal.zip
          8.83 MB

              mtaylor1@redhat.com Martyn Taylor (Inactive)
              eduda_jira Erich Duda (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: