Loading...

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: None
Affects Version/s: None
Component/s: JMS
Labels:
- activemq
- intermittent

Steps to Reproduce:
Hide

Steps to reproduce (not 100% reproducer):

git clone git://git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ groovy -DEAP_ZIP_URL=https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/early-testing-messaging-prepare/273//artifact/jboss-eap.zip PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=DedicatedFailoverCoreBridges#testFailoverKillWithBridgeWithStaticConnectors -Deap7.org.jboss.qa.hornetq.apps.clients.version=7.1521544306-SNAPSHOT -DfailIfNoTests=false -Deap=7x | tee log
Show
Steps to reproduce (not 100% reproducer) : git clone git: //git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ groovy -DEAP_ZIP_URL=https: //eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/early-testing-messaging-prepare/273//artifact/jboss-eap.zip PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=DedicatedFailoverCoreBridges#testFailoverKillWithBridgeWithStaticConnectors -Deap7.org.jboss.qa.hornetq.apps.clients.version=7.1521544306-SNAPSHOT -DfailIfNoTests= false -Deap=7x | tee log

There is missing one message in scenario where JMS bridge (with DUPS_OK) resending messages from one server to another. Durint resending target server is killed and bridge does failover to backup.

After failover in server logs there are lots of warns/errors:

05:39:01,697 WARN  [org.apache.activemq.artemis.jms.bridge] (Thread-144) AMQ342009: JMS Bridge N/A failed to send + acknowledge batch, closing JMS objects: javax.jms.JMSException: Duplicate message detected - me
ssage will not be routed. Message information:LargeServerMessage[messageID=2147485500,durable=true,userID=d8ea8e0a-326b-11e8-b74b-fa163e64a220,priority=4, timestamp=Wed Mar 28 05:39:01 EDT 2018,expiration=0, dur
able=true, address=jms.queue.OutQueue,size=410263,properties=TypedProperties[__AMQ_CID=bd95a355-326b-11e8-b74b-fa163e64a220,_AMQ_ROUTING_TYPE=1,AMQ_BRIDGE_MSG_ID_LIST=ID:c13a41dc-326b-11e8-949b-fa163e64a220,JMSX
DeliveryCount=10,count=989,color=RED,counter=990,_AMQ_DUPL_ID=fde672e0-e858-4c50-a148-a0b7a4a223591522229901951,_AMQ_LARGE_SIZE=409615]]@1304406481
        at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:423) [artemis-core-client-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.sendBlocking(ChannelImpl.java:319) [artemis-core-client-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.core.protocol.core.impl.ActiveMQSessionContext.simpleCommit(ActiveMQSessionContext.java:381) [artemis-core-client-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:862) [artemis-core-client-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.core.client.impl.ClientSessionImpl.commit(ClientSessionImpl.java:835) [artemis-core-client-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.jms.client.ActiveMQSession.commit(ActiveMQSession.java:231) [artemis-jms-client-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.jms.bridge.impl.JMSBridgeImpl.sendBatchNonTransacted(JMSBridgeImpl.java:1338) [artemis-jms-server-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.jms.bridge.impl.JMSBridgeImpl.sendBatch(JMSBridgeImpl.java:1298) [artemis-jms-server-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.jms.bridge.impl.JMSBridgeImpl.access$1700(JMSBridgeImpl.java:74) [artemis-jms-server-2.5.0.jar:2.5.0]
        at org.apache.activemq.artemis.jms.bridge.impl.JMSBridgeImpl$SourceReceiver.run(JMSBridgeImpl.java:1694) [artemis-jms-server-2.5.0.jar:2.5.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_162]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_162]
        at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_162]
Caused by: ActiveMQDuplicateIdException[errorType=DUPLICATE_ID_REJECTED message=Duplicate message detected - message will not be routed. Message information:LargeServerMessage[messageID=2147485500,durable=true,u
serID=d8ea8e0a-326b-11e8-b74b-fa163e64a220,priority=4, timestamp=Wed Mar 28 05:39:01 EDT 2018,expiration=0, durable=true, address=jms.queue.OutQueue,size=410263,properties=TypedProperties[__AMQ_CID=bd95a355-326b
-11e8-b74b-fa163e64a220,_AMQ_ROUTING_TYPE=1,AMQ_BRIDGE_MSG_ID_LIST=ID:c13a41dc-326b-11e8-949b-fa163e64a220,JMSXDeliveryCount=10,count=989,color=RED,counter=990,_AMQ_DUPL_ID=fde672e0-e858-4c50-a148-a0b7a4a2235915
22229901951,_AMQ_LARGE_SIZE=409615]]@1304406481]
        ... 13 more

05:39:01,699 WARN  [org.apache.activemq.artemis.core.server] (Thread-23 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@2076a02)) AMQ222149: Message Reference[3056]:RELIABLE:Co
reMessage[messageID=3056,durable=true,userID=c1347573-326b-11e8-949b-fa163e64a220,priority=4, timestamp=Wed Mar 28 05:38:21 EDT 2018,expiration=0, durable=true, address=jms.queue.InQueue,size=10673,properties=Ty
pedProperties[__AMQ_CID=be528a5c-326b-11e8-949b-fa163e64a220,_AMQ_ROUTING_TYPE=1,count=980,counter=981,_AMQ_DUPL_ID=13adf1ec-581f-42e4-835b-0e6e33a4e1191522229901913,color=GREEN]]@511432790 has reached maximum d
elivery attempts, sending it to Dead Letter Address jms.queue.DLQ from jms.queue.InQueue
05:39:01,700 WARN  [org.apache.activemq.artemis.core.server] (Thread-23 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@2076a02)) AMQ222149: Message Reference[3058]:RELIABLE:CoreMessage[messageID=3058,durable=true,userID=c1349c84-326b-11e8-949b-fa163e64a220,priority=4, timestamp=Wed Mar 28 05:38:21 EDT 2018,expiration=0, durable=true, address=jms.queue.InQueue,size=20914,properties=TypedProperties[__AMQ_CID=be528a5c-326b-11e8-949b-fa163e64a220,_AMQ_ROUTING_TYPE=1,count=981,counter=982,_AMQ_DUPL_ID=6feb66cd-8616-4dff-8775-3f85a4b9bdbb1522229901914,color=RED]]@1150646585 has reached maximum delivery attempts, sending it to Dead Letter Address jms.queue.DLQ from jms.queue.InQueue
....
05:39:01,706 WARN  [org.apache.activemq.artemis.core.server] (Thread-23 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$5@2076a02)) AMQ222149: Message Reference[3082]:RELIABLE:CoreMessage[messageID=3082,durable=true,userID=c13d4f1d-326b-11e8-949b-fa163e64a220,priority=4, timestamp=Wed Mar 28 05:38:21 EDT 2018,expiration=0, durable=true, address=jms.queue.InQueue,size=10673,properties=TypedProperties[__AMQ_CID=be528a5c-326b-11e8-949b-fa163e64a220,_AMQ_ROUTING_TYPE=1,count=990,counter=991,_AMQ_DUPL_ID=f802f4b7-e0a1-4dee-acc7-0b11963bec401522229901971,color=GREEN]]@2708478 has reached maximum delivery attempts, sending it to Dead Letter Address jms.queue.DLQ from jms.queue.InQueue

So the missing message ended up in DLQ as it could not be delivered to target queue. It looks like that _AMQ_DUPL_ID might trigger dup message detection in target queueu however such message did not get there.

blocks

WFLY-10320 Upgrade artemis from 1.5.x to 2.x.x

Closed

is cloned by

ENTMQBR-1183 Missing message JMS bridge with DUPS_OK quality of service

Closed

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates