Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-2068

some messages received but not delivered during HA fail-over, fail-back scenario

    XMLWordPrintable

Details

    • Release Notes
    • Hide
      Currently, if a broker fails over to its slave while an OpenWire client is sending messages, messages being delivered to the broker when failover occurs could be lost. To work around this issue, ensure that the broker persists the messages before acknowledging them.
      Show
      Currently, if a broker fails over to its slave while an OpenWire client is sending messages, messages being delivered to the broker when failover occurs could be lost. To work around this issue, ensure that the broker persists the messages before acknowledging them.
    • Documented as Known Issue
    • Hide

      Use either an AMQP client or the Core JMS Client

      Show
      Use either an AMQP client or the Core JMS Client
    • Hide

      reproducer available

      Show
      reproducer available

    Description

      In a high througput producer/consumer scenario, replicated HA configuration, a small number of messages are lost during fail-over and subsequent fail-back. The messages are received by the broker but never delivered to the consumer. Sample "lost" message trace (queue names have been scrubbed). These logs are all from the slave, the master never sees the message:

      2018-10-19 20:10:11,553 DEBUG [org.apache.activemq.artemis.core.server.plugin.impl] AMQ843008: beforeSend called with message: CoreMessage[messageID=-1,durable=true,userID=null,priority=4, timestamp=Fri Oct 19 20:10:11 UTC 2018,expiration=0, durable=true, address=q.test,size=14882,properties=TypedProperties[__AMQ_CID=ID:localhost.localdomain-45939-1539979753396-0:3,__HDR_BROKER_IN_TIME=1539979811553,_AMQ_ROUTING_TYPE=1,__HDR_ARRIVAL=0,__HDR_GROUP_SEQUENCE=0,__HDR_COMMAND_ID=493,__HDR_PRODUCER_ID=[0000 0044 7B01 0030 4944 3A6C 6F63 616C 686F 7374 2E6C 6F63 616C 646F 6D61  ...  3939 3739 3735 3333 3936 2D31 3A33 0000 0000 0000 0001 0000 0000 0000 0002),_AMQ_DUPL_ID=ID:localhost.localdomain-45939-1539979753396-1:3:2:1:233,__HDR_MESSAGE_ID=[0000 0057 6E00 017B 0100 3049 443A 6C6F 6361 6C68 6F73 742E 6C6F 6361 6C64  ... 00 0000 0000 0100 0000 0000 0000 0200 0000 0000 0000 E900 0000 0000 0000 00),__HDR_DROPPABLE=false]]@1604091705, tx: null, session: ServerSessionImpl(), direct: false, noAutoCreateQueue: false

2018-10-19 20:10:11,553 DEBUG [org.apache.activemq.artemis.core.server.plugin.impl] AMQ843010: beforeMessageRoute called with message: CoreMessage[messageID=2147615336,durable=true,userID=null,priority=4, timestamp=Fri Oct 19 20:10:11 UTC 2018,expiration=0, durable=true, address=q.test,size=14882,properties=TypedProperties[__AMQ_CID=ID:localhost.localdomain-45939-1539979753396-0:3,__HDR_BROKER_IN_TIME=1539979811553,_AMQ_ROUTING_TYPE=1,__HDR_ARRIVAL=0,__HDR_GROUP_SEQUENCE=0,__HDR_COMMAND_ID=493,__HDR_PRODUCER_ID=[0000 0044 7B01 0030 4944 3A6C 6F63 616C 686F 7374 2E6C 6F63 616C 646F 6D61  ...  3939 3739 3735 3333 3936 2D31 3A33 0000 0000 0000 0001 0000 0000 0000 0002),_AMQ_DUPL_ID=ID:localhost.localdomain-45939-1539979753396-1:3:2:1:233,__HDR_MESSAGE_ID=[0000 0057 6E00 017B 0100 3049 443A 6C6F 6361 6C68 6F73 742E 6C6F 6361 6C64  ... 00 0000 0000 0100 0000 0000 0000 0200 0000 0000 0000 E900 0000 0000 0000 00),__HDR_DROPPABLE=false]]@1604091705, context: org.apache.activemq.artemis.core.server.impl.RoutingContextImpl@340ab8f0, direct: false, rejectDuplicates: true

2018-10-19 20:10:11,553 DEBUG [org.apache.activemq.artemis.core.server.plugin.impl] AMQ843011: afterMessageRoute message: CoreMessage[messageID=2147615336,durable=true,userID=null,priority=4, timestamp=Fri Oct 19 20:10:11 UTC 2018,expiration=0, durable=true, address=q.test,size=14882,properties=TypedProperties[__AMQ_CID=ID:localhost.localdomain-45939-1539979753396-0:3,__HDR_BROKER_IN_TIME=1539979811553,_AMQ_ROUTING_TYPE=1,__HDR_ARRIVAL=0,__HDR_GROUP_SEQUENCE=0,__HDR_COMMAND_ID=493,__HDR_PRODUCER_ID=[0000 0044 7B01 0030 4944 3A6C 6F63 616C 686F 7374 2E6C 6F63 616C 646F 6D61  ...  3939 3739 3735 3333 3936 2D31 3A33 0000 0000 0000 0001 0000 0000 0000 0002),_AMQ_DUPL_ID=ID:localhost.localdomain-45939-1539979753396-1:3:2:1:233,__HDR_MESSAGE_ID=[0000 0057 6E00 017B 0100 3049 443A 6C6F 6361 6C68 6F73 742E 6C6F 6361 6C64  ... 00 0000 0000 0100 0000 0000 0000 0200 0000 0000 0000 E900 0000 0000 0000 00),__HDR_DROPPABLE=false]]@1604091705, with context: org.apache.activemq.artemis.core.server.impl.RoutingContextImpl@340ab8f0, direct: false, rejectDuplicates: true

2018-10-19 20:10:11,553 DEBUG [org.apache.activemq.artemis.core.server.plugin.impl] AMQ843009: message ID: 2147615336, message CoreMessage[messageID=2147615336,durable=true,userID=null,priority=4, timestamp=Fri Oct 19 20:10:11 UTC 2018,expiration=0, durable=true, address=q.test,size=14882,properties=TypedProperties[__AMQ_CID=ID:localhost.localdomain-45939-1539979753396-0:3,__HDR_BROKER_IN_TIME=1539979811553,_AMQ_ROUTING_TYPE=1,__HDR_ARRIVAL=0,__HDR_GROUP_SEQUENCE=0,__HDR_COMMAND_ID=493,__HDR_PRODUCER_ID=[0000 0044 7B01 0030 4944 3A6C 6F63 616C 686F 7374 2E6C 6F63 616C 646F 6D61  ...  3939 3739 3735 3333 3936 2D31 3A33 0000 0000 0000 0001 0000 0000 0000 0002),_AMQ_DUPL_ID=ID:localhost.localdomain-45939-1539979753396-1:3:2:1:233,__HDR_MESSAGE_ID=[0000 0057 6E00 017B 0100 3049 443A 6C6F 6361 6C68 6F73 742E 6C6F 6361 6C64  ... 00 0000 0000 0100 0000 0000 0000 0200 0000 0000 0000 E900 0000 0000 0000 00),__HDR_DROPPABLE=false]]@1604091705, session name: ID:localhost.localdomain-45939-1539979753396-1:3:2 with tx: null, session: ServerSessionImpl(), direct: false, noAutoCreateQueue: false

      This is the last log for this message; other messages in the test are successfully delivered.

      Attachments

        Activity

          People

            rh-ee-ataylor Andy Taylor
            rhn-support-shiggs Stephen Higgs
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: