Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-8062

Core bridge does not reconnect after target destination is full

    XMLWordPrintable

Details

    • False
    • None
    • False
    • Hide

      There is a zip file called "reproducer.zip"

      To run the reproducer:
      A) Unzip "reproducer.zip"
      B) Check "reproducer.sh" env variables and change
      C) Run "reproducer.sh"

      Following the steps that the script does:

      1) Create dest and orig brokers, on ports 61616 and 61617
          The brokers are created with an already provided "broker.xml"
      2) Start a 200 messages consumer on target/dest broker

      $ broker1711/bin/artemis consumer --protocol amqp --destination fqqn://dest-topíc-pubsub::queue1.dest-topíc-pubsub --message-count 200 --url tcp://localhost:61617

      3) Start a 400 messages producer on source/orig broker

      $ broker0711/bin/artemis producer --protocol amqp --destination topic://origin-topic-pubsub --text-size 1024 --message-count 400 

      4) The consumer gets all 200 messages OK
      5) The producer starts to pile up messages on the bridge
      6) After nearly 129 (sometimes 127, or 130)  messages, appears "Address is full" on both logs

      On source/orig broker
      ....2023-05-19 09:40:35,330 WARN [org.apache.activemq.artemis.protocol.amqp.broker.AMQPSessionCallback] AMQ229102: Address "origin-topic-pubsub" is full.
      ...

      This message also appears sporadically

      2023-05-19 09:40:35,178 WARN [org.apache.activemq.artemis.core.server] AMQ222703: Address "dest-topíc-pubsub" is full. Bridge my-bridge will disconnect
       
       

      7) When I check the multicast queue, appears 71 (sometimes 73. sometimes 70) messages on source/orig and target/dest brokers

      $ printf "\n----\norig\n---\n"; broker0711/bin/artemis queue stat --url tcp://localhost:61616 | grep pubsub ; printf "\n---\ndest\n----\n" ; broker1711/bin/artemis queue stat --url tcp://localhost:61617  | grep pubsub


      orig


      origin-queue-pubsub       origin-topic-pubsub       0             73           346           72               273           0               MULTICAST  

      dest


      queue1.dest-topíc-pubsub dest-topíc-pubsub         0             73           273           0               200           0               MULTICAST  

      8) I start a consumer again; consumer gets all 73 messages on dest/target broker
      9) However, the same 73 messages appear in source/orig broker, because the bridge did not reconnect

      $ printf "\n----\norig\n---\n"; broker0711/bin/artemis queue stat --url tcp://localhost:61616 | grep pubsub ; printf "\n---\ndest\n----\n" ; broker1711/bin/artemis queue stat --url tcp://localhost:61617  | grep pubsub


      orig


      origin-queue-pubsub       origin-topic-pubsub       0             73           346           72               273           0               MULTICAST  

      dest


      queue1.dest-topíc-pubsub dest-topíc-pubsub         0             2             273           0               271           0               MULTICAST  
      Show
      There is a zip file called "reproducer.zip" To run the reproducer: A) Unzip "reproducer.zip" B) Check "reproducer.sh" env variables and change C) Run "reproducer.sh" Following the steps that the script does: 1) Create dest and orig brokers, on ports 61616 and 61617     The brokers are created with an already provided "broker.xml" 2) Start a 200 messages consumer on target/dest broker $ broker1711/bin/artemis consumer --protocol amqp --destination fqqn://dest-topíc-pubsub::queue1.dest-topíc-pubsub --message-count 200 --url tcp://localhost:61617 3) Start a 400 messages producer on source/orig broker $ broker0711/bin/artemis producer --protocol amqp --destination topic://origin-topic-pubsub --text-size 1024 --message-count 400  4) The consumer gets all 200 messages OK 5) The producer starts to pile up messages on the bridge 6) After nearly 129 (sometimes 127, or 130)  messages, appears "Address is full" on both logs On source/orig broker ....2023-05-19 09:40:35,330 WARN [org.apache.activemq.artemis.protocol.amqp.broker.AMQPSessionCallback] AMQ229102: Address "origin-topic-pubsub" is full. ... This message also appears sporadically 2023-05-19 09:40:35,178 WARN [org.apache.activemq.artemis.core.server] AMQ222703: Address "dest-topíc-pubsub" is full. Bridge my-bridge will disconnect     7) When I check the multicast queue, appears 71 (sometimes 73. sometimes 70) messages on source/orig and target/dest brokers $ printf "\n---- \norig\n --- \n"; broker0711/bin/artemis queue stat --url tcp://localhost:61616 | grep pubsub ; printf "\n --- \ndest\n ----\n" ; broker1711/bin/artemis queue stat --url tcp://localhost:61617  | grep pubsub orig origin-queue-pubsub       origin-topic-pubsub       0             73           346           72               273           0               MULTICAST   dest queue1.dest-topíc-pubsub dest-topíc-pubsub         0             73           273           0               200           0               MULTICAST   8) I start a consumer again; consumer gets all 73 messages on dest/target broker 9) However, the same 73 messages appear in source/orig broker, because the bridge did not reconnect $ printf "\n---- \norig\n --- \n"; broker0711/bin/artemis queue stat --url tcp://localhost:61616 | grep pubsub ; printf "\n --- \ndest\n ----\n" ; broker1711/bin/artemis queue stat --url tcp://localhost:61617  | grep pubsub orig origin-queue-pubsub       origin-topic-pubsub       0             73           346           72               273           0               MULTICAST   dest queue1.dest-topíc-pubsub dest-topíc-pubsub         0             2             273           0               271           0               MULTICAST  

    Description

      When using AMQP protocol and core bridges, when a producer sends more messages than allowed by the target destination, and the target destination reports that the address is full, the bridge disconnects, and a consumer does not get the messages from the source queue again.

      Before the message AMQ229102: Address "origin-topic-pubsub" is full.  [condition = amqp:resource-limit-exceeded], I see the messages that the consumer gets from the target destination, are also deleted from the source destination. But it does not happen after I get the message, even if I start a new consumer

      The error is reported originally on A-MQ 7.8.0, but I could reproduce also in A-MQ 7.11

       

      Attachments

        Activity

          People

            csuconic@redhat.com Clebert Suconic
            rhn-support-anarvaez Alfredo Narvaez
            Samuel Gajdos Samuel Gajdos
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: