Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-2468

java.lang.NullPointerException with message replication

XMLWordPrintable

    • Release Notes
    • +
    • Hide
      When the values of the `config-delete-addresses` and `config-delete-queues` parameters of your broker configuration are set to `force`, the broker reloads the `broker.xml` configuration file after a change. When the broker reloads the configuration, any auto-created, clustered queues are deleted. In prior releases, this deletion caused a failure that prevented the broker from starting. This issue is now resolved. For any queues that are deleted, the broker catches an exception and logs a warning, but starts as expected.
      Show
      When the values of the `config-delete-addresses` and `config-delete-queues` parameters of your broker configuration are set to `force`, the broker reloads the `broker.xml` configuration file after a change. When the broker reloads the configuration, any auto-created, clustered queues are deleted. In prior releases, this deletion caused a failure that prevented the broker from starting. This issue is now resolved. For any queues that are deleted, the broker catches an exception and logs a warning, but starts as expected.
    • Documented as Resolved Issue
    • ARTEMIS-2480
    • Verified in a release

      2 pair of master/slave brokers with High Availability replication. Attached are the configuration files.

      The below logs appear,

      2019-04-19 14:41:46,941 WARN  [org.apache.activemq.artemis.core.server] AMQ222151: removing consumer which did not handle a message, consumer=ClusterConnectionBridge@36b7dfb5 [name=$.artemis.internal.sf.cluster-amq-mainqueues-mlb1ldv.2f4ef65d-5d0e-11e9-970d-0050561fd2aa, queue=QueueImpl[name=$.artemis.internal.sf.cluster-amq-mainqueues-mlb1ldv.2f4ef65d-5d0e-11e9-970d-0050561fd2aa, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=30705356-5211-11e9-8e0e-00505629c4a1], temp=false]@63b0ee25 targetConnector=ServerLocatorImpl (identity=(Cluster-connection-bridge::ClusterConnectionBridge@36b7dfb5 [name=$.artemis.internal.sf.cluster-amq-mainqueues-mlb1ldv.2f4ef65d-5d0e-11e9-970d-0050561fd2aa, queue=QueueImpl[name=$.artemis.internal.sf.cluster-amq-mainqueues-mlb1ldv.2f4ef65d-5d0e-11e9-970d-0050561fd2aa, postOffice=PostOfficeImpl [server=ActiveMQServerImpl::serverUUID=30705356-5211-11e9-8e0e-00505629c4a1], temp=false]@63b0ee25 targetConnector=ServerLocatorImpl [initialConnectors=[TransportConfiguration(name=DL01927V-61616, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=dl01927v-ops-vdab-be], discoveryGroupConfiguration=null]]::ClusterConnectionImpl@497769258[nodeUUID=30705356-5211-11e9-8e0e-00505629c4a1, connector=TransportConfiguration(name=DL01928V-61616, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=dl01928v-ops-vdab-be, address=, server=ActiveMQServerImpl::serverUUID=30705356-5211-11e9-8e0e-00505629c4a1])) [initialConnectors=[TransportConfiguration(name=DL01927V-61616, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&host=dl01927v-ops-vdab-be], discoveryGroupConfiguration=null]], message=Reference[6770590887]:RELIABLE:CoreMessage[messageID=6770590887,durable=true,userID=7e869832-62a0-11e9-bddc-005056a61ff5,priority=4, timestamp=Fri Apr 19 14:41:46 CEST 2019,expiration=0, durable=true, address=dienstbeheer2projectbeheer.boekingen.queue,size=1223,properties=TypedProperties[__AMQ_CID=7e7cd42f-62a0-11e9-bddc-005056a61ff5,_AMQ_ROUTING_TYPE=1,ReplyToName=projectbeheer2dienstbeheer.boekingen.feedback.queue,JMSCorrelationID=64aef3dbfb864af4becb9830c9357659,_AMQ_VALIDATED_USER=cn=VDABCOR1,ou=USERS,ou=INTERN,o=VDAB,major=1,JMSXAppID=dienstbeheer,commandName=RegistreerBoeking,_AMQ_ROUTE_TO$.artemis.internal.sf.cluster-amq-mainqueues-mlb1ldv.2f4ef65d-5d0e-11e9-970d-0050561fd2aa=[0000 0000 0000 02C3),bytesAsLongs(707]]]@693237909: java.lang.NullPointerException
        at org.apache.activemq.artemis.core.server.cluster.impl.BridgeImpl.handle(BridgeImpl.java:581) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.handle(QueueImpl.java:2989) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2340) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2000(QueueImpl.java:107) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:3216) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191]
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]

2019-04-19 14:41:46,947 WARN  [org.apache.activemq.artemis.core.server.impl.QueueImpl] null: java.util.NoSuchElementException
        at org.apache.activemq.artemis.utils.collections.PriorityLinkedListImpl$PriorityLinkedListIterator.repeat(PriorityLinkedListImpl.java:172) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.deliver(QueueImpl.java:2359) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl.access$2000(QueueImpl.java:107) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.core.server.impl.QueueImpl$DeliverRunner.run(QueueImpl.java:3216) [artemis-server-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_191]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_191]
        at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.6.3.redhat-00020.jar:2.6.3.redhat-00020]

      There are huge number of messages in the queues below the activemq.notifications addresses, on one machine there are 640315 messages, on the other one, there are 647102 messages. The messages are of type "_AMQ_NotifType: CONSUMER_CLOSED/CONSUMER_ADDED) and the content of the messages are 4 null bytes.

      Concern: messages are not redistributed between the masters and the the number of messages in activemq.notifications are raising.

      The messages in the artemis.internal queue are not consumed anymore by just doing a touch (causing a reload of the configuration) on the broker.xml file on the masters (with 2 or 3 masters).

      What is causing the NPE to occur ?

        1. test-cluster-setup.zip
          22 kB

              rh-ee-ataylor Andy Taylor
              rhn-support-vgohel Viral Gohel (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: