Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-14175

Core bridge does not reconnect when target node is restarted

    XMLWordPrintable

Details

    • Bug
    • Resolution: Duplicate
    • Blocker
    • None
    • 7.2.0.GA
    • JMS
    • Hide 
      git clone git://git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git
cd eap-tests-hornetq/scripts/
groovy -DEAP_ZIP_URL=https://eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/early-testing-messaging-prepare/136//artifact/jboss-eap.zip PrepareServers7.groovy
export WORKSPACE=$PWD
export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap
export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap
export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap
export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap

cd ../jboss-hornetq-testsuite/

mvn clean test -Dtest=Lodh4TestCase#testFailOfOneServer -Deap7.org.jboss.qa.hornetq.apps.clients.version=7.1517906210-SNAPSHOT -DfailIfNoTests=false -Deap=7x | tee log
      Show
      git clone git: //git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ groovy -DEAP_ZIP_URL=https: //eap-qe-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/early-testing-messaging-prepare/136//artifact/jboss-eap.zip PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=Lodh4TestCase#testFailOfOneServer -Deap7.org.jboss.qa.hornetq.apps.clients.version=7.1517906210-SNAPSHOT -DfailIfNoTests= false -Deap=7x | tee log

    Description

      There is regression in scenario with reconnecting core bridge. It was introduced by adding line [1] to BridgeImpl.

      Test scenario:

      • Start server 1 and server 2 with core bridge deployed on server1
      • Cleanly shutdown server2
      • Start server 2 again, it's expected that core bridge will reconnect

      Result: Core bridge does not reconnect. After server2 is cleanly shutdown there is NPE on server1:

      12:05:37,314 WARN  [org.apache.activemq.artemis.core.server] (Thread-0 (ActiveMQ-client-global-threads)) AMQ222095: Connection failed with failedOver=false
12:05:37,316 ERROR [org.apache.activemq.artemis.core.client] (Thread-0 (ActiveMQ-client-global-threads)) AMQ214002: Failed to execute failure listener: java.lang.NullPointerException
	at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936) [rt.jar:1.8.0_131]
	at org.apache.activemq.artemis.core.client.impl.Topology.removeMember(Topology.java:296) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.notifyNodeDown(ServerLocatorImpl.java:1426) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.server.cluster.impl.BridgeImpl.fail(BridgeImpl.java:782) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.server.cluster.impl.BridgeImpl.connectionFailed(BridgeImpl.java:660) [artemis-server-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.callSessionFailureListeners(ClientSessionFactoryImpl.java:701) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.failoverOrReconnect(ClientSessionFactoryImpl.java:637) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:504) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.handleConnectionFailure(ClientSessionFactoryImpl.java:497) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl.access$100(ClientSessionFactoryImpl.java:72) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.core.client.impl.ClientSessionFactoryImpl$1.run(ClientSessionFactoryImpl.java:360) [artemis-core-client-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:42) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.OrderedExecutor.doTask(OrderedExecutor.java:31) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at org.apache.activemq.artemis.utils.actors.ProcessorBase.executePendingTasks(ProcessorBase.java:66) [artemis-commons-2.5.0-SNAPSHOT.jar:2.5.0-SNAPSHOT]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [rt.jar:1.8.0_131]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [rt.jar:1.8.0_131]
	at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_131]

      Investigation showed that line [1] is called with variable targetNodeID=null which is not initialized after bridge is connected to target node (server2). This NPE kill reconnection logic. I tried to remove line [1] and core bridge successfully reconnected.

      [1] https://github.com/apache/activemq-artemis/blob/b66d0f7ac40001cce14ca7146e74720504ff9eb1/artemis-server/src/main/java/org/apache/activemq/artemis/core/server/cluster/impl/BridgeImpl.java#L782

      Attachments

        Issue Links

          blocks

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-10320 Upgrade artemis from 1.5.x to 2.x.x

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed
          duplicates

          Bug - A problem that impairs or prevents the functions of the product. WFWIP-13 Regression in messaging cluster tests with network failures

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Resolved

          Bug - A problem that impairs or prevents the functions of the product. ENTMQBR-1078 Regression in cluster tests with network failures

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. WFWIP-13 Regression in messaging cluster tests with network failures

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Resolved

          Bug - A problem that impairs or prevents the functions of the product. ENTMQBR-1078 Regression in cluster tests with network failures

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed

          Activity

            People

              mtaylor1@redhat.com Martyn Taylor (Inactive)
              mnovak1@redhat.com Miroslav Novak
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: