Uploaded image for project: 'JBoss Enterprise Application Platform 4 and 5'
  1. JBoss Enterprise Application Platform 4 and 5
  2. JBPAPP-2292

Channel reconnect in a cluster on clean shutdown and boot fails

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • EAP 5.0.0.CR2
    • EAP 5.0.0.BETA
    • Clustering
    • None

    • Brno's jawas

      Server have problem reconnecting after some have been shut down and started again.

      Test scenario:

      1/ all nodes start up
      2/ clients starts sending requests

      <->
      3/ node x shuts down
      4/ node x starts up

      Problem occurs when (in this case 3rd) node tries to join the cluster.

      http://volga.englab.brq.redhat.com/hudson/view/eap5-failover-matrix/job/eap5-failover-matrix-http-clean/SF_BUDDY_REPL=true,SF_CACHE_MODE=REPL_SYNC,jdk=java16_default,label=jawa11/49/console-jawa03/

      Excerpt from logging:

      [JBoss] 14:58:22,671 TRACE [org.jgroups.protocols.pbcast.GMS] processing [SUSPECT(10.34.32.156:47225)]
      [JBoss] 14:58:22,671 TRACE [org.jgroups.protocols.pbcast.GMS] processing [SUSPECT(10.34.32.156:47225)]
      [JBoss] 14:58:22,935 TRACE [org.jgroups.protocols.pbcast.GMS] processing [SUSPECT(10.34.32.156:47225)]
      [JBoss] 14:58:22,935 TRACE [org.jgroups.protocols.pbcast.GMS] processing [SUSPECT(10.34.32.156:47225)]
      [JBoss] 14:58:25,271 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.156:47225 timed out (after 3000 ms), retrying
      [JBoss] 14:58:25,271 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.156:47225 timed out (after 3000 ms), retrying
      [JBoss] 14:58:25,272 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=false], [own_addr=10.34.32.154:59773, coord_addr=10.34.32.154:59773, is_server=false], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:25,272 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=false], [own_addr=10.34.32.154:59773, coord_addr=10.34.32.154:59773, is_server=false], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:25,273 DEBUG [org.jgroups.protocols.pbcast.GMS] election results:

      {10.34.32.156:47225=1}

      [JBoss] 14:58:25,273 DEBUG [org.jgroups.protocols.pbcast.GMS] election results:

      {10.34.32.156:47225=1}

      [JBoss] 14:58:25,273 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:25,273 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:25,339 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.153:33086 timed out (after 3000 ms), retrying
      [JBoss] 14:58:25,339 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.153:33086 timed out (after 3000 ms), retrying
      [JBoss] 14:58:25,340 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=true], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:25,340 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=true], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:25,340 WARN [org.jgroups.protocols.pbcast.GMS] there was more than 1 candidate for coordinator:

      {10.34.32.156:47225=1, 10.34.32.153:33086=1}

      [JBoss] 14:58:25,340 WARN [org.jgroups.protocols.pbcast.GMS] there was more than 1 candidate for coordinator:

      {10.34.32.156:47225=1, 10.34.32.153:33086=1}

      [JBoss] 14:58:25,340 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:25,340 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:28,275 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.156:47225 timed out (after 3000 ms), retrying
      [JBoss] 14:58:28,275 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.156:47225 timed out (after 3000 ms), retrying
      [JBoss] 14:58:28,280 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=false], [own_addr=10.34.32.154:59773, coord_addr=10.34.32.154:59773, is_server=false], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:28,280 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=false], [own_addr=10.34.32.154:59773, coord_addr=10.34.32.154:59773, is_server=false], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:28,280 DEBUG [org.jgroups.protocols.pbcast.GMS] election results:

      {10.34.32.156:47225=1}

      [JBoss] 14:58:28,280 DEBUG [org.jgroups.protocols.pbcast.GMS] election results:

      {10.34.32.156:47225=1}

      [JBoss] 14:58:28,280 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:28,280 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:28,343 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.156:47225 timed out (after 3000 ms), retrying
      [JBoss] 14:58:28,343 WARN [org.jgroups.protocols.pbcast.GMS] join(10.34.32.155:50281) sent to 10.34.32.156:47225 timed out (after 3000 ms), retrying
      [JBoss] 14:58:28,344 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=true], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:28,344 DEBUG [org.jgroups.protocols.pbcast.GMS] initial_mbrs are [[own_addr=10.34.32.153:33086, coord_addr=10.34.32.153:33086, is_server=true], [own_addr=10.34.32.156:47225, coord_addr=10.34.32.156:47225, is_server=true]]
      [JBoss] 14:58:28,344 WARN [org.jgroups.protocols.pbcast.GMS] there was more than 1 candidate for coordinator:

      {10.34.32.156:47225=1, 10.34.32.153:33086=1}

      [JBoss] 14:58:28,344 WARN [org.jgroups.protocols.pbcast.GMS] there was more than 1 candidate for coordinator:

      {10.34.32.156:47225=1, 10.34.32.153:33086=1}

      [JBoss] 14:58:28,344 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225
      [JBoss] 14:58:28,344 DEBUG [org.jgroups.protocols.pbcast.GMS] sending handleJoin(10.34.32.155:50281) to 10.34.32.156:47225

              pferraro@redhat.com Paul Ferraro
              rhn-engineering-rhusar Radoslav Husar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: