Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-3035

Members can re-appear by itself in the consistent hash after leaving

    XMLWordPrintable

Details

    • Bug
    • Resolution: Obsolete
    • Major
    • 9.0.0.Final
    • 5.2.5.Final, 5.3.0.Alpha1
    • State Transfer
    • None

    Description

      Seen as an intermittent failure in DataRehashedEventTest:

      2013-04-23 14:07:45,459 DEBUG (testng-DataRehashedEventTest) [org.infinispan.manager.DefaultCacheManager] Stopping cache manager ISPN on NodeC-58711
      2013-04-23 14:07:45,468 INFO  (testng-DataRehashedEventTest) [org.infinispan.remoting.transport.jgroups.JGroupsTransport] ISPN000080: Disconnecting and closing JGroups Channel
      2013-04-23 14:07:46,469 DEBUG (testng-DataRehashedEventTest) [org.jgroups.protocols.pbcast.GMS] NodeC-58711: sending LEAVE request to NodeA-28008
      2013-04-23 14:07:46,489 DEBUG (Incoming-2,ISPN,NodeA-28008) [org.jgroups.protocols.pbcast.GMS] NodeA-28008: installing [NodeA-28008|4] [NodeA-28008, NodeB-46156, NodeC-58711]
      2013-04-23 14:07:46,491 DEBUG (asyncTransportThread-0,NodeA) [org.infinispan.topology.ClusterTopologyManagerImpl] Starting cluster-wide rebalance for cache ___defaultcache, topology = CacheTopology{id=8, currentCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeA-28008, NodeB-46156]}, pendingCH=DefaultConsistentHash{numSegments=60, numOwners=2, members=[NodeA-28008, NodeB-46156, NodeC-58711]}}
      2013-04-23 14:07:49,493 ERROR (testng-DataRehashedEventTest) [org.infinispan.test.fwk.UnitTestTestNGListener] Test testJoinAndLeave(org.infinispan.statetransfer.DataRehashedEventTest) failed.
      java.lang.AssertionError: expected [2] but found [6]
          at org.testng.Assert.fail(Assert.java:94)
          at org.testng.Assert.failNotEquals(Assert.java:494)
          at org.testng.Assert.assertEquals(Assert.java:123)
          at org.testng.Assert.assertEquals(Assert.java:370)
          at org.testng.Assert.assertEquals(Assert.java:380)
          at org.infinispan.statetransfer.DataRehashedEventTest.testJoinAndLeave(DataRehashedEventTest.java:114)
      

      The initial cluster has 3 nodes: A, B, C. C is killed, but somehow remains in the ClusterCacheStatus on the coordinator.

      Then C re-appears in the JGroups view (possibly a JGroups issue). The problem in Infinispan is that the coordinator now sees C as a joiner, and it rebalances the cache to include C in the consistent hash again.

      Attachments

        1. dret.log
          59 kB
        2. dret2.log
          237 kB

        Issue Links

          Activity

            People

              Unassigned Unassigned
              dberinde@redhat.com Dan Berindei (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: