Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-8480

Data Inconsistency in case of Topology change in Infinispan Cluster

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

      Data Inconsistency in case of Topology change in Infinispan Cluster

      Infinispan Version : 8.2.5
      Hibernate Version : 5.2.8
      JGROUPS Version : 3.6.7

      Clustering Mode : Replication
      We have tested the same with invalidation mode too.

      Refer below config cache-config for hibernate L2 entity types:

      <replicated-cache-configuration name="entity" mode="SYNC" remote-timeout="20000" statistics="false" statistics-available="false">
      <state-transfer enabled="false" timeout="20000000"/>
      <locking isolation="READ_COMMITTED" concurrency-level="1000" acquire-timeout="15000" striping="false"/>
      <transaction mode="NONE" auto-commit="false" locking="OPTIMISTIC"/>
      <eviction size="-1" strategy="NONE"/>
      <expiration max-idle="-1" interval="5000" lifespan="-1" />
      </replicated-cache-configuration>

      When a disconnected node rejoins the cluster, data remains inconsistent on the reconnected node.
      This is happening for both Hibernate L2 Cache and some custom cache (AdvancedCache).
      The below 4 scenarios should explain the issue.

      For Example:
      Scenario (Issue) 1:
      -Initially the cluster comprises of 4 nodes, namely

      {A,B,C,D}.
      -Somehow, node D gets removed from the cluster view.
      -Then some updates/inserts in hibernate L2 cache is done on Node B.
      -These updates/inserts gets propagated to all the nodes in the current cluster view, i.e. {A,B,C}.
      -And these updates/inserts doesn't get propagated to the Node D.
      -Now D has stale state of the L2 cache.
      -We expect, Node D should get the updated state of L2 Cache from {A,B,C}.

      Scenario (Issue) 2:
      -Initially the cluster comprises of 4 nodes, namely {A,B,C,D}

      .
      -Somehow, node D gets removed from the cluster view.
      -Then some updates/inserts in hibernate L2 cache is done on Node B.
      -These updates/inserts gets propagated to all the nodes in the current cluster view, i.e.

      {A,B,C}.
      -And these updates/inserts doesn't get propagated to the Node D.
      -Now D has stale state of the L2 cache.
      -Now D rejoins the cluster {A,B,C}

      .
      -Now the updated cluster view is

      {A,B,C,D}.
      -Still D has stale state of the L2 cache.
      -We expect, Node D should get the updated state of L2 Cache from {A,B,C}.

      Scenario (Issue) 3:
      -Initially the cluster comprises of 4 nodes, namely {A,B,C,D}

      .
      -Somehow, node D gets removed from the cluster view.
      -Then some updates/inserts in hibernate L2 cache is done on Node B.
      -These updates/inserts gets propagated to all the nodes in the current cluster view, i.e.

      {A,B,C}.
      -And these updates/inserts doesn't get propagated to the Node D.
      -Subsequently, some updates/inserts in hibernate L2 cache is done on Node D too. These updates are done on some other keys, and not on the keys on which the updates were done by Node B.
      -Now {A,B,C}

      and Node D have different but updated state of the L2 cache, And are not in sync.
      -Now D rejoins the cluster

      {A,B,C}.
      -Now the updated cluster view is {A,B,C,D}.
      -Still {A,B,C}

      and Node D have different but updated state of the L2 cache, And are not in sync.
      -We expect, updates from

      {A,B,C} and Node D should get merged to all the nodes in cluster.

      Scenario (Issue) 4:
      -Initially the cluster comprises of 4 nodes, namely {A,B,C,D}.
      -Somehow, node D gets removed from the cluster view.
      -Then some updates/inserts in hibernate L2 cache is done on Node B.
      -These updates/inserts gets propagated to all the nodes in the current cluster view, i.e. {A,B,C}

      .
      -And these updates/inserts doesn't get propagated to the Node D.
      -Subsequently, some updates/inserts in hibernate L2 cache is done on Node D too. These updates might be on the same keys on which the updates were done by Node B.
      -Now D has more updated state of the L2 cache.
      -And

      {A,B,C} are having stale state of the L2 Cache.
      -Now D rejoins the cluster {A,B,C}

      .
      -Now the updated cluster view is

      {A,B,C,D}

      .
      -Still

      {A,B,C} have stale state of the L2 cache.
      -We expect, {A,B,C}

      should get the updated state of L2 Cache from Node D.

              Unassigned Unassigned
              rohit6600 Rohit Singh (Inactive)
              Archiver:
              rhn-support-adongare Amol Dongare

                Created:
                Updated:
                Resolved:
                Archived: