Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-13362

State transfer errors with rebalancing off and multiple nodes removed

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • 13.0.0.C2
    • None
    • None

      Create the Cluster
      Add entries in the cache
      Change replicas to 3
      Disable Rebalance via ISPN Console UI
      Kill pod0 and pod1
      The cluster become unusable

      apiVersion: infinispan.org/v1
      kind: Infinispan
      metadata:
        name: example-clustera
      spec:
        replicas: 2
        expose:
          type: Route
        security:
          endpointSecretName: connect-secret-clustera
        service:
          type: DataGrid
          sites:
            local:
              name: SiteA
              expose:
                type: ClusterIP
            locations:
              - name: SiteA
                clusterName: example-clustera
                namespace: dlovison-v4
              - name: SiteB
                clusterName: example-clusterb
                namespace: dlovison-v4
      
      apiVersion: v1
      kind: Secret
      metadata:
        name: connect-secret-clustera
      type: Opaque
      stringData:
        identities.yaml: |-
          credentials:
          - username: user-sitea
            password: pwd-sitea
      
      apiVersion: infinispan.org/v2alpha1
      kind: Cache
      metadata:
        name: mycache-site-a 
      spec:
        clusterName: example-clustera
        name: mycache 
        template: |-
          <distributed-cache name="mycache" mode="SYNC">
            <encoding>
              <key media-type="application/x-protostream"/>
              <value media-type="application/x-protostream"/>
            </encoding>
            <backups>
              <backup site="SiteB" strategy="SYNC" timeout="5000">
                <take-offline after-failures="10" min-wait="10000" />
              </backup>
            </backups>
          </distributed-cache>
      
      17:48:34,452 ERROR (timeout-thread--p4-t1) [org.infinispan.interceptors.impl.InvocationContextInterceptor] ISPN000136: Error executing command SizeCommand on Cache 'mycache', writing keys [] org.infinispan.util.concurrent.TimeoutException: ISPN000637: Timeout waiting for topology 21, current topology is 20
          at org.infinispan.statetransfer.StateTransferLockImpl.lambda$topologyFuture$4(StateTransferLockImpl.java:173)
          at org.infinispan.util.concurrent.ConditionFuture.lambda$newConditionStage$0(ConditionFuture.java:59)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:829)
      

            Unassigned Unassigned
            dlovison@redhat.com Diego Lovison
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: