Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-14030

locked keys after node crash

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • 8.1.7.Final, 12.1.7.Final, 13.0.10.Final
    • Transactions
    • None
      1. build and deploy demonstrator app to two appservers on different nodes
      2. use `client.sh` to start multiple clients accessing
      3. power off a single node
      4. wait 15s and inspect the logs of the remaining node

      In a two node cluster scenario, in a Java EE server (Weblogic 12.2.1.4), with a replicated-cache in REPETABLE_READ isolation level, we see the following exceptions when one node crashes (hosting VM is turned off):

      Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 15 seconds for key WrappedByteArray[ACED0005\t0001\9 (8 bytes)] and requestor GlobalTransaction\{id=496, addr=HOST1-43695, remote=false, xid=Xid{formatId=48801, globalTransactionId=01EF088AA6508A806BC5,branchQualifier=6F72672E696E66696E697370616E2E7472616E73616374696F6E2E78612E5472616E73616374696F6E586141646170746572}, internalId=562954248389104}. Lock is held by GlobalTransaction\{id=337, addr=BAGOSACP3x06-43695, remote=false, xid=Xid{formatId=48801, globalTransactionId=0150088AA6508A806BC5,branchQualifier=6F72672E696E66696E697370616E2E7472616E73616374696F6E2E78612E5472616E73616374696F6E586141646170746572}, internalId=281479271678289} (pending) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.timeout(DefaultPendingLockManager.java:148) at org.infinispan.util.concurrent.locks.impl.DefaultPendingLockManager.access$200(DefaultPendingLockManager.java:49) ... 9 more .
      

      We created a demonstration app. Please find it on github.

      The lock seems to be never released (we waited longer than the configured JTA timeout) and the cache keys stay blocked for writing indefinitely.

      Please note, that we are aware of issues: ISPN-13219, ISPN-13352, ISPN-13537 and ISPN-12714 - but as there was no response to these, we decided to create a new one.

      Please let us know, if you need additional information.

              pruivo@redhat.com Pedro Ruivo
              schosven Sven Schober (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: