Uploaded image for project: 'ModeShape'
  1. ModeShape
  2. MODE-2484

Possible Infinispan deadlock when processing binary values

    XMLWordPrintable

Details

    Description

      When setting binary values on properties, ModeShape uses a reference counting mechanism to optimize multiple properties using the same value.
      This mechanism is implemented via "special" ISPN documents which hold the SHA1 of the binary value and the BinaryKey.

      Currently, whenever a binary usage changes, ModeShape will attempt to read & update this document from ISPN within the context of an existing transaction. When making a set of changes, ModeShape preemptively locks the nodes in ISPN which are part of those changes to ensure strong serializable consistency. Those locks are places only on node keys however, not on binary keys. The binary keys are locked later, one by one, during the logic which processes the node changes.

      The above means that there can be 2 transactions: T1 and T2, both operating on disjoint nodes keys which can intersect on the binary keys they lock later on. This intersection can lead to ISPN deadlocks, resulting in exceptions similar to:

      00:00:01,117 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (OOB-6,shared=tcp) ISPN000136: Execution error: org.infinispan.util.concurrent.TimeoutException: Unable to acquire lock after [45 seconds] on key [9e504611ee3f17e8ab6265af313358d7ad487b5d-ref] for requestor [GlobalTransaction:<dvqe08:server08/modeshape>:288724:remote]! Lock held by [GlobalTransaction:<dvqe10:server10/modeshape>:59532:remote]
      	at org.infinispan.util.concurrent.locks.LockManagerImpl.lock(LockManagerImpl.java:213)
      	at org.infinispan.util.concurrent.locks.LockManagerImpl.acquireLock(LockManagerImpl.java:186)
      	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockKeyAndCheckOwnership(AbstractTxLockingInterceptor.java:192)
      	at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAndRegisterBackupLock(AbstractTxLockingInterceptor.java:129)
      	at org.infinispan.interceptors.locking.PessimisticLockingInterceptor.visitLockControlCommand(PessimisticLockingInterceptor.java:253)
      	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
      	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
      	at org.infinispan.commands.AbstractVisitor.visitLockControlCommand(AbstractVisitor.java:177)
      	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
      	at org.infinispan.interceptors.TxInterceptor.invokeNextInterceptorAndVerifyTransaction(TxInterceptor.java:128)
      	at org.infinispan.interceptors.TxInterceptor.visitLockControlCommand(TxInterceptor.java:186)
      	at org.infinispan.commands.control.LockControlCommand.acceptVisitor(LockControlCommand.java:131)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:120)
      	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:134)
      

      Attachments

        Activity

          People

            hchiorean Horia Chiorean (Inactive)
            hchiorean Horia Chiorean (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: