Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-11093

WriteSkewException possible with concurrent passivation

    XMLWordPrintable

Details

    • Bug
    • Resolution: Obsolete
    • Major
    • None
    • 10.1.0.CR1
    • Core
    • DataGrid Sprint #38, DataGrid Sprint #39

    Description

      VersionedRepeatableReadEntry.skewed() replaces the context entry's version with the version returned by CacheLoaderInterceptor.loadAndStoreInDataContainer(), apparently for the benefit of total order transactions, but running for all transactions, not just total order. Somehow loadAndStoreInDataContainer() can miss the entries in PassivationPersistenceManager's temporary passivation map, causing random failures in WriteSkewGetGroupKeysTest.testRemoveGroupKeysWithPersistenceAndPassivation[NON_OWNER, DIST_SYNC, TO=true, isolation=REPEATABLE_READ].

      12:48:34,192 TRACE (async-thread-Test-NodeA-p4378-t4:[]) [PassivationPersistenceManager] Storing entry temporarily during passivation for key GroupKey{group='test-group', key=5}
      12:48:34,192 TRACE (async-thread-Test-NodeA-p4378-t4:[]) [PersistenceManagerImpl] Writing to all stores for id 11566
      12:48:34,192 TRACE (async-thread-Test-NodeA-p4378-t4:[]) [VersionedRepeatableReadEntry] No entry for key GroupKey{group='test-group', key=5} found in data container
      12:48:34,192 TRACE (async-thread-Test-NodeA-p4378-t4:[]) [VersionedRepeatableReadEntry] Updating version in metadata SimpleClusteredVersion{topologyId=9, version=1} -> SimpleClusteredVersion{topologyId=0, version=0}
      12:48:34,192 TRACE (async-thread-Test-NodeA-p4378-t4:[]) [VersionedRepeatableReadEntry] Comparing versions SimpleClusteredVersion{topologyId=0, version=0} and SimpleClusteredVersion{topologyId=9, version=1} for key GroupKey{group='test-group', key=5}: BEFORE
      12:48:34,193 TRACE (persistence-thread-Test-NodeA-p4387-t3:[]) [DummyInMemoryStore] Store MarshallableEntryImpl{keyBytes=ByteBufferImpl{length=67, offset=0, bytes=8201\/\o\r\g\.\i\n\f\i\n\i\s\p\a\n\.\t\e\s\t\.\c\o\r\e\.\G\r\o\u\p\T\e\s\t\s\S\C\I\.\G\r\o\u\p\K\e\y8A010E0A0A\t\e\s\t\-\g\r\o\u\p1005}, valueBytes=ByteBufferImpl{length=4, offset=0, bytes=\J02\v\5}, metadataBytes=ByteBufferImpl{length=12, offset=0, bytes=9801048A0106120408091001}, key=GroupKey{group='test-group', key=5}, value=v5, metadata=EmbeddedMetadata{version=SimpleClusteredVersion{topologyId=9, version=1}}, created=-1, lastUsed=-1, marshaller=org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl@64ddf323} in dummy map store@21df61dc
      12:48:34,194 TRACE (async-thread-Test-NodeA-p4378-t6:[]) [CompletionStages] Continuing execution of id 11566
      12:48:34,194 TRACE (async-thread-Test-NodeA-p4378-t6:[]) [PassivationPersistenceManager] Removed temporary entry during passivation for key GroupKey{group='test-group', key=5}
      12:48:34,207 TRACE (async-thread-Test-NodeA-p4378-t4:[]) [TotalOrderRemoteTransactionState] [Test-NodeB-45589:2742] Current status is [PREPARING], setting status to: PREPARED
      12:48:34,207 DEBUG (async-thread-Test-NodeA-p4378-t4:[]) [TotalOrderInterceptor] Exception while preparing for transaction Test-NodeB-45589:2742. Local=false
      org.infinispan.transaction.WriteSkewException: Write skew detected on key GroupKey{group='test-group', key=5} for transaction RemoteTransaction{modifications=[..., RemoveCommand{key=GroupKey{group='test-group', key=5},
      value=null, metadata=null, flags=[IGNORE_RETURN_VALUES], commandInvocationId=CommandInvocation:local:0, valueMatcher=MATCH_ALWAYS, topologyId=9}, ...], lookedUpEntries={..., GroupKey{group='test-group', key=5}=VersionedRepeatableReadEntry(66deadfb){key=GroupKey{group='test-group', key=5}, value=null, isCreated=false, isChanged=true, isRemoved=true, isExpired=false, skipLookup=true, metadata=EmbeddedMetadata{version=SimpleClusteredVersion{topologyId=0, version=0}}}, ...}, lockedKeys=[], backupKeyLocks=[], lookedUpEntriesTopology=9, isMarkedForRollback=false, tx=GlobalTx:Test-NodeB-45589:2742, state=TotalOrderRemoteTransactionState{transactionState=[PREPARING], globalTransaction='Test-NodeB-45589:2742'}}
      	at org.infinispan.transaction.impl.WriteSkewHelper.lambda$performTotalOrderWriteSkewCheckAndReturnNewVersions$1(WriteSkewHelper.java:113) ~[classes/:?]
      
      12:48:34,248 ERROR (testng-Test:[]) [TestSuiteProgress] Test failed: org.infinispan.distribution.groups.WriteSkewGetGroupKeysTest.testRemoveGroupKeysWithPersistenceAndPassivation[NON_OWNER, DIST_SYNC, TO=true, isolation=REPEATABLE_READ]
      org.infinispan.commons.CacheException: Could not commit implicit transaction
      	at org.infinispan.cache.impl.CacheImpl.tryCommit(CacheImpl.java:1940) ~[classes/:?]
      	at org.infinispan.cache.impl.CacheImpl.transactionalRemoveGroup(CacheImpl.java:655) ~[classes/:?]
      	at org.infinispan.cache.impl.CacheImpl.removeGroup(CacheImpl.java:638) ~[classes/:?]
      	at org.infinispan.cache.impl.CacheImpl.removeGroup(CacheImpl.java:631) ~[classes/:?]
      	at org.infinispan.cache.impl.AbstractDelegatingAdvancedCache.removeGroup(AbstractDelegatingAdvancedCache.java:303) ~[classes/:?]
      	at org.infinispan.distribution.groups.GetGroupKeysTest.testRemoveGroupKeysWithPersistenceAndPassivation(GetGroupKeysTest.java:166) ~[test-classes/:?]
      

      FWIW the trace log of the test doesn't include any line with the text "Retrieved entry for key %s from temporary passivation map", for any key.

      Attachments

        Activity

          People

            wburns@redhat.com Will Burns
            dberinde@redhat.com Dan Berindei (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: