Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-277

LRU data container endlesly looping or exhibiting heavy contention

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • 4.0.0.CR3
    • 4.0.0.CR2
    • Eviction
    • None
    • High

    Description

      Something around LRU container is not working fine. The attached log from an concurrency test in the 2nd level cache shows that in 3 thread dumps taken over 30 seconds appart, UserRunnerThread-5 is stuck in:

      "UserRunnerThread-5" prio=10 tid=0x6f65bc00 nid=0xdea runnable [0x05efb000]
      java.lang.Thread.State: RUNNABLE
      at org.infinispan.container.FIFODataContainer$LinkedEntry.casNext(FIFODataContainer.java:180)
      at org.infinispan.container.FIFODataContainer.correctPrev(FIFODataContainer.java:329)
      at org.infinispan.container.FIFODataContainer.linkAtEnd(FIFODataContainer.java:252)
      at org.infinispan.container.LRUDataContainer.put(LRUDataContainer.java:70)
      at org.infinispan.container.entries.ReadCommittedEntry.commit(ReadCommittedEntry.java:161)
      at org.infinispan.interceptors.LockingInterceptor.commitEntry(LockingInterceptor.java:298)
      at org.infinispan.interceptors.LockingInterceptor.cleanupLocks(LockingInterceptor.java:281)
      at org.infinispan.interceptors.LockingInterceptor.doAfterCall(LockingInterceptor.java:243)
      at org.infinispan.interceptors.LockingInterceptor.visitPutKeyValueCommand(LockingInterceptor.java:200)
      at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
      at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:132)
      at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
      at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
      at org.infinispan.interceptors.MarshalledValueInterceptor.visitPutKeyValueCommand(MarshalledValueInterceptor.java:93)
      at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
      at org.infinispan.interceptors.TxInterceptor.enlistWriteAndInvokeNext(TxInterceptor.java:185)
      at org.infinispan.interceptors.TxInterceptor.visitPutKeyValueCommand(TxInterceptor.java:132)
      at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
      at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118)
      at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:48)
      at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:34)
      at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:57)
      at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:76)
      at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:269)
      at org.infinispan.CacheDelegate.putIfAbsent(CacheDelegate.java:422)
      at org.infinispan.CacheDelegate.putIfAbsent(CacheDelegate.java:153)
      at org.infinispan.CacheDelegate.putForExternalRead(CacheDelegate.java:243)
      at org.hibernate.cache.infinispan.util.CacheAdapterImpl.putForExternalRead(CacheAdapterImpl.java:115)
      at org.hibernate.cache.infinispan.access.TransactionalAccessDelegate.putFromLoad(TransactionalAccessDelegate.java:91)
      at org.hibernate.cache.infinispan.collection.TransactionalAccess.putFromLoad(TransactionalAccess.java:44)
      at org.hibernate.engine.loading.CollectionLoadContext.addCollectionToCache(CollectionLoadContext.java:333)
      at org.hibernate.engine.loading.CollectionLoadContext.endLoadingCollection(CollectionLoadContext.java:279)
      at org.hibernate.engine.loading.CollectionLoadContext.endLoadingCollections(CollectionLoadContext.java:245)
      at org.hibernate.engine.loading.CollectionLoadContext.endLoadingCollections(CollectionLoadContext.java:218)
      at org.hibernate.loader.Loader.endCollectionLoad(Loader.java:901)
      at org.hibernate.loader.Loader.initializeEntitiesAndCollections(Loader.java:886)
      at org.hibernate.loader.Loader.doQuery(Loader.java:750)
      at org.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:257)
      at org.hibernate.loader.Loader.loadCollection(Loader.java:2019)
      at org.hibernate.loader.collection.CollectionLoader.initialize(CollectionLoader.java:62)
      at org.hibernate.persister.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:628)
      at org.hibernate.event.def.DefaultInitializeCollectionEventListener.onInitializeCollection(DefaultInitializeCollectionEventListener.java:83)
      at org.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:1817)
      at org.hibernate.collection.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:366)
      at org.hibernate.collection.AbstractPersistentCollection.read(AbstractPersistentCollection.java:108)
      at org.hibernate.collection.AbstractPersistentCollection.readSize(AbstractPersistentCollection.java:131)
      at org.hibernate.collection.PersistentSet.isEmpty(PersistentSet.java:169)
      at org.hibernate.test.cache.infinispan.functional.ConcurrentWriteTest.getFirstContact(ConcurrentWriteTest.java:405)
      at org.hibernate.test.cache.infinispan.functional.ConcurrentWriteTest.access$0(ConcurrentWriteTest.java:397)
      at org.hibernate.test.cache.infinispan.functional.ConcurrentWriteTest$UserRunner.contactExists(ConcurrentWriteTest.java:519)
      at org.hibernate.test.cache.infinispan.functional.ConcurrentWriteTest$UserRunner.call(ConcurrentWriteTest.java:540)
      at org.hibernate.test.cache.infinispan.functional.ConcurrentWriteTest$UserRunner.call(ConcurrentWriteTest.java:1)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
      at java.lang.Thread.run(Thread.java:636)

      This concurrency test does not get to finish in over a minute. However, once LRU is switched to NONE or FIFO, the test runs in 5 seconds.
      So, something looks fishy with LRU.

      Attachments

        1. 3dumps-fail.txt
          280 kB
        2. td2.txt
          250 kB

        Issue Links

          Activity

            People

              manik_jira Manik Surtani (Inactive)
              rh-ee-galder Galder Zamarreño
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: