Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-13549

Data race in EntryWrappingInterceptor handling expired entries

    XMLWordPrintable

Details

    Description

      EntryWrappingInterceptor.visitPutMapCommand processes the potential expiration of keys in the input map in parallel, and wraps the entries in the context in the threads that processed the expiration response.

      But NonTxInvocationContext is not thread-safe, as it uses a HashMap internally. When a PutMapCommand affects 2 or more entries can can both expire, and their keys map to the same HashMap bucket, one of the updates may be lost and the distribution interceptor will later fail with an IllegalStateException: Entry should be always wrapped!.

      [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (thread-623,ejb,agw17-253) ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='Cache', command=PutMapCommand{map={a=1, b=2, c=3, d=4}, flags=[IGNORE_RETURN_VALUES], metadata=EmbeddedExpirableMetadata{lifespan=1800000, maxIdle=1800000, version=null}, isForwarded=true}}: java.lang.IllegalStateException: Entry should be always wrapped!
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.getKeysByOwner(BaseDistributionInterceptor.java:598)
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.doRemoteGetMany(BaseDistributionInterceptor.java:364)
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.remoteGetMany(BaseDistributionInterceptor.java:357)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleRemoteReadWriteManyCommand(NonTxDistributionInterceptor.java:419)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleReadWriteManyCommand(NonTxDistributionInterceptor.java:306)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutMapCommand(NonTxDistributionInterceptor.java:146)
	at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:79)
	at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:55)
	at org.infinispan.interceptors.BaseAsyncInterceptor.lambda$new$0(BaseAsyncInterceptor.java:23)
	at org.infinispan.interceptors.InvocationSuccessFunction.apply(InvocationSuccessFunction.java:25)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.complete(CompletionStages.java:288)
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.accept(CompletionStages.java:256)
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.accept(CompletionStages.java:240)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:106)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:67)
	at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:57)

      Attachments

        Issue Links

          causes

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-23095 (8.0.z) Data race in EntryWrappingInterceptor handling expired entries

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-23085 (7.4.z) ISPN-13549 - Data race in EntryWrappingInterceptor handling expired entries

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified

          Bug - A problem that impairs or prevents the functions of the product. JDG-5028 Data race in EntryWrappingInterceptor handling expired entries

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is incorporated by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFLY-15939 Upgrade Infinispan to 13.0.6.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-7889 BaseDistributionInterceptor.remoteGet may cause concurrency issues

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-13619 Deadlock when putAll writes to expired entries in a scattered cache

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • To Do

          Bug - A problem that impairs or prevents the functions of the product. ISPN-13618 Deadlock when putAll writes to expired entries with optimistic locking

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • In Progress

          Activity

            People

              dberinde@redhat.com Dan Berindei (Inactive)
              dberinde@redhat.com Dan Berindei (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: