Uploaded image for project: 'Red Hat Data Grid'
  1. Red Hat Data Grid
  2. JDG-5028

Data race in EntryWrappingInterceptor handling expired entries

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • RHDG 8.3 GA
    • RHDG 8.2.1 GA
    • None
    • None
    • False
    • False
    • CR3
    • Hide

      Use putAsync instead of putAll, e.g.

            Cache<Object, Object> cache = ...;
      Map<Object, Object> m = ...;
      int size = m.size();
      CompletableFuture<?>[] putFutures = new CompletableFuture[size];
      int i = 0;
      for (Map.Entry<Object, Object> e : m.entrySet()) {
         putFutures[i++] = cache.putAsync(e.getKey(), e.getValue());
      }
      CompletableFuture.allOf(putFutures).get(timeout, TimeUnit.SECONDS);
      Show
      Use putAsync instead of putAll , e.g. Cache< Object , Object > cache = ...; Map< Object , Object > m = ...; int size = m.size(); CompletableFuture<?>[] putFutures = new CompletableFuture[size]; int i = 0; for (Map.Entry< Object , Object > e : m.entrySet()) { putFutures[i++] = cache.putAsync(e.getKey(), e.getValue()); } CompletableFuture.allOf(putFutures).get(timeout, TimeUnit.SECONDS);

      EntryWrappingInterceptor.visitPutMapCommand processes the potential expiration of keys in the input map in parallel, and wraps the entries in the context in the threads that processed the expiration response.

      But NonTxInvocationContext is not thread-safe, as it uses a HashMap internally. When a PutMapCommand affects 2 or more entries can can both expire, and their keys map to the same HashMap bucket, one of the updates may be lost and the distribution interceptor will later fail with an IllegalStateException: Entry should be always wrapped!.

      [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (thread-623,ejb,agw17-253) ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='Cache', command=PutMapCommand{map={a=1, b=2, c=3, d=4}, flags=[IGNORE_RETURN_VALUES], metadata=EmbeddedExpirableMetadata{lifespan=1800000, maxIdle=1800000, version=null}, isForwarded=true}}: java.lang.IllegalStateException: Entry should be always wrapped!
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.getKeysByOwner(BaseDistributionInterceptor.java:598)
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.doRemoteGetMany(BaseDistributionInterceptor.java:364)
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.remoteGetMany(BaseDistributionInterceptor.java:357)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleRemoteReadWriteManyCommand(NonTxDistributionInterceptor.java:419)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleReadWriteManyCommand(NonTxDistributionInterceptor.java:306)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutMapCommand(NonTxDistributionInterceptor.java:146)
	at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:79)
	at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:55)
	at org.infinispan.interceptors.BaseAsyncInterceptor.lambda$new$0(BaseAsyncInterceptor.java:23)
	at org.infinispan.interceptors.InvocationSuccessFunction.apply(InvocationSuccessFunction.java:25)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.complete(CompletionStages.java:288)
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.accept(CompletionStages.java:256)
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.accept(CompletionStages.java:240)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:106)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:67)
	at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:57)

              dberinde@redhat.com Dan Berindei (Inactive)
              rhn-support-wfink Wolf Fink
              Diego Lovison Diego Lovison
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: