Uploaded image for project: 'Red Hat Data Grid'
  1. Red Hat Data Grid
  2. JDG-5028

Data race in EntryWrappingInterceptor handling expired entries

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • RHDG 8.3 GA
    • RHDG 8.2.1 GA
    • None
    • None
    • False
    • False
    • CR3
    • Hide

      Use putAsync instead of putAll, e.g.

            Cache<Object, Object> cache = ...;
      Map<Object, Object> m = ...;
      int size = m.size();
      CompletableFuture<?>[] putFutures = new CompletableFuture[size];
      int i = 0;
      for (Map.Entry<Object, Object> e : m.entrySet()) {
         putFutures[i++] = cache.putAsync(e.getKey(), e.getValue());
      }
      CompletableFuture.allOf(putFutures).get(timeout, TimeUnit.SECONDS);
      Show
      Use putAsync instead of putAll , e.g. Cache< Object , Object > cache = ...; Map< Object , Object > m = ...; int size = m.size(); CompletableFuture<?>[] putFutures = new CompletableFuture[size]; int i = 0; for (Map.Entry< Object , Object > e : m.entrySet()) { putFutures[i++] = cache.putAsync(e.getKey(), e.getValue()); } CompletableFuture.allOf(putFutures).get(timeout, TimeUnit.SECONDS);

    Description

      EntryWrappingInterceptor.visitPutMapCommand processes the potential expiration of keys in the input map in parallel, and wraps the entries in the context in the threads that processed the expiration response.

      But NonTxInvocationContext is not thread-safe, as it uses a HashMap internally. When a PutMapCommand affects 2 or more entries can can both expire, and their keys map to the same HashMap bucket, one of the updates may be lost and the distribution interceptor will later fail with an IllegalStateException: Entry should be always wrapped!.

      [org.infinispan.remoting.inboundhandler.NonTotalOrderPerCacheInboundInvocationHandler] (thread-623,ejb,agw17-253) ISPN000071: Caught exception when handling command SingleRpcCommand{cacheName='Cache', command=PutMapCommand{map={a=1, b=2, c=3, d=4}, flags=[IGNORE_RETURN_VALUES], metadata=EmbeddedExpirableMetadata{lifespan=1800000, maxIdle=1800000, version=null}, isForwarded=true}}: java.lang.IllegalStateException: Entry should be always wrapped!
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.getKeysByOwner(BaseDistributionInterceptor.java:598)
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.doRemoteGetMany(BaseDistributionInterceptor.java:364)
	at org.infinispan.interceptors.distribution.BaseDistributionInterceptor.remoteGetMany(BaseDistributionInterceptor.java:357)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleRemoteReadWriteManyCommand(NonTxDistributionInterceptor.java:419)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.handleReadWriteManyCommand(NonTxDistributionInterceptor.java:306)
	at org.infinispan.interceptors.distribution.NonTxDistributionInterceptor.visitPutMapCommand(NonTxDistributionInterceptor.java:146)
	at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:79)
	at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:55)
	at org.infinispan.interceptors.BaseAsyncInterceptor.lambda$new$0(BaseAsyncInterceptor.java:23)
	at org.infinispan.interceptors.InvocationSuccessFunction.apply(InvocationSuccessFunction.java:25)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:118)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.complete(CompletionStages.java:288)
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.accept(CompletionStages.java:256)
	at org.infinispan.util.concurrent.CompletionStages$AbstractAggregateCompletionStage.accept(CompletionStages.java:240)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.invokeQueuedHandlers(QueueAsyncInvocationStage.java:106)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:81)
	at org.infinispan.interceptors.impl.QueueAsyncInvocationStage.accept(QueueAsyncInvocationStage.java:30)
	at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) [rt.jar:1.8.0_212]
	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962) [rt.jar:1.8.0_212]
	at org.infinispan.remoting.transport.AbstractRequest.complete(AbstractRequest.java:67)
	at org.infinispan.remoting.transport.impl.SingleTargetRequest.receiveResponse(SingleTargetRequest.java:57)

      Attachments

        Issue Links

          causes

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-23085 (7.4.z) ISPN-13549 - Data race in EntryWrappingInterceptor handling expired entries

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified
          clones

          Bug - A problem that impairs or prevents the functions of the product. ISPN-13549 Data race in EntryWrappingInterceptor handling expired entries

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-7889 BaseDistributionInterceptor.remoteGet may cause concurrency issues

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          relates to

          Bug - A problem that impairs or prevents the functions of the product. JDG-5087 Deadlock when putAll writes to expired entries with optimistic locking

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • New

          Activity

            People

              dberinde@redhat.com Dan Berindei (Inactive)
              rhn-support-wfink Wolf Fink
              Diego Lovison Diego Lovison
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: