Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-3779

Error executing command PrepareCommand/GetKeyValueCommand due to replication timeout

    Details

    • Type: Bug
    • Status: Verified (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 7.0.0.ER6, 7.0.1.GA, 7.1.0.DR2
    • Fix Version/s: 7.1.0.DR9
    • Component/s: Clustering
    • Labels:
      None
    • Target Release:
    • Affects:
      Release Notes
    • Release Notes Docs Status:
      Documented as Known Issue

      Description

      This is a separate issue originally logged as part of JBEAP-794 (#2 in this comment).

      It comes with 2 different messages:

      1. Error executing command GetKeyValueCommand
      [JBossINF] 03:23:38,763 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-92) ISPN000136: Error executing command GetKeyValueCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf19
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$175(JGroupsTransport.java:612)
      [JBossINF] 	at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
      [JBossINF] 	at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
      [JBossINF] 	at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      [JBossINF] 	at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
      [JBossINF] 	at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
      [JBossINF] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      [JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      [JBossINF] 	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      [JBossINF] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      [JBossINF] 	at java.lang.Thread.run(Thread.java:745)
      

      This one appears exactly in the same scenarios and numbers as JBEAP-3696 - see occurrences report
      (ejb-ejbservlet and http-session scenarios with REPL cache)

      2. Error executing command PrepareCommand
      [JBossINF] [0m[31m03:23:38,454 ERROR [org.infinispan.interceptors.InvocationContextInterceptor] (default task-24) ISPN000136: Error executing command PrepareCommand, writing keys [UOzA9Tq6F40Y0pcaLUpLe6YLgsYPuFE3SZQoH7iW, UOzA9Tq6F40Y0pcaLUpLe6YLgsYPuFE3SZQoH7iW, UOzA9Tq6F40Y0pcaLUpLe6YLgsYPuFE3SZQoH7iW]: org.infinispan.util.concurrent.TimeoutException: Replication timeout for perf19
      [JBossINF]     at org.infinispan.remoting.transport.jgroups.JGroupsTransport.checkRsp(JGroupsTransport.java:765)
      [JBossINF]     at org.infinispan.remoting.transport.jgroups.JGroupsTransport.lambda$invokeRemotelyAsync$175(JGroupsTransport.java:612)
      [JBossINF]     at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602)
      [JBossINF]     at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:577)
      [JBossINF]     at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474)
      [JBossINF]     at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1962)
      [JBossINF]     at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:47)
      [JBossINF]     at org.infinispan.remoting.transport.jgroups.RspListFuture.call(RspListFuture.java:16)
      [JBossINF]     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      [JBossINF]     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      [JBossINF]     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      [JBossINF]     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      [JBossINF]     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      [JBossINF]     at java.lang.Thread.run(Thread.java:745)
      

      This one appears exactly in the same scenarios and numbers as JBEAP-3780, JBEAP-3781 (and partially also JBEAP-3782) - see occurrences report
      (ejb-remote, ejb-ejbservlet and http-session scenarios with REPL cache and SYNC replication)

      Server link:
      http://jenkins.mw.lab.eng.bos.redhat.com/hudson/job/eap-7x-failover-http-session-shutdown-repl-sync/18/console-perf18/

      Occurrences report:
      http://download.eng.brq.redhat.com/scratch/mvinkler/reports/occurrences/ER6/timeoutExceptions.html

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  pferraro Paul Ferraro
                  Reporter:
                  mvinkler Michal Vinkler
                • Votes:
                  2 Vote for this issue
                  Watchers:
                  10 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: