Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-2975

Random failures in DistributedExecutorTest.testBasicTargetRemoteDistributedCallableWithHighFutureAndLowTaskTimeout and subclasses

    Details

      Description

      DistributedTaskBuilder supports setting a timeout for the distributed task. That timeout is used in two places: both a timeout for the FutureTask submitted to the local executor, and as a timeout for RPC invocations.

      The test testBasicTargetRemoteDistributedCallableWithHighFutureAndLowTaskTimeout expects a TimeoutException, which I think is correct. But the distributed task future doesn't throw a TimeoutException if the RPC timed out, instead it throws something like this:

      org.testng.TestException:  Expected exception java.util.concurrent.TimeoutException but got java.util.concurrent.ExecutionException: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to ReplSyncDistributedExecutorTest-NodeV-34575org.testng.TestException:Expected exception java.util.concurrent.TimeoutException but got java.util.concurrent.ExecutionException: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to ReplSyncDistributedExecutorTest-NodeV-34575
          at org.testng.internal.Invoker.handleInvocationResults(Invoker.java:1497)
          at org.testng.internal.Invoker.invokeMethod(Invoker.java:754)
          at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
          at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
          at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
          at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
          at org.testng.TestRunner.privateRun(TestRunner.java:767)
          at org.testng.TestRunner.run(TestRunner.java:617)
          at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
          at org.testng.SuiteRunner.access$000(SuiteRunner.java:37)
          at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:368)
          at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          at java.util.concurrent.FutureTask.run(FutureTask.java:138)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
          at java.lang.Thread.run(Thread.java:662)
      Caused by: java.util.concurrent.ExecutionException: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to ReplSyncDistributedExecutorTest-NodeV-34575
          at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
          at java.util.concurrent.FutureTask.get(FutureTask.java:91)
          at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.innerGet(DefaultExecutorService.java:948)
          at org.infinispan.distexec.DefaultExecutorService$DistributedTaskPart.get(DefaultExecutorService.java:927)
          at org.infinispan.distexec.DistributedExecutorTest.testBasicTargetRemoteDistributedCallableWithHighFutureAndLowTaskTimeout(DistributedExecutorTest.java:159)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at java.lang.reflect.Method.invoke(Method.java:597)
          at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
          at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)    ... 15 more
      Caused by: org.infinispan.CacheException: org.jgroups.TimeoutException: timeout sending message to ReplSyncDistributedExecutorTest-NodeV-34575
          at org.infinispan.util.Util.rewrapAsCacheException(Util.java:542)
          at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:186)
          at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:515)
          at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:169)
          at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:190)
          at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:247)
          at org.infinispan.remoting.rpc.RpcManagerImpl.access$000(RpcManagerImpl.java:79)
          at org.infinispan.remoting.rpc.RpcManagerImpl$1.call(RpcManagerImpl.java:281)    ... 5 more
      Caused by: org.jgroups.TimeoutException: timeout sending message to ReplSyncDistributedExecutorTest-NodeV-34575
          at org.jgroups.blocks.MessageDispatcher.sendMessage(MessageDispatcher.java:418)
          at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.processSingleCall(CommandAwareRpcDispatcher.java:301)
          at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.invokeRemoteCommand(CommandAwareRpcDispatcher.java:179)    ... 11 more
      

      I believe CommandAwareRpcDispatcher should wrap org.jgroups.TimeoutException in a org.infinispan.util.concurrent.TimeoutException, and DistributedTaskPart.innerGet should also treat ExecutionExceptions containing TimeoutExceptions specially.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  dan.berindei Dan Berindei
                  Reporter:
                  dan.berindei Dan Berindei
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: