Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-5498

ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover random failures

    Details

      Description

      It appears the cluster event notifier wants to send some events to a node that no longer exists, and the exception is propagated all the way to the HR client:

      12:04:03,088 ERROR (HotRodServerWorker-78-1:) [InvocationContextInterceptor] ISPN000136: Execution errorjava.lang.IllegalArgumentException: Target node ClientClusterFailoverEventsTest-NodeA-33510 is not a cluster member, members are [ClientClusterFailoverEventsTest-NodeB-14182]
      	at org.infinispan.distexec.DefaultExecutorService.submit(DefaultExecutorService.java:414)
      	at org.infinispan.distexec.DefaultExecutorService.submit(DefaultExecutorService.java:403)
      	at org.infinispan.distexec.DistributedExecutionCompletionService.submit(DistributedExecutionCompletionService.java:178)
      	at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl$UnicastEventContext.sendToTargets(BatchingClusterEventManagerImpl.java:97)
      	at org.infinispan.notifications.cachelistener.cluster.impl.BatchingClusterEventManagerImpl.sendEvents(BatchingClusterEventManagerImpl.java:50)
      	at org.infinispan.notifications.cachelistener.CacheNotifierImpl.notifyCacheEntryCreated(CacheNotifierImpl.java:292)
      	at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.notifyCommitEntry(ClusteringDependentLogic.java:138)
      	at org.infinispan.interceptors.locking.ClusteringDependentLogic$DistributionLogic.commitSingleEntry(ClusteringDependentLogic.java:493)
      	at org.infinispan.interceptors.locking.ClusteringDependentLogic$AbstractClusteringDependentLogic.commitEntry(ClusteringDependentLogic.java:108)
      	at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntry(EntryWrappingInterceptor.java:367)
      	at org.infinispan.interceptors.EntryWrappingInterceptor.commitEntryIfNeeded(EntryWrappingInterceptor.java:545)
      	at org.infinispan.interceptors.EntryWrappingInterceptor.commitContextEntries(EntryWrappingInterceptor.java:344)
      	at org.infinispan.interceptors.EntryWrappingInterceptor.invokeNextAndApplyChanges(EntryWrappingInterceptor.java:418)
      	at org.infinispan.interceptors.EntryWrappingInterceptor.setSkipRemoteGetsAndInvokeNextForDataCommand(EntryWrappingInterceptor.java:449)
      	at org.infinispan.interceptors.EntryWrappingInterceptor.visitPutKeyValueCommand(EntryWrappingInterceptor.java:195)
      	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
      	at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitNonTxDataWriteCommand(AbstractLockingInterceptor.java:88)
      	at org.infinispan.interceptors.locking.NonTransactionalLockingInterceptor.visitDataWriteCommand(NonTransactionalLockingInterceptor.java:40)
      	at org.infinispan.interceptors.locking.AbstractLockingInterceptor.visitPutKeyValueCommand(AbstractLockingInterceptor.java:55)
      	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
      	at org.infinispan.interceptors.base.CommandInterceptor.handleDefault(CommandInterceptor.java:111)
      	at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
      	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
      	at org.infinispan.statetransfer.StateTransferInterceptor.handleNonTxWriteCommand(StateTransferInterceptor.java:324)
      	at org.infinispan.statetransfer.StateTransferInterceptor.handleWriteCommand(StateTransferInterceptor.java:256)
      	at org.infinispan.statetransfer.StateTransferInterceptor.visitPutKeyValueCommand(StateTransferInterceptor.java:115)
      	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
      	at org.infinispan.interceptors.CacheMgmtInterceptor.updateStoreStatistics(CacheMgmtInterceptor.java:191)
      	at org.infinispan.interceptors.CacheMgmtInterceptor.visitPutKeyValueCommand(CacheMgmtInterceptor.java:177)
      	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
      	at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:97)
      	at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:102)
      	at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:71)
      	at org.infinispan.commands.AbstractVisitor.visitPutKeyValueCommand(AbstractVisitor.java:44)
      	at org.infinispan.commands.write.PutKeyValueCommand.acceptVisitor(PutKeyValueCommand.java:71)
      	at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:336)
      	at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1617)
      	at org.infinispan.cache.impl.CacheImpl.putInternal(CacheImpl.java:1097)
      	at org.infinispan.cache.impl.CacheImpl.put(CacheImpl.java:1089)
      	at org.infinispan.cache.impl.DecoratedCache.put(DecoratedCache.java:522)
      	at org.infinispan.server.hotrod.CacheDecodeContext.put(CacheDecodeContext.scala:216)
      	at org.infinispan.server.hotrod.HotRodDecoder.org$infinispan$server$hotrod$HotRodDecoder$$decodeValue(HotRodDecoder.scala:132)
      	at org.infinispan.server.hotrod.HotRodDecoder$$anonfun$decode$1.apply$mcV$sp(HotRodDecoder.scala:50)
      	at org.infinispan.server.hotrod.HotRodDecoder.wrapSecurity(HotRodDecoder.scala:208)
      	at org.infinispan.server.hotrod.HotRodDecoder.decode(HotRodDecoder.scala:45)
      org.infinispan.client.hotrod.exceptions.HotRodClientException:Request for message id[922] returned server error (status=0x85): java.lang.IllegalArgumentException: Target node ClientClusterFailoverEventsTest-NodeA-33510 is not a cluster member, members are [ClientClusterFailoverEventsTest-NodeB-14182]
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.checkForErrorsInResponseStatus(Codec20.java:336)
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.readPartialHeader(Codec20.java:126)
      	at org.infinispan.client.hotrod.impl.protocol.Codec20.readHeader(Codec20.java:112)
      	at org.infinispan.client.hotrod.impl.operations.HotRodOperation.readHeaderAndValidate(HotRodOperation.java:56)
      	at org.infinispan.client.hotrod.impl.operations.AbstractKeyValueOperation.sendPutOperation(AbstractKeyValueOperation.java:57)
      	at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:31)
      	at org.infinispan.client.hotrod.impl.operations.PutOperation.executeOperation(PutOperation.java:20)
      	at org.infinispan.client.hotrod.impl.operations.RetryOnFailureOperation.execute(RetryOnFailureOperation.java:52)
      	at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:247)
      	at org.infinispan.client.hotrod.impl.RemoteCacheSupport.put(RemoteCacheSupport.java:79)
      	at org.infinispan.client.hotrod.event.ClientClusterFailoverEventsTest.testEventReplayWithAndWithoutStateAfterFailover(ClientClusterFailoverEventsTest.java:59)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:497)
      	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
      	at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
      	at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
      	at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
      	at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
      	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
      	at org.testng.TestRunner.privateRun(TestRunner.java:767)
      	at org.testng.TestRunner.run(TestRunner.java:617)
      	at org.testng.SuiteRunner.runTest(SuiteRunner.java:348)
      	at org.testng.SuiteRunner.access$000(SuiteRunner.java:38)
      	at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:382)
      	at org.testng.internal.thread.ThreadUtil$2.call(ThreadUtil.java:64)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  Unassigned
                  Reporter:
                  dan.berindei Dan Berindei
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated: