Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-5454

XSite: RetryMechanismTest random failures

XMLWordPrintable

      ClusteredCacheBackupReceiver.awaitRemoteTask() doesn't respect the state push command's timeout, at least when it's smaller than the sync replication timeout in the target cache. When that happens, the state provider will resend the state, and there will be 2 state push commands executing at the same time.

      RetryMechanismTest changes the state push timeout to 2 seconds, but the sync replication timeout stays at 15 seconds. This causes failures in testRetryLocally and testFailRetryLocally, if it takes more than 2 seconds to suspect the killed node.

      10:02:13,007 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)]
      10:02:16,008 TRACE (asyncTransportThread-8,NodeN:) [RetryOnFailureXSiteCommand] Sending XSiteStatePushCommand{cacheName=___defaultcache, timeout=2000 (1 keys)} to [NYC (sync, timeout=2000)]
      10:02:16,040 TRACE (asyncTransportThread-4,NodeP:) [RpcManagerImpl] replication exception: 
      org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected
      10:02:16,040 TRACE (asyncTransportThread-0,NodeP:) [RpcManagerImpl] replication exception: 
      org.infinispan.remoting.transport.jgroups.SuspectException: Node NodeQ-56809 was suspected
      10:02:19,147 ERROR (testng-RetryMechanismTest:) [UnitTestTestNGListener] Test testFailRetryLocally(org.infinispan.xsite.statetransfer.failures.RetryMechanismTest) failed.
      java.lang.AssertionError: expected:<2> but was:<3>
      	at org.testng.AssertJUnit.fail(AssertJUnit.java:59)
      	at org.testng.AssertJUnit.failNotEquals(AssertJUnit.java:364)
      	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:80)
      	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:245)
      	at org.testng.AssertJUnit.assertEquals(AssertJUnit.java:252)
      	at org.infinispan.xsite.statetransfer.failures.RetryMechanismTest.testFailRetryLocally(RetryMechanismTest.java:227)
      

            pruivo@redhat.com Pedro Ruivo
            dberinde@redhat.com Dan Berindei (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: