Details
-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
DataGrid Sprint #69
Description
Fix for:
java.lang.AssertionError: Tombstone map is not empty! at org.infinispan.test.AbstractInfinispanTest.eventually(AbstractInfinispanTest.java:237) at org.infinispan.test.AbstractInfinispanTest.eventually(AbstractInfinispanTest.java:253) at org.infinispan.test.AbstractInfinispanTest.eventually(AbstractInfinispanTest.java:390) at org.infinispan.xsite.AbstractMultipleSitesTest.assertNoDataLeak(AbstractMultipleSitesTest.java:150) at org.infinispan.xsite.irac.Irac3SitesConflictTest.doTest(Irac3SitesConflictTest.java:292) at org.infinispan.xsite.irac.Irac3SitesConflictTest.testConditionalRemove(Irac3SitesConflictTest.java:98)
There is a problem with org.infinispan.xsite.AbstractMultipleSitesTest#assertNoDataLeak. First, it should wait for all the IRAC queues are empty before triggering the tombstone cleanup round.
Otherwise, the cleanup fails.
Trace logs:
# LON sends request to NYC to check if the tombstone can be removed 2021-11-30 18:39:33,957 TRACE (blocking-thread-Irac3SitesConflictTest-NodeA-p48-t1) [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Irac3SitesConflictTest-NodeA sending backup request 16 to SiteMaster(NYC-2): IracCleanupTombstoneCommand{cacheName=defaultcache, key=k0-testConditionalRemove, tombstone=null} # At the same time, NYC finished replicate the key but it hasn't removed the key from the map yet 2021-11-30 18:39:33,957 TRACE (jgroups-11,bridge-org.infinispan.xsite.irac.Irac3SitesConflictTest[configMode=OPTIMISTIC_TX_RC],_Irac3SitesConflictTest-NodeD:NYC-2) [org.infinispan.xsite.irac.DefaultIracManager] Replication completed for key 'k0-testConditionalRemove'. /* ... */ # The LON request is processed and it replies with true (== I have the key in the map) 2021-11-30 18:39:33,957 TRACE (jgroups-5,bridge-org.infinispan.xsite.irac.Irac3SitesConflictTest[configMode=OPTIMISTIC_TX_RC],_Irac3SitesConflictTest-NodeD:NYC-2) [org.infinispan.remoting.transport.jgroups.JGroupsTransport] Irac3SitesConflictTest-NodeD sending response for request 16 to Irac3SitesConflictTest-NodeA:LON-1: SuccessfulResponse(true) # Finally, the key is removed from the queue (too late, the tombstone will be cleanup in the next round, but the test fails before it happens.) 2021-11-30 18:39:33,957 TRACE (jgroups-11,bridge-org.infinispan.xsite.irac.Irac3SitesConflictTest[configMode=OPTIMISTIC_TX_RC],_Irac3SitesConflictTest-NodeD:NYC-2) [org.infinispan.xsite.irac.DefaultIracManager] Removing key 'k0-testConditionalRemove'. /* ... */, removed=true