18:06:41,318 DEBUG (testng-VersionedReplStateTransferTest:) [VersionedReplStateTransferTest] Creating cache managers 18:06:41,398 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,398 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,399 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,399 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,399 DEBUG (testng-VersionedReplStateTransferTest:) [ModuleProperties] No module lifecycle SPI classes available 18:06:41,406 DEBUG (testng-VersionedReplStateTransferTest:) [ModuleProperties] No module command extensions to load 18:06:41,481 DEBUG (testng-VersionedReplStateTransferTest:) [AbstractJBossMarshaller] Using JBoss Marshalling 18:06:41,485 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute cacheManagerStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:41,543 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterMembers [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute nodeAddress [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute physicalAddresses [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute createdCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute version [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheNames [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute runningCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterName [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute name [r=true,w=false,is=false,type=java.lang.String] 18:06:41,548 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterSize [r=true,w=false,is=false,type=int] 18:06:41,549 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:41,549 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:41,549 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@12098e0a under infinispan75:type=CacheManager,name="DefaultCacheManager",component=CacheManager 18:06:41,549 DEBUG (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Started cache manager ISPN on NodeA 18:06:41,549 TRACE (testng-VersionedReplStateTransferTest:) [TestCacheManagerFactory] Adding DCM (NodeA) for method: 'org.infinispan.container.versioning.VersionedReplStateTransferTest.createCacheManagers(VersionedReplStateTransferTest.java:59)' 18:06:41,552 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,552 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,552 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,552 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,553 DEBUG (testng-VersionedReplStateTransferTest:) [ModuleProperties] No module lifecycle SPI classes available 18:06:41,559 DEBUG (testng-VersionedReplStateTransferTest:) [ModuleProperties] No module command extensions to load 18:06:41,584 DEBUG (testng-VersionedReplStateTransferTest:) [AbstractJBossMarshaller] Using JBoss Marshalling 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterName [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute createdCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute cacheManagerStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute runningCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterSize [r=true,w=false,is=false,type=int] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute nodeAddress [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterMembers [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheNames [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute name [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute version [r=true,w=false,is=false,type=java.lang.String] 18:06:41,585 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute physicalAddresses [r=true,w=false,is=false,type=java.lang.String] 18:06:41,600 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:41,607 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:41,607 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@7c3b6b35 under infinispan77:type=CacheManager,name="DefaultCacheManager",component=CacheManager 18:06:41,607 DEBUG (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Started cache manager ISPN on NodeB 18:06:41,607 TRACE (testng-VersionedReplStateTransferTest:) [TestCacheManagerFactory] Adding DCM (NodeB) for method: 'org.infinispan.container.versioning.VersionedReplStateTransferTest.createCacheManagers(VersionedReplStateTransferTest.java:59)' 18:06:41,609 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000078: Starting JGroups Channel 18:06:41,733 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [Configurator] set property TCP.diagnostics_addr to default value /224.0.75.75 18:06:41,803 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [FRAG2] received CONFIG event: {bind_addr=/127.0.0.1} 18:06:41,819 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] address=NodeA-3607, cluster=ISPN, physical address=127.0.0.1:9500 18:06:41,833 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Discoveries for DiscoveryKey{clusterName='ISPN', testName='org.infinispan.container.versioning.VersionedReplStateTransferTest'} are : {} 18:06:41,833 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Add discovery for NodeA-3607 to cache. The cache now contains: {NodeA-3607=TEST_PING@NodeA-3607} 18:06:41,833 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Skipping sending discovery to self 18:06:41,833 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] discovery took 0 ms: responses: 0 total (0 servers (0 coord), 0 clients) 18:06:41,838 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [NAKACK2] [NodeA-3607 setDigest()] existing digest: [] new digest: NodeA-3607: [0 (0)] resulting digest: NodeA-3607: [0 (0)] 18:06:41,848 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] NodeA-3607: installing view [NodeA-3607|0] [NodeA-3607] 18:06:41,855 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [STABLE] resuming message garbage collection 18:06:41,856 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [STABLE] [ergonomics] setting max_bytes to 400KB (1 members) 18:06:41,856 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] New view accepted: [NodeA-3607|0] [NodeA-3607] 18:06:41,856 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeA-3607|0] [NodeA-3607] 18:06:41,856 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [STABLE] resuming message garbage collection 18:06:41,856 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] created group (first member). My view is [NodeA-3607|0], impl is org.jgroups.protocols.pbcast.CoordGmsImpl 18:06:41,856 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000079: Cache local address is NodeA-3607, physical addresses are [127.0.0.1:9500] 18:06:41,856 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] Waiting on view being accepted 18:06:41,856 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,856 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:41,856 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [DefaultCacheManager] About to wire and start cache ___defaultcache 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for JBoss AS 7 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [java:jboss/TransactionManager (JBoss AS 7)] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for JBoss AS 4 ~ 6, JRun4 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [java:/TransactionManager (JBoss AS 4 ~ 6, JRun4)] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for Resin 3.x 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [java:comp/TransactionManager (Resin 3.x)] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for Sun Glassfish 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [java:appserver/TransactionManager (Sun Glassfish)] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for Borland, Sun 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [java:pm/TransactionManager (Borland, Sun)] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for BEA WebLogic 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [javax.transaction.TransactionManager (BEA WebLogic)] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying to lookup TransactionManager for Resin, Orion, JOnAS (JOTM) 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Failed to perform a lookup for [java:comp/UserTransaction (Resin, Orion, JOnAS (JOTM))] 18:06:41,858 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying WebSphere 5.1: com.ibm.ws.Transaction.TransactionManagerFactory 18:06:41,859 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying WebSphere 5.0: com.ibm.ejs.jts.jta.TransactionManagerFactory 18:06:41,859 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Trying WebSphere 4: com.ibm.ejs.jts.jta.JTSXA 18:06:41,860 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GenericTransactionManagerLookup] Couldn't find any WebSphere TransactionManager factory class, neither for WebSphere version 5.1 nor 5.0 nor 4 18:06:41,860 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JBossStandaloneJTAManagerLookup] ISPN000107: Retrieving transaction manager Transaction: unknown 18:06:41,870 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractJBossMarshaller] Using JBoss Marshalling 18:06:41,871 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [InterceptorChain] Interceptor chain size: 8 18:06:41,871 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [InterceptorChain] Interceptor chain is: >> org.infinispan.interceptors.InvocationContextInterceptor >> org.infinispan.statetransfer.StateTransferInterceptor >> org.infinispan.interceptors.TxInterceptor >> org.infinispan.interceptors.NotificationInterceptor >> org.infinispan.interceptors.locking.OptimisticLockingInterceptor >> org.infinispan.interceptors.VersionedEntryWrappingInterceptor >> org.infinispan.interceptors.VersionedReplicationInterceptor >> org.infinispan.interceptors.CallInterceptor 18:06:41,871 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionFactory] Setting factory enum to NODLD_NORECOVERY_NOXA 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheName [r=true,w=false,is=false,type=java.lang.String] 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute configurationAsXmlString [r=true,w=false,is=false,type=java.lang.String] 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void stop 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void start 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheName [r=true,w=false,is=false,type=java.lang.String] 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:41,872 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute configurationAsXmlString [r=true,w=false,is=false,type=java.lang.String] 18:06:41,876 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void stop 18:06:41,907 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void start 18:06:41,907 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:41,907 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute rollbacks [r=true,w=false,is=false,type=long] 18:06:41,907 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute commits [r=true,w=false,is=false,type=long] 18:06:41,907 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute prepares [r=true,w=false,is=false,type=long] 18:06:41,907 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void resetStatistics 18:06:41,908 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:41,914 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:41,932 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forget 18:06:41,938 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forget 18:06:41,938 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:41,938 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String showInDoubtTransactions 18:06:41,938 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:41,938 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute joinComplete [r=true,w=false,is=true,type=boolean] 18:06:41,963 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute stateTransferInProgress [r=true,w=false,is=true,type=boolean] 18:06:41,963 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute numberOfLocksHeld [r=true,w=false,is=false,type=int] 18:06:41,963 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute concurrencyLevel [r=true,w=false,is=false,type=int] 18:06:41,974 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute numberOfLocksAvailable [r=true,w=false,is=false,type=int] 18:06:42,003 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void recordKnownGlobalKeyset 18:06:42,003 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute replicationFailures [r=true,w=false,is=false,type=long] 18:06:42,003 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute replicationCount [r=true,w=false,is=false,type=long] 18:06:42,003 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute committedViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:42,003 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute averageReplicationTime [r=true,w=false,is=false,type=long] 18:06:42,003 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute successRatio [r=true,w=false,is=false,type=java.lang.String] 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute pendingViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute successRatioFloatingPoint [r=true,w=false,is=false,type=double] 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void resetStatistics 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@7b8cf3b1 under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Cache 18:06:42,004 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Object name infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Cache already registered 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@151deaa3 under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Transactions 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@5f177e29 under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RecoveryAdmin 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@7f5be684 under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=StateTransferManager 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@6a7b256e under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=LockManager 18:06:42,004 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@3ad68f5f under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RollingUpgradeManager 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@5c5ec426 under infinispan75:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RpcManager 18:06:42,005 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [CacheJmxRegistration] ISPN000031: MBeans were successfully registered to the platform MBean server. 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Starting StateTransferManager of cache ___defaultcache on node NodeA-3607 18:06:42,005 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTopologyManagerImpl] Node NodeA-3607 joining cache ___defaultcache 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ClusterTopologyManagerImpl] Received new cluster view: 0 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ClusterTopologyManagerImpl] Updating cluster members for all the caches. New list is [NodeA-3607] 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache ___defaultcache) [DefaultRebalancePolicy] Initializing rebalance policy for cache ___defaultcache 18:06:42,005 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=0, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=null} 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=0, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=null} 18:06:42,005 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607]} 18:06:42,006 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [StateConsumerImpl] Adding state transfer for segments: [0] 18:06:42,006 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:42,006 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Changing minimum view ID from -1 to 0 18:06:42,027 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Waiting for initial state transfer to finish 18:06:42,039 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [CacheImpl] Started cache ___defaultcache on NodeA-3607 18:06:42,040 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000078: Starting JGroups Channel 18:06:42,113 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [Configurator] set property TCP.diagnostics_addr to default value /224.0.75.75 18:06:42,170 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [FRAG2] received CONFIG event: {bind_addr=/127.0.0.1} 18:06:42,203 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] address=NodeB-43205, cluster=ISPN, physical address=127.0.0.1:9501 18:06:42,210 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Discoveries for DiscoveryKey{clusterName='ISPN', testName='org.infinispan.container.versioning.VersionedReplStateTransferTest'} are : {NodeA-3607=TEST_PING@NodeA-3607} 18:06:42,228 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Add discovery for NodeB-43205 to cache. The cache now contains: {NodeB-43205=TEST_PING@NodeB-43205, NodeA-3607=TEST_PING@NodeA-3607} 18:06:42,228 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Skipping sending discovery to self 18:06:42,228 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Map NodeA-3607 with physical address 127.0.0.1:9500 in TEST_PING@NodeB-43205 18:06:42,228 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Map NodeB-43205 with physical address 127.0.0.1:9501 in TEST_PING@NodeA-3607 18:06:42,228 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Returning ping rsp: NodeA-3607, view_id=[NodeA-3607|0] ([NodeA-3607|0] [NodeA-3607]), is_server=true, is_coord=true, logical_name=NodeA-3607, physical_addrs=127.0.0.1:9500 18:06:42,228 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] discovery took 18 ms: responses: 1 total (1 servers (1 coord), 0 clients) 18:06:42,228 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] election results: {NodeA-3607=1} 18:06:42,228 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] sending JOIN(NodeB-43205) to NodeA-3607 18:06:42,294 DEBUG (ViewHandler,ISPN,NodeA-3607:) [STABLE] suspending message garbage collection 18:06:42,300 DEBUG (ViewHandler,ISPN,NodeA-3607:) [STABLE] resume task started, max_suspend_time=33000 18:06:42,326 DEBUG (Incoming-1,ISPN,NodeA-3607:) [NAKACK2] [NodeA-3607 setDigest()] existing digest: NodeA-3607: [1 (1)] new digest: NodeA-3607: [0 (0)], NodeB-43205: [0 (0)] resulting digest: NodeA-3607: [1 (1)], NodeB-43205: [0 (0)] 18:06:42,332 DEBUG (Incoming-1,ISPN,NodeA-3607:) [GMS] NodeA-3607: installing view [NodeA-3607|1] [NodeA-3607, NodeB-43205] 18:06:42,333 DEBUG (Incoming-1,ISPN,NodeA-3607:) [STABLE] [ergonomics] setting max_bytes to 800KB (2 members) 18:06:42,364 DEBUG (Incoming-1,ISPN,NodeA-3607:) [JGroupsTransport] New view accepted: [NodeA-3607|1] [NodeA-3607, NodeB-43205] 18:06:42,364 INFO (Incoming-1,ISPN,NodeA-3607:) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeA-3607|1] [NodeA-3607, NodeB-43205] 18:06:42,365 TRACE (Incoming-1,ISPN,NodeA-3607:) [ClusterIdGenerator] Calculated rank based on view EventImpl{type=VIEW_CHANGED, newMembers=[NodeA-3607, NodeB-43205], oldMembers=[NodeA-3607], localAddress=NodeA-3607, viewId=1, subgroupsMerged=null, mergeView=false} and result was 281479271677952 18:06:42,369 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [NAKACK2] [NodeB-43205 setDigest()] existing digest: [] new digest: NodeA-3607: [0 (0)], NodeB-43205: [0 (0)] resulting digest: NodeA-3607: [0 (0)], NodeB-43205: [0 (0)] 18:06:42,369 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] NodeB-43205: installing view [NodeA-3607|1] [NodeA-3607, NodeB-43205] 18:06:42,370 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [STABLE] [ergonomics] setting max_bytes to 800KB (2 members) 18:06:42,370 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] New view accepted: [NodeA-3607|1] [NodeA-3607, NodeB-43205] 18:06:42,393 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeA-3607|1] [NodeA-3607, NodeB-43205] 18:06:42,393 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000079: Cache local address is NodeB-43205, physical addresses are [127.0.0.1:9501] 18:06:42,393 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] Waiting on view being accepted 18:06:42,393 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:42,394 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:42,394 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [DefaultCacheManager] About to wire and start cache ___defaultcache 18:06:42,397 DEBUG (ViewHandler,ISPN,NodeA-3607:) [STABLE] resuming message garbage collection 18:06:42,368 TRACE (notification-thread-0,NodeA:) [ClusterTopologyManagerImpl] Received new cluster view: 1 18:06:42,404 TRACE (notification-thread-0,NodeA:) [ClusterTopologyManagerImpl] Updating cluster members for all the caches. New list is [NodeA-3607, NodeB-43205] 18:06:42,406 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractJBossMarshaller] Using JBoss Marshalling 18:06:42,411 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [InterceptorChain] Interceptor chain size: 8 18:06:42,438 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [InterceptorChain] Interceptor chain is: >> org.infinispan.interceptors.InvocationContextInterceptor >> org.infinispan.statetransfer.StateTransferInterceptor >> org.infinispan.interceptors.TxInterceptor >> org.infinispan.interceptors.NotificationInterceptor >> org.infinispan.interceptors.locking.OptimisticLockingInterceptor >> org.infinispan.interceptors.VersionedEntryWrappingInterceptor >> org.infinispan.interceptors.VersionedReplicationInterceptor >> org.infinispan.interceptors.CallInterceptor 18:06:42,438 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionFactory] Setting factory enum to NODLD_NORECOVERY_NOXA 18:06:42,440 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:42,440 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:42,441 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forget 18:06:42,442 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String showInDoubtTransactions 18:06:42,442 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:42,442 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:42,444 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forget 18:06:42,461 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:42,463 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute prepares [r=true,w=false,is=false,type=long] 18:06:42,469 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute rollbacks [r=true,w=false,is=false,type=long] 18:06:42,469 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute commits [r=true,w=false,is=false,type=long] 18:06:42,469 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void resetStatistics 18:06:42,469 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute joinComplete [r=true,w=false,is=true,type=boolean] 18:06:42,473 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute stateTransferInProgress [r=true,w=false,is=true,type=boolean] 18:06:42,485 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void recordKnownGlobalKeyset 18:06:42,485 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute concurrencyLevel [r=true,w=false,is=false,type=int] 18:06:42,492 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute numberOfLocksAvailable [r=true,w=false,is=false,type=int] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute numberOfLocksHeld [r=true,w=false,is=false,type=int] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute committedViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute replicationCount [r=true,w=false,is=false,type=long] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute successRatioFloatingPoint [r=true,w=false,is=false,type=double] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute replicationFailures [r=true,w=false,is=false,type=long] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute successRatio [r=true,w=false,is=false,type=java.lang.String] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute averageReplicationTime [r=true,w=false,is=false,type=long] 18:06:42,496 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute pendingViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void resetStatistics 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheName [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute configurationAsXmlString [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void start 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void stop 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheName [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute configurationAsXmlString [r=true,w=false,is=false,type=java.lang.String] 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void start 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void stop 18:06:42,497 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@22462413 under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RecoveryAdmin 18:06:42,507 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@24ad08e0 under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Transactions 18:06:42,507 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@5c21bcef under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=StateTransferManager 18:06:42,507 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@2afb1e2a under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RollingUpgradeManager 18:06:42,507 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@4c796d71 under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=LockManager 18:06:42,508 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@723da84d under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RpcManager 18:06:42,508 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@46d7bef0 under infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Cache 18:06:42,508 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Object name infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Cache already registered 18:06:42,516 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [CacheJmxRegistration] ISPN000031: MBeans were successfully registered to the platform MBean server. 18:06:42,518 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Starting StateTransferManager of cache ___defaultcache on node NodeB-43205 18:06:42,518 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTopologyManagerImpl] Node NodeB-43205 joining cache ___defaultcache 18:06:42,518 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=CacheTopologyControlCommand{cache=___defaultcache, type=JOIN, sender=NodeB-43205, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@2bf0f9d6, hashFunction=org.infinispan.commons.hash.MurmurHash3@59a26723, numSegments=12, numOwners=2, timeout=240000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=1}, mode=SYNCHRONOUS, timeout=240000 18:06:42,518 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=JOIN, sender=NodeB-43205, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@2bf0f9d6, hashFunction=org.infinispan.commons.hash.MurmurHash3@59a26723, numSegments=12, numOwners=2, timeout=240000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=1} to single recipient NodeA-3607 with response mode GET_ALL 18:06:42,519 TRACE (OOB-1,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 207 18:06:42,521 TRACE (OOB-1,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=JOIN, sender=NodeB-43205, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@7c9b2f60, hashFunction=org.infinispan.commons.hash.MurmurHash3@30438789, numSegments=12, numOwners=2, timeout=240000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=1} [sender=NodeB-43205] 18:06:42,521 TRACE (OOB-1,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[NodeB-43205], topology=CacheTopology{id=0, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=null} 18:06:42,521 TRACE (OOB-1,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Triggering rebalance for cache ___defaultcache 18:06:42,527 TRACE (OOB-1,ISPN,NodeA-3607:) [RequestCorrelator] sending rsp for 207 to NodeB-43205 18:06:42,542 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Response: SuccessfulResponse{responseValue=CacheTopology{id=0, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=null}} 18:06:42,542 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=0, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=null} 18:06:42,527 TRACE (asyncTransportThread-0,NodeA:) [ClusterTopologyManagerImpl] Rebalancing consistent hash for cache ___defaultcache, members are [NodeA-3607, NodeB-43205] 18:06:42,542 TRACE (asyncTransportThread-0,NodeA:) [ClusterTopologyManagerImpl] Updating cache ___defaultcache topology for rebalance: CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}} 18:06:42,542 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=0, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=null} 18:06:42,602 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607]} 18:06:42,546 TRACE (asyncTransportThread-0,NodeA:) [RebalanceConfirmationCollector] Initialized rebalance confirmation collector 1@___defaultcache, initial list is [NodeA-3607, NodeB-43205] 18:06:42,615 TRACE (asyncTransportThread-0,NodeA:) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[NodeB-43205], topology=CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}} 18:06:42,616 DEBUG (asyncTransportThread-0,NodeA:) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}} 18:06:42,602 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:42,652 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Changing minimum view ID from -1 to 0 18:06:42,628 TRACE (asyncTransportThread-1,NodeA:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, throwable=null, viewId=1}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:42,662 DEBUG (asyncTransportThread-2,NodeA:___defaultcache) [LocalTopologyManagerImpl] Starting local rebalance for cache ___defaultcache, topology = CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}} 18:06:42,664 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeA-3607]}} 18:06:42,664 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeB-43205, NodeA-3607]} 18:06:42,664 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:42,666 DEBUG (asyncTransportThread-2,NodeA:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:42,654 TRACE (asyncTransportThread-1,NodeA:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, throwable=null, viewId=1} to addresses null with response mode GET_ALL 18:06:42,653 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Waiting for initial state transfer to finish 18:06:42,669 TRACE (asyncTransportThread-1,NodeA:) [CommandAwareRpcDispatcher] real_dests=[NodeB-43205] 18:06:42,667 DEBUG (asyncTransportThread-3,NodeA:___defaultcache) [ClusterTopologyManagerImpl] Finished local rebalance for cache ___defaultcache on node NodeA-3607, topology id = 1 18:06:42,666 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [TransactionTable] Changing minimum view ID from 0 to 1 18:06:42,684 TRACE (asyncTransportThread-3,NodeA:___defaultcache) [RebalanceConfirmationCollector] Rebalance confirmation collector 1@___defaultcache received confirmation for NodeA-3607, remaining list is [NodeB-43205] 18:06:42,697 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 214 18:06:42,702 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, throwable=null, viewId=1} [sender=NodeA-3607] 18:06:42,703 TRACE (OOB-1,ISPN,NodeB-43205:) [ClusterTopologyManagerImpl] Received new cluster view: 1 18:06:42,704 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [LocalTopologyManagerImpl] Starting local rebalance for cache ___defaultcache, topology = CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}} 18:06:42,704 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=1, currentCH=ReplicatedConsistentHash{members=[NodeA-3607]}, pendingCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeA-3607]}} 18:06:42,704 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeB-43205, NodeA-3607]} 18:06:42,704 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [] 18:06:42,705 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Adding state transfer for segments: [0] 18:06:42,705 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [InboundTransferTask] Requesting transactions for segments [0] 18:06:42,705 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=GET_TRANSACTIONS, topologyId=1, segments=[0]}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:42,705 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=GET_TRANSACTIONS, topologyId=1, segments=[0]} to addresses [NodeA-3607] with response mode GET_ALL 18:06:42,707 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 215 18:06:42,707 TRACE (OOB-2,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute command: StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=GET_TRANSACTIONS, topologyId=1, segments=[0]} [sender=NodeB-43205] 18:06:42,738 TRACE (OOB-2,ISPN,NodeA-3607:) [InboundInvocationHandlerImpl] Calling perform() on StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=GET_TRANSACTIONS, topologyId=1, segments=[0]} 18:06:42,738 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [StateProviderImpl] Received request for transactions from node NodeB-43205 for segments [0] with topology id 1 18:06:42,738 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [StateProviderImpl] Found 0 transaction(s) to transfer 18:06:42,738 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] sending rsp for 215 to NodeB-43205 18:06:42,746 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Responses: [sender=NodeA-3607, retval=SuccessfulResponse{responseValue=[]} , received=true, suspected=false] 18:06:42,746 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Applying 0 transactions transferred from NodeA-3607 18:06:42,746 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=START_STATE_TRANSFER, topologyId=1, segments=[0]}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:42,746 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=START_STATE_TRANSFER, topologyId=1, segments=[0]} to addresses [NodeA-3607] with response mode GET_ALL 18:06:42,747 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 218 18:06:42,755 TRACE (OOB-2,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute command: StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=START_STATE_TRANSFER, topologyId=1, segments=[0]} [sender=NodeB-43205] 18:06:42,755 TRACE (OOB-2,ISPN,NodeA-3607:) [InboundInvocationHandlerImpl] Calling perform() on StateRequestCommand{cache=___defaultcache, origin=NodeB-43205, type=START_STATE_TRANSFER, topologyId=1, segments=[0]} 18:06:42,827 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [StateProviderImpl] Starting outbound transfer of segments [0] to node NodeB-43205 with topology id 1 18:06:42,858 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [StateProviderImpl] Adding outbound transfer of segments [0] to NodeB-43205 18:06:42,858 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] sending rsp for 218 to NodeB-43205 18:06:42,860 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Responses: [sender=NodeA-3607, retval=SuccessfulResponse{responseValue=true} , received=true, suspected=false] 18:06:42,860 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:42,899 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Changing minimum view ID from 0 to 1 18:06:42,899 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 214 to NodeA-3607 18:06:42,858 TRACE (asyncTransportThread-2,NodeA:) [OutboundTransferTask] No cache store or the cache store is shared, no need to send any stored cache entries for segments: [0] 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [OutboundTransferTask] Sending 0 cache entries from segments [] to node NodeB-43205 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [RpcManagerImpl] NodeA-3607 invoking in future call StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]} to recipient list [NodeB-43205] 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [OutboundTransferTask] Outbound transfer of segments [0] to NodeB-43205 is complete 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [StateProviderImpl] Removing completed outbound transfer of segments [0] to NodeB-43205 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [RpcManagerImpl] NodeA-3607 broadcasting call StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]} to recipient list [NodeB-43205] 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [JGroupsTransport] dests=[NodeB-43205], command=StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]}, mode=SYNCHRONOUS, timeout=240000 18:06:42,901 TRACE (asyncTransportThread-2,NodeA:) [CommandAwareRpcDispatcher] Replication task sending StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]} to single recipient NodeB-43205 with response mode GET_ALL 18:06:42,906 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 224 18:06:42,906 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]} [sender=NodeA-3607] 18:06:42,906 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]} 18:06:42,906 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Applying new state for segment 0 from NodeA-3607: received 0 cache entries 18:06:42,906 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Received keys: [] 18:06:42,901 TRACE (asyncTransportThread-1,NodeA:) [CommandAwareRpcDispatcher] Responses: [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:06:42,907 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeB-43205, joinInfo=null, topologyId=1, currentCH=null, pendingCH=null, throwable=null, viewId=1}, mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=0 18:06:42,907 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeB-43205, joinInfo=null, topologyId=1, currentCH=null, pendingCH=null, throwable=null, viewId=1} to single recipient NodeA-3607 with response mode GET_NONE 18:06:42,907 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 225 18:06:42,908 TRACE (OOB-2,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeB-43205, joinInfo=null, topologyId=1, currentCH=null, pendingCH=null, throwable=null, viewId=1} [sender=NodeB-43205] 18:06:42,908 DEBUG (OOB-2,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Finished local rebalance for cache ___defaultcache on node NodeB-43205, topology id = 1 18:06:42,911 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [RebalanceConfirmationCollector] Rebalance confirmation collector 1@___defaultcache received confirmation for NodeB-43205, remaining list is [] 18:06:42,907 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:42,964 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] After applying the received state the data container has 0 keys 18:06:42,964 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Segments not received yet: {} 18:06:42,964 DEBUG (OOB-2,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Finished cluster-wide rebalance for cache ___defaultcache, topology id = 1 18:06:42,969 DEBUG (OOB-2,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:42,970 DEBUG (asyncTransportThread-1,NodeA:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:43,008 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:43,008 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]} 18:06:43,008 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:43,008 DEBUG (asyncTransportThread-1,NodeA:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:43,008 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [TransactionTable] Changing minimum view ID from 1 to 2 18:06:42,969 TRACE (asyncTransportThread-3,NodeA:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null, throwable=null, viewId=1}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:42,964 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 224 to NodeA-3607 18:06:43,011 TRACE (asyncTransportThread-2,NodeA:) [CommandAwareRpcDispatcher] Response: null 18:06:43,011 TRACE (asyncTransportThread-2,NodeA:) [RpcManagerImpl] Response(s) to StateResponseCommand{cache=___defaultcache, origin=NodeA-3607, topologyId=1, stateChunks=[StateChunk{segmentId=0, cacheEntries=[], isLastChunk=true}]} is {NodeB-43205=null} 18:06:43,009 TRACE (asyncTransportThread-3,NodeA:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null, throwable=null, viewId=1} to addresses null with response mode GET_ALL 18:06:43,012 TRACE (asyncTransportThread-3,NodeA:) [CommandAwareRpcDispatcher] real_dests=[NodeB-43205] 18:06:43,013 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 230 18:06:43,023 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null, throwable=null, viewId=1} [sender=NodeA-3607] 18:06:43,024 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:43,024 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:43,024 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]} 18:06:43,024 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:43,024 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:43,024 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Changing minimum view ID from 1 to 2 18:06:43,025 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [CacheImpl] Started cache ___defaultcache on NodeB-43205 18:06:43,025 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 230 to NodeA-3607 18:06:43,026 TRACE (asyncTransportThread-3,NodeA:) [CommandAwareRpcDispatcher] Responses: [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:06:43,026 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[], topology=CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:43,027 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Not triggering rebalance for cache ___defaultcache, no joiners and the current consistent hash is already balanced 18:06:43,125 DEBUG (testng-VersionedReplStateTransferTest:) [VersionedReplStateTransferTest] Cache managers created, ready to start the test 18:06:43,125 INFO (testng-VersionedReplStateTransferTest:) [UnitTestTestNGListener] Starting test testStateTransfer(org.infinispan.container.versioning.VersionedReplStateTransferTest) 18:06:43,126 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Implicit transaction started! 18:06:43,126 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Marked tx as implicit. 18:06:43,126 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@2eb54532] 18:06:43,126 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}, originLocal=true 18:06:43,126 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Created a new local transaction: LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, viewId=2} org.infinispan.transaction.synchronization.SyncLocalTransaction@ab 18:06:43,126 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:43,127 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Retrieved from container null 18:06:43,127 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Creating new entry. 18:06:43,127 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}. 18:06:43,127 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTransaction] Adding modification PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}. Mod list is null 18:06:43,127 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Committing transaction as it was implicit: TransactionImple < ac, BasicAction: 0:ffff7f000001:9ca1:5079837d:b8 status: ActionStatus.RUNNING > 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:) [SynchronizationAdapter] beforeCompletion called for LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, viewId=2} org.infinispan.transaction.synchronization.SyncLocalTransaction@ab 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:) [TransactionCoordinator] Sending prepare command through the chain: VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'} 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@7ed1ed92] 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'}, originLocal=true 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [OptimisticLockingInterceptor] Not using lock reordering as we have a single key. 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [OptimisticLockingInterceptor] Locking key hello, no need to check for pending locks. 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [LockManagerImpl] Attempting to lock hello with acquisition timeout of 10000 millis 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [LockManagerImpl] Successfully acquired lock hello! 18:06:43,128 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractCacheTransaction] Registering locked key: hello 18:06:43,129 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Suppressing invocation of method handlePrepareCommand. 18:06:43,129 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] NodeA-3607 broadcasting call VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'} to recipient list null 18:06:43,129 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=null, command=VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'}, mode=SYNCHRONOUS, timeout=15000 18:06:43,129 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'} to single recipient NodeB-43205 with response mode GET_ALL 18:06:43,130 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 237 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'} [sender=NodeA-3607] 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:remote, cacheName='___defaultcache'} 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:) [TransactionTable] Created and registered remote transaction RemoteTransaction{modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], lookedUpEntries={}, lockedKeys= null, backupKeyLocks null, missingLookedUpEntries false, tx=GlobalTransaction::171:remote} 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:) [PrepareCommand] Invoking remotely originated prepare: VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:remote, cacheName='___defaultcache'} with invocation context: org.infinispan.context.impl.RemoteTxInvocationContext@96b8feb3 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [InvocationContextInterceptor] Invoked with command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:remote, cacheName='___defaultcache'} and InvocationContext [org.infinispan.context.impl.RemoteTxInvocationContext@96b8feb3] 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:remote, cacheName='___defaultcache'}, originLocal=false 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [OptimisticLockingInterceptor] Not using lock reordering as we have a single key. 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [EntryFactoryImpl] Retrieved from container null 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [EntryFactoryImpl] Creating new entry. 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [RemoteTransaction] Adding key hello to tx GlobalTransaction::171:remote 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CallInterceptor] Executing command: PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}. 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CallInterceptor] Suppressing invocation of method handlePrepareCommand. 18:06:43,131 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] CommandTopologyId=2, localTopologyId=2 18:06:43,132 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 237 to NodeA-3607 18:06:43,133 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:43,133 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] Response(s) to VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::171:local, cacheName='___defaultcache'} is {NodeB-43205=null} 18:06:43,133 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTransaction] Adding remote locks on [NodeA-3607, NodeB-43205]. Remote locks are null 18:06:43,133 TRACE (testng-VersionedReplStateTransferTest:) [SynchronizationAdapter] afterCompletion(3) called for LocalTransaction{remoteLockedNodes=[NodeA-3607, NodeB-43205], isMarkedForRollback=false, lockedKeys=[hello], backupKeyLocks=null, viewId=2} org.infinispan.transaction.synchronization.SyncLocalTransaction@ab. 18:06:43,133 TRACE (testng-VersionedReplStateTransferTest:) [TransactionCoordinator] Committing transaction GlobalTransaction::171:local 18:06:43,151 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@74b3dc5c] 18:06:43,153 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=-1}, originLocal=true 18:06:43,153 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] NodeA-3607 broadcasting call CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=-1} to recipient list null 18:06:43,193 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=null, command=CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=2}, mode=SYNCHRONOUS, timeout=15000 18:06:43,193 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=2} to single recipient NodeB-43205 with response mode GET_ALL 18:06:43,209 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 242 18:06:43,265 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=2} [sender=NodeA-3607] 18:06:43,302 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on CommitCommand {gtx=GlobalTransaction::171:remote, cacheName='___defaultcache', topologyId=2} 18:06:43,302 TRACE (OOB-1,ISPN,NodeB-43205:) [AbstractTransactionBoundaryCommand] About to execute tx command CommitCommand {gtx=GlobalTransaction::171:remote, cacheName='___defaultcache', topologyId=2} 18:06:43,302 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [InvocationContextInterceptor] Invoked with command CommitCommand {gtx=GlobalTransaction::171:remote, cacheName='___defaultcache', topologyId=2} and InvocationContext [org.infinispan.context.impl.RemoteTxInvocationContext@96b8feb3] 18:06:43,302 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command CommitCommand {gtx=GlobalTransaction::171:remote, cacheName='___defaultcache', topologyId=2}, originLocal=false 18:06:43,339 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CallInterceptor] Suppressing invocation of method handleCommitCommand. 18:06:43,339 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [ReadCommittedEntry] Updating entry (key=hello removed=false valid=true changed=true created=true value=world] 18:06:43,339 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [EntryWrappingInterceptor] Committed entry ClusteredRepeatableReadEntry(5d852771){key=hello, value=world, oldValue=null, isCreated=false, isChanged=false, isRemoved=false, isValid=true} 18:06:43,339 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] CommandTopologyId=2, localTopologyId=2 18:06:43,355 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 242 to NodeA-3607 18:06:43,356 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:43,381 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] Response(s) to CommitCommand {gtx=GlobalTransaction::171:local, cacheName='___defaultcache', topologyId=2} is {NodeB-43205=null} 18:06:43,402 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Suppressing invocation of method handleCommitCommand. 18:06:43,402 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ReadCommittedEntry] Updating entry (key=hello removed=false valid=true changed=true created=true value=world] 18:06:43,402 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryWrappingInterceptor] Committed entry ClusteredRepeatableReadEntry(c2aaf79){key=hello, value=world, oldValue=null, isCreated=false, isChanged=false, isRemoved=false, isValid=true} 18:06:43,519 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [LockManagerImpl] Attempting to unlock hello 18:06:43,520 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractCacheTransaction] Clearing locked keys: [hello] 18:06:43,541 TRACE (testng-VersionedReplStateTransferTest:) [TransactionTable] Removed LocalTransaction{remoteLockedNodes=[NodeA-3607, NodeB-43205], isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, viewId=2} org.infinispan.transaction.synchronization.SyncLocalTransaction@ab from transaction table. 18:06:43,541 TRACE (testng-VersionedReplStateTransferTest:) [AbstractCacheTransaction] Transaction GlobalTransaction::171:local has completed, notifying listening threads. 18:06:43,541 TRACE (testng-VersionedReplStateTransferTest:) [AbstractEnlistmentAdapter] About to invoke tx completion notification on nodes null 18:06:43,541 TRACE (testng-VersionedReplStateTransferTest:) [RpcManagerImpl] NodeA-3607 broadcasting call TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=0, gtx=GlobalTransaction::171:local, cacheName=___defaultcache} to recipient list null 18:06:43,541 TRACE (testng-VersionedReplStateTransferTest:) [JGroupsTransport] dests=null, command=TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=2, gtx=GlobalTransaction::171:local, cacheName=___defaultcache} , mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=15000 18:06:43,541 TRACE (testng-VersionedReplStateTransferTest:) [CommandAwareRpcDispatcher] Replication task sending TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=2, gtx=GlobalTransaction::171:local, cacheName=___defaultcache} to single recipient NodeB-43205 with response mode GET_NONE 18:06:43,542 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 254 18:06:43,576 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=2, gtx=GlobalTransaction::171:local, cacheName=___defaultcache} [sender=NodeA-3607] 18:06:43,542 TRACE (testng-VersionedReplStateTransferTest:) [CommandAwareRpcDispatcher] Response: null 18:06:43,576 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=2, gtx=GlobalTransaction::171:local, cacheName=___defaultcache} 18:06:43,576 TRACE (testng-VersionedReplStateTransferTest:) [RpcManagerImpl] Response(s) to TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=2, gtx=GlobalTransaction::171:local, cacheName=___defaultcache} is {} 18:06:43,576 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command GetKeyValueCommand {key=hello, flags=null} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@7ae17f7a] 18:06:43,576 TRACE (OOB-1,ISPN,NodeB-43205:) [TxCompletionNotificationCommand] Processing completed transaction GlobalTransaction::171:local 18:06:43,585 TRACE (OOB-1,ISPN,NodeB-43205:) [TransactionTable] Removed RemoteTransaction{modifications=[PutKeyValueCommand{key=hello, value=world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], lookedUpEntries={hello=ClusteredRepeatableReadEntry(5d852771){key=hello, value=world, oldValue=null, isCreated=false, isChanged=false, isRemoved=false, isValid=true}}, lockedKeys= null, backupKeyLocks [hello], missingLookedUpEntries false, tx=GlobalTransaction::171:remote} from transaction table. 18:06:43,605 TRACE (OOB-1,ISPN,NodeB-43205:) [AbstractCacheTransaction] Transaction GlobalTransaction::171:remote has completed, notifying listening threads. 18:06:43,605 TRACE (OOB-1,ISPN,NodeB-43205:) [StateTransferManagerImpl] CommandTopologyId=2, localTopologyId=2 18:06:43,605 TRACE (OOB-1,ISPN,NodeB-43205:) [LockManagerImpl] Attempting to unlock keys [] 18:06:43,585 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command GetKeyValueCommand {key=hello, flags=null}, originLocal=true 18:06:43,605 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:43,607 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Retrieved from container ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}} 18:06:43,608 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: GetKeyValueCommand {key=hello, flags=null}. 18:06:43,608 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [GetKeyValueCommand] Found value world 18:06:43,609 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command GetKeyValueCommand {key=hello, flags=null} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@24429290] 18:06:43,625 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command GetKeyValueCommand {key=hello, flags=null}, originLocal=true 18:06:43,625 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:43,625 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Retrieved from container ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}} 18:06:43,625 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: GetKeyValueCommand {key=hello, flags=null}. 18:06:43,625 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [GetKeyValueCommand] Found value world 18:06:43,626 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command GetKeyValueCommand {key=hello, flags=null} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@58e43f50] 18:06:43,654 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command GetKeyValueCommand {key=hello, flags=null}, originLocal=true 18:06:43,654 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Created a new local transaction: LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, viewId=2} org.infinispan.transaction.synchronization.SyncLocalTransaction@c5 18:06:43,654 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:43,663 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Retrieved from container ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}} 18:06:43,663 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: GetKeyValueCommand {key=hello, flags=null}. 18:06:43,663 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [GetKeyValueCommand] Found value world 18:06:43,664 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:43,664 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:43,664 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:43,665 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:43,665 DEBUG (testng-VersionedReplStateTransferTest:) [ModuleProperties] No module lifecycle SPI classes available 18:06:43,673 DEBUG (testng-VersionedReplStateTransferTest:) [ModuleProperties] No module command extensions to load 18:06:43,675 DEBUG (testng-VersionedReplStateTransferTest:) [AbstractJBossMarshaller] Using JBoss Marshalling 18:06:43,676 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheNames [r=true,w=false,is=false,type=java.lang.String] 18:06:43,676 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute createdCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:43,676 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute runningCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:43,676 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute version [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterSize [r=true,w=false,is=false,type=int] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute cacheManagerStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute name [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterMembers [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute physicalAddresses [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterName [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute nodeAddress [r=true,w=false,is=false,type=java.lang.String] 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:43,677 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@3c68076d under infinispan99:type=CacheManager,name="DefaultCacheManager",component=CacheManager 18:06:43,677 DEBUG (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Started cache manager ISPN on NodeC 18:06:43,678 TRACE (testng-VersionedReplStateTransferTest:) [TestCacheManagerFactory] Adding DCM (NodeC) for method: 'org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:77)' 18:06:43,678 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000078: Starting JGroups Channel 18:06:43,731 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [Configurator] set property TCP.diagnostics_addr to default value /224.0.75.75 18:06:43,741 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [FRAG2] received CONFIG event: {bind_addr=/127.0.0.1} 18:06:43,769 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] address=NodeC-45489, cluster=ISPN, physical address=127.0.0.1:9502 18:06:43,770 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Discoveries for DiscoveryKey{clusterName='ISPN', testName='org.infinispan.container.versioning.VersionedReplStateTransferTest'} are : {NodeB-43205=TEST_PING@NodeB-43205, NodeA-3607=TEST_PING@NodeA-3607} 18:06:43,770 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Add discovery for NodeC-45489 to cache. The cache now contains: {NodeB-43205=TEST_PING@NodeB-43205, NodeA-3607=TEST_PING@NodeA-3607, NodeC-45489=TEST_PING@NodeC-45489} 18:06:43,770 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Map NodeB-43205 with physical address 127.0.0.1:9501 in TEST_PING@NodeC-45489 18:06:43,778 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Map NodeC-45489 with physical address 127.0.0.1:9502 in TEST_PING@NodeB-43205 18:06:43,779 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Returning ping rsp: NodeB-43205, view_id=[NodeA-3607|1] ([NodeA-3607|1] [NodeA-3607, NodeB-43205]), is_server=true, is_coord=false, logical_name=NodeB-43205, physical_addrs=127.0.0.1:9501 18:06:43,799 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Map NodeA-3607 with physical address 127.0.0.1:9500 in TEST_PING@NodeC-45489 18:06:43,799 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Map NodeC-45489 with physical address 127.0.0.1:9502 in TEST_PING@NodeA-3607 18:06:43,800 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Returning ping rsp: NodeA-3607, view_id=[NodeA-3607|1] ([NodeA-3607|1] [NodeA-3607, NodeB-43205]), is_server=true, is_coord=true, logical_name=NodeA-3607, physical_addrs=127.0.0.1:9500 18:06:43,800 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] Skipping sending discovery to self 18:06:43,800 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TEST_PING] discovery took 30 ms: responses: 2 total (2 servers (1 coord), 0 clients) 18:06:43,811 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] election results: {NodeA-3607=2} 18:06:43,811 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] sending JOIN(NodeC-45489) to NodeA-3607 18:06:43,861 DEBUG (ViewHandler,ISPN,NodeA-3607:) [STABLE] suspending message garbage collection 18:06:43,912 DEBUG (ViewHandler,ISPN,NodeA-3607:) [STABLE] resume task started, max_suspend_time=33000 18:06:43,913 DEBUG (Incoming-1,ISPN,NodeA-3607:) [NAKACK2] [NodeA-3607 setDigest()] existing digest: NodeA-3607: [4 (4)], NodeB-43205: [0 (0)] new digest: NodeA-3607: [3 (3)], NodeB-43205: [0 (0)], NodeC-45489: [0 (0)] resulting digest: NodeC-45489: [0 (0)], NodeA-3607: [4 (4)], NodeB-43205: [0 (0)] 18:06:43,913 DEBUG (Incoming-1,ISPN,NodeA-3607:) [GMS] NodeA-3607: installing view [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:43,913 DEBUG (Incoming-1,ISPN,NodeA-3607:) [STABLE] [ergonomics] setting max_bytes to 1.2MB (3 members) 18:06:43,913 DEBUG (Incoming-1,ISPN,NodeA-3607:) [JGroupsTransport] New view accepted: [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:43,914 INFO (Incoming-1,ISPN,NodeA-3607:) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:43,914 TRACE (Incoming-1,ISPN,NodeA-3607:) [ClusterIdGenerator] Calculated rank based on view EventImpl{type=VIEW_CHANGED, newMembers=[NodeA-3607, NodeB-43205, NodeC-45489], oldMembers=[NodeA-3607, NodeB-43205], localAddress=NodeA-3607, viewId=2, subgroupsMerged=null, mergeView=false} and result was 562954248388608 18:06:43,914 TRACE (notification-thread-0,NodeA:) [ClusterTopologyManagerImpl] Received new cluster view: 2 18:06:43,914 TRACE (notification-thread-0,NodeA:) [ClusterTopologyManagerImpl] Updating cluster members for all the caches. New list is [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:43,946 DEBUG (Incoming-1,ISPN,NodeB-43205:) [NAKACK2] [NodeB-43205 setDigest()] existing digest: NodeA-3607: [4 (4)], NodeB-43205: [0 (0)] new digest: NodeA-3607: [3 (3)], NodeB-43205: [0 (0)], NodeC-45489: [0 (0)] resulting digest: NodeC-45489: [0 (0)], NodeA-3607: [4 (4)], NodeB-43205: [0 (0)] 18:06:44,068 DEBUG (Incoming-1,ISPN,NodeB-43205:) [GMS] NodeB-43205: installing view [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,076 DEBUG (Incoming-1,ISPN,NodeB-43205:) [STABLE] [ergonomics] setting max_bytes to 1.2MB (3 members) 18:06:44,113 DEBUG (Incoming-1,ISPN,NodeB-43205:) [JGroupsTransport] New view accepted: [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,114 INFO (Incoming-1,ISPN,NodeB-43205:) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,114 TRACE (Incoming-1,ISPN,NodeB-43205:) [ClusterIdGenerator] Calculated rank based on view EventImpl{type=VIEW_CHANGED, newMembers=[NodeA-3607, NodeB-43205, NodeC-45489], oldMembers=[NodeA-3607, NodeB-43205], localAddress=NodeB-43205, viewId=2, subgroupsMerged=null, mergeView=false} and result was 562958543355904 18:06:44,129 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [NAKACK2] [NodeC-45489 setDigest()] existing digest: [] new digest: NodeA-3607: [3 (3)], NodeB-43205: [0 (0)], NodeC-45489: [0 (0)] resulting digest: NodeC-45489: [0 (0)], NodeA-3607: [3 (3)], NodeB-43205: [0 (0)] 18:06:44,129 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [GMS] NodeC-45489: installing view [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,130 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [STABLE] [ergonomics] setting max_bytes to 1.2MB (3 members) 18:06:44,130 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] New view accepted: [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,150 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeA-3607|2] [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,151 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] ISPN000079: Cache local address is NodeC-45489, physical addresses are [127.0.0.1:9502] 18:06:44,151 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] Waiting on view being accepted 18:06:44,128 TRACE (notification-thread-0,NodeB:) [ClusterTopologyManagerImpl] Received new cluster view: 2 18:06:44,152 DEBUG (ViewHandler,ISPN,NodeA-3607:) [STABLE] resuming message garbage collection 18:06:44,152 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:44,189 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferConfigurationBuilder] Cache is distributed or replicated but state transfer was not defined, enabling it by default 18:06:44,189 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [DefaultCacheManager] About to wire and start cache ___defaultcache 18:06:44,208 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractJBossMarshaller] Using JBoss Marshalling 18:06:44,225 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [InterceptorChain] Interceptor chain size: 8 18:06:44,225 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [InterceptorChain] Interceptor chain is: >> org.infinispan.interceptors.InvocationContextInterceptor >> org.infinispan.statetransfer.StateTransferInterceptor >> org.infinispan.interceptors.TxInterceptor >> org.infinispan.interceptors.NotificationInterceptor >> org.infinispan.interceptors.locking.OptimisticLockingInterceptor >> org.infinispan.interceptors.VersionedEntryWrappingInterceptor >> org.infinispan.interceptors.VersionedReplicationInterceptor >> org.infinispan.interceptors.CallInterceptor 18:06:44,226 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionFactory] Setting factory enum to NODLD_NORECOVERY_NOXA 18:06:44,226 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String showInDoubtTransactions 18:06:44,238 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:44,250 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:44,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forget 18:06:44,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forget 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute joinComplete [r=true,w=false,is=true,type=boolean] 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute stateTransferInProgress [r=true,w=false,is=true,type=boolean] 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute pendingViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:44,252 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute replicationFailures [r=true,w=false,is=false,type=long] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute averageReplicationTime [r=true,w=false,is=false,type=long] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute committedViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute successRatioFloatingPoint [r=true,w=false,is=false,type=double] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute replicationCount [r=true,w=false,is=false,type=long] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute successRatio [r=true,w=false,is=false,type=java.lang.String] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void resetStatistics 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheName [r=true,w=false,is=false,type=java.lang.String] 18:06:44,254 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute configurationAsXmlString [r=true,w=false,is=false,type=java.lang.String] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void stop 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void start 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute numberOfLocksHeld [r=true,w=false,is=false,type=int] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute numberOfLocksAvailable [r=true,w=false,is=false,type=int] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute concurrencyLevel [r=true,w=false,is=false,type=int] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute prepares [r=true,w=false,is=false,type=long] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute rollbacks [r=true,w=false,is=false,type=long] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute commits [r=true,w=false,is=false,type=long] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void resetStatistics 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute cacheName [r=true,w=false,is=false,type=java.lang.String] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Attribute configurationAsXmlString [r=true,w=false,is=false,type=java.lang.String] 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void stop 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void start 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [ResourceDMBean] Operation void recordKnownGlobalKeyset 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@26e949f7 under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RecoveryAdmin 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@1df70b16 under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=StateTransferManager 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@45f8e0b8 under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RpcManager 18:06:44,255 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@566973eb under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Cache 18:06:44,256 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@2539759f under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=LockManager 18:06:44,256 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@5f592cd0 under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Transactions 18:06:44,256 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Object name infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Cache already registered 18:06:44,256 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JmxUtil] Registered org.infinispan.jmx.ResourceDMBean@578d3c97 under infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RollingUpgradeManager 18:06:44,256 INFO (testng-VersionedReplStateTransferTest:___defaultcache) [CacheJmxRegistration] ISPN000031: MBeans were successfully registered to the platform MBean server. 18:06:44,256 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Starting StateTransferManager of cache ___defaultcache on node NodeC-45489 18:06:44,256 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTopologyManagerImpl] Node NodeC-45489 joining cache ___defaultcache 18:06:44,256 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=CacheTopologyControlCommand{cache=___defaultcache, type=JOIN, sender=NodeC-45489, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@79850a68, hashFunction=org.infinispan.commons.hash.MurmurHash3@59a26723, numSegments=12, numOwners=2, timeout=240000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=2}, mode=SYNCHRONOUS, timeout=240000 18:06:44,256 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=JOIN, sender=NodeC-45489, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@79850a68, hashFunction=org.infinispan.commons.hash.MurmurHash3@59a26723, numSegments=12, numOwners=2, timeout=240000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=2} to single recipient NodeA-3607 with response mode GET_ALL 18:06:44,273 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 278 18:06:44,281 TRACE (OOB-2,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=JOIN, sender=NodeC-45489, joinInfo=CacheJoinInfo{consistentHashFactory=org.infinispan.distribution.ch.ReplicatedConsistentHashFactory@30a93685, hashFunction=org.infinispan.commons.hash.MurmurHash3@353a6eb8, numSegments=12, numOwners=2, timeout=240000}, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=2} [sender=NodeC-45489] 18:06:44,282 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[NodeC-45489], topology=CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:44,315 TRACE (OOB-2,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Triggering rebalance for cache ___defaultcache 18:06:44,316 TRACE (OOB-2,ISPN,NodeA-3607:) [RequestCorrelator] sending rsp for 278 to NodeC-45489 18:06:44,316 TRACE (asyncTransportThread-0,NodeA:) [ClusterTopologyManagerImpl] Rebalancing consistent hash for cache ___defaultcache, members are [NodeA-3607, NodeB-43205, NodeC-45489] 18:06:44,317 TRACE (asyncTransportThread-0,NodeA:) [ClusterTopologyManagerImpl] Updating cache ___defaultcache topology for rebalance: CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}} 18:06:44,317 TRACE (asyncTransportThread-0,NodeA:) [RebalanceConfirmationCollector] Initialized rebalance confirmation collector 3@___defaultcache, initial list is [NodeC-45489, NodeA-3607, NodeB-43205] 18:06:44,317 TRACE (asyncTransportThread-0,NodeA:) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[NodeC-45489], topology=CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}} 18:06:44,317 DEBUG (asyncTransportThread-0,NodeA:) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}} 18:06:44,317 TRACE (asyncTransportThread-1,NodeA:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, throwable=null, viewId=2}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:44,317 TRACE (asyncTransportThread-1,NodeA:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, throwable=null, viewId=2} to addresses null with response mode GET_ALL 18:06:44,317 TRACE (asyncTransportThread-1,NodeA:) [CommandAwareRpcDispatcher] real_dests=[NodeC-45489, NodeB-43205] 18:06:44,318 DEBUG (asyncTransportThread-2,NodeA:___defaultcache) [LocalTopologyManagerImpl] Starting local rebalance for cache ___defaultcache, topology = CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}} 18:06:44,318 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeC-45489, NodeB-43205]}} 18:06:44,318 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeC-45489, NodeB-43205]} 18:06:44,318 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,319 DEBUG (asyncTransportThread-2,NodeA:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,319 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [TransactionTable] Changing minimum view ID from 2 to 3 18:06:44,319 DEBUG (asyncTransportThread-2,NodeA:___defaultcache) [ClusterTopologyManagerImpl] Finished local rebalance for cache ___defaultcache on node NodeA-3607, topology id = 3 18:06:44,319 TRACE (asyncTransportThread-2,NodeA:___defaultcache) [RebalanceConfirmationCollector] Rebalance confirmation collector 3@___defaultcache received confirmation for NodeA-3607, remaining list is [NodeC-45489, NodeB-43205] 18:06:44,319 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 281 18:06:44,319 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, throwable=null, viewId=2} [sender=NodeA-3607] 18:06:44,319 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [LocalTopologyManagerImpl] Starting local rebalance for cache ___defaultcache, topology = CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}} 18:06:44,319 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeC-45489, NodeB-43205]}} 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeC-45489, NodeB-43205]} 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeB-43205, joinInfo=null, topologyId=3, currentCH=null, pendingCH=null, throwable=null, viewId=2}, mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=0 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeB-43205, joinInfo=null, topologyId=3, currentCH=null, pendingCH=null, throwable=null, viewId=2} to single recipient NodeA-3607 with response mode GET_NONE 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:44,320 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Minimum view ID still is 2; nothing to change 18:06:44,320 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 281 to NodeA-3607 18:06:44,321 TRACE (OOB-1,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 282 18:06:44,321 TRACE (OOB-1,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeB-43205, joinInfo=null, topologyId=3, currentCH=null, pendingCH=null, throwable=null, viewId=2} [sender=NodeB-43205] 18:06:44,321 DEBUG (OOB-1,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Finished local rebalance for cache ___defaultcache on node NodeB-43205, topology id = 3 18:06:44,333 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Response: SuccessfulResponse{responseValue=CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null}} 18:06:44,333 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:44,333 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=2, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=null} 18:06:44,333 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]} 18:06:44,333 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,333 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Changing minimum view ID from -1 to 2 18:06:44,324 TRACE (OOB-1,ISPN,NodeA-3607:___defaultcache) [RebalanceConfirmationCollector] Rebalance confirmation collector 3@___defaultcache received confirmation for NodeB-43205, remaining list is [NodeC-45489] 18:06:44,342 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferManagerImpl] Waiting for initial state transfer to finish 18:06:44,346 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 281 18:06:44,348 TRACE (OOB-2,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_START, sender=NodeA-3607, joinInfo=null, topologyId=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, throwable=null, viewId=2} [sender=NodeA-3607] 18:06:44,363 TRACE (OOB-2,ISPN,NodeC-45489:) [ClusterTopologyManagerImpl] Received new cluster view: 2 18:06:44,366 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [LocalTopologyManagerImpl] Starting local rebalance for cache ___defaultcache, topology = CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}} 18:06:44,366 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=3, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205]}, pendingCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeC-45489, NodeB-43205]}} 18:06:44,366 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeC-45489, NodeB-43205]} 18:06:44,366 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [] 18:06:44,391 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Adding state transfer for segments: [0] 18:06:44,396 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [InboundTransferTask] Requesting transactions for segments [0] 18:06:44,425 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [JGroupsTransport] dests=[NodeB-43205], command=StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=GET_TRANSACTIONS, topologyId=3, segments=[0]}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:44,425 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=GET_TRANSACTIONS, topologyId=3, segments=[0]} to addresses [NodeB-43205] with response mode GET_ALL 18:06:44,442 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 288 18:06:44,470 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=GET_TRANSACTIONS, topologyId=3, segments=[0]} [sender=NodeC-45489] 18:06:44,470 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=GET_TRANSACTIONS, topologyId=3, segments=[0]} 18:06:44,470 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateProviderImpl] Received request for transactions from node NodeC-45489 for segments [0] with topology id 3 18:06:44,477 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateProviderImpl] Found 0 transaction(s) to transfer 18:06:44,477 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 288 to NodeC-45489 18:06:44,479 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [CommandAwareRpcDispatcher] Responses: [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=[]} , received=true, suspected=false] 18:06:44,479 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Applying 0 transactions transferred from NodeB-43205 18:06:44,479 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [JGroupsTransport] dests=[NodeB-43205], command=StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=START_STATE_TRANSFER, topologyId=3, segments=[0]}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:44,479 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=START_STATE_TRANSFER, topologyId=3, segments=[0]} to addresses [NodeB-43205] with response mode GET_ALL 18:06:44,483 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 292 18:06:44,483 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=START_STATE_TRANSFER, topologyId=3, segments=[0]} [sender=NodeC-45489] 18:06:44,483 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on StateRequestCommand{cache=___defaultcache, origin=NodeC-45489, type=START_STATE_TRANSFER, topologyId=3, segments=[0]} 18:06:44,483 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateProviderImpl] Starting outbound transfer of segments [0] to node NodeC-45489 with topology id 3 18:06:44,483 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateProviderImpl] Adding outbound transfer of segments [0] to NodeC-45489 18:06:44,487 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 292 to NodeC-45489 18:06:44,534 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [CommandAwareRpcDispatcher] Responses: [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=true} , received=true, suspected=false] 18:06:44,534 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,534 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Changing minimum view ID from 2 to 3 18:06:44,536 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 281 to NodeA-3607 18:06:44,537 TRACE (asyncTransportThread-1,NodeA:) [CommandAwareRpcDispatcher] Responses: [sender=NodeC-45489, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:06:44,500 TRACE (asyncTransportThread-0,NodeB:) [OutboundTransferTask] No cache store or the cache store is shared, no need to send any stored cache entries for segments: [0] 18:06:44,555 TRACE (asyncTransportThread-0,NodeB:) [OutboundTransferTask] Sending 1 cache entries from segments [0] to node NodeC-45489 18:06:44,578 TRACE (asyncTransportThread-0,NodeB:) [RpcManagerImpl] NodeB-43205 invoking in future call StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]} to recipient list [NodeC-45489] 18:06:44,578 TRACE (asyncTransportThread-1,NodeB:) [RpcManagerImpl] NodeB-43205 broadcasting call StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]} to recipient list [NodeC-45489] 18:06:44,615 TRACE (asyncTransportThread-1,NodeB:) [JGroupsTransport] dests=[NodeC-45489], command=StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]}, mode=SYNCHRONOUS, timeout=240000 18:06:44,615 TRACE (asyncTransportThread-1,NodeB:) [CommandAwareRpcDispatcher] Replication task sending StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]} to single recipient NodeC-45489 with response mode GET_ALL 18:06:44,578 TRACE (asyncTransportThread-0,NodeB:) [OutboundTransferTask] Outbound transfer of segments [0] to NodeC-45489 is complete 18:06:44,616 TRACE (asyncTransportThread-0,NodeB:) [StateProviderImpl] Removing completed outbound transfer of segments [0] to NodeC-45489 18:06:44,618 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 299 18:06:44,630 TRACE (OOB-2,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute command: StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]} [sender=NodeB-43205] 18:06:44,630 TRACE (OOB-2,ISPN,NodeC-45489:) [InboundInvocationHandlerImpl] Calling perform() on StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]} 18:06:44,630 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Applying new state for segment 0 from NodeB-43205: received 1 cache entries 18:06:44,630 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Received keys: [hello] 18:06:44,637 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [InvocationContextInterceptor] Invoked with command VersionedPutKeyValueCommand{key=hello, value=world, version=SimpleClusteredVersion{viewId=1, version=1}, flags=[CACHE_MODE_LOCAL, SKIP_SHARED_CACHE_STORE, SKIP_OWNERSHIP_CHECK, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=true, lifespanMillis=-1, maxIdleTimeMillis=-1} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@c4dc0a8] 18:06:44,637 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command VersionedPutKeyValueCommand{key=hello, value=world, version=SimpleClusteredVersion{viewId=1, version=1}, flags=[CACHE_MODE_LOCAL, SKIP_SHARED_CACHE_STORE, SKIP_OWNERSHIP_CHECK, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=true, lifespanMillis=-1, maxIdleTimeMillis=-1}, originLocal=false 18:06:44,637 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:44,637 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [EntryFactoryImpl] Retrieved from container null 18:06:44,637 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [EntryFactoryImpl] Creating new entry. 18:06:44,638 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [CallInterceptor] Executing command: VersionedPutKeyValueCommand{key=hello, value=world, version=SimpleClusteredVersion{viewId=1, version=1}, flags=[CACHE_MODE_LOCAL, SKIP_SHARED_CACHE_STORE, SKIP_OWNERSHIP_CHECK, IGNORE_RETURN_VALUES, SKIP_XSITE_BACKUP], putIfAbsent=true, lifespanMillis=-1, maxIdleTimeMillis=-1}. 18:06:44,638 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [VersionedReplicationInterceptor] LOCAL mode forced on invocation. Suppressing clustered events. 18:06:44,638 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [ReadCommittedEntry] Updating entry (key=hello removed=false valid=true changed=true created=true value=world] 18:06:44,638 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache ___defaultcache) [EntryWrappingInterceptor] Committed entry ClusteredRepeatableReadEntry(76361df9){key=hello, value=world, oldValue=null, isCreated=false, isChanged=false, isRemoved=false, isValid=true} 18:06:44,638 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [JGroupsTransport] dests=[NodeA-3607], command=CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeC-45489, joinInfo=null, topologyId=3, currentCH=null, pendingCH=null, throwable=null, viewId=2}, mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=0 18:06:44,638 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeC-45489, joinInfo=null, topologyId=3, currentCH=null, pendingCH=null, throwable=null, viewId=2} to single recipient NodeA-3607 with response mode GET_NONE 18:06:44,677 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:44,677 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] After applying the received state the data container has 1 keys 18:06:44,677 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Segments not received yet: {} 18:06:44,677 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 299 to NodeB-43205 18:06:44,678 TRACE (OOB-1,ISPN,NodeA-3607:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 305 18:06:44,678 TRACE (asyncTransportThread-1,NodeB:) [CommandAwareRpcDispatcher] Response: null 18:06:44,678 TRACE (OOB-1,ISPN,NodeA-3607:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=REBALANCE_CONFIRM, sender=NodeC-45489, joinInfo=null, topologyId=3, currentCH=null, pendingCH=null, throwable=null, viewId=2} [sender=NodeC-45489] 18:06:44,678 TRACE (asyncTransportThread-1,NodeB:) [RpcManagerImpl] Response(s) to StateResponseCommand{cache=___defaultcache, origin=NodeB-43205, topologyId=3, stateChunks=[StateChunk{segmentId=0, cacheEntries=[ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}}], isLastChunk=true}]} is {NodeC-45489=null} 18:06:44,678 DEBUG (OOB-1,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Finished local rebalance for cache ___defaultcache on node NodeC-45489, topology id = 3 18:06:44,681 TRACE (OOB-1,ISPN,NodeA-3607:___defaultcache) [RebalanceConfirmationCollector] Rebalance confirmation collector 3@___defaultcache received confirmation for NodeC-45489, remaining list is [] 18:06:44,681 DEBUG (OOB-1,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Finished cluster-wide rebalance for cache ___defaultcache, topology id = 3 18:06:44,681 DEBUG (OOB-1,ISPN,NodeA-3607:___defaultcache) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,681 TRACE (asyncTransportThread-2,NodeA:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:44,682 TRACE (asyncTransportThread-2,NodeA:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2} to addresses null with response mode GET_ALL 18:06:44,682 TRACE (asyncTransportThread-2,NodeA:) [CommandAwareRpcDispatcher] real_dests=[NodeC-45489, NodeB-43205] 18:06:44,681 DEBUG (asyncTransportThread-3,NodeA:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,697 TRACE (asyncTransportThread-3,NodeA:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,697 TRACE (asyncTransportThread-3,NodeA:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]} 18:06:44,697 TRACE (asyncTransportThread-3,NodeA:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,697 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 307 18:06:44,697 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 307 18:06:44,697 DEBUG (asyncTransportThread-3,NodeA:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,720 TRACE (asyncTransportThread-3,NodeA:___defaultcache) [TransactionTable] Changing minimum view ID from 3 to 4 18:06:44,697 TRACE (OOB-2,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2} [sender=NodeA-3607] 18:06:44,697 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2} [sender=NodeA-3607] 18:06:44,721 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,721 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,721 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,721 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,721 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]} 18:06:44,721 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,721 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,721 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Changing minimum view ID from 3 to 4 18:06:44,740 DEBUG (testng-VersionedReplStateTransferTest:___defaultcache) [CacheImpl] Started cache ___defaultcache on NodeC-45489 18:06:44,721 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]} 18:06:44,752 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,752 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,752 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Minimum view ID still is 2; nothing to change 18:06:44,752 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 307 to NodeA-3607 18:06:44,740 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command GetKeyValueCommand {key=hello, flags=null} and InvocationContext [org.infinispan.context.SingleKeyNonTxInvocationContext@49a32cdb] 18:06:44,766 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command GetKeyValueCommand {key=hello, flags=null}, originLocal=true 18:06:44,766 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:44,766 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Retrieved from container ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}} 18:06:44,766 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: GetKeyValueCommand {key=hello, flags=null}. 18:06:44,766 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [GetKeyValueCommand] Found value world 18:06:44,766 DEBUG (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Stopping cache manager ISPN on NodeA-3607 18:06:44,767 DEBUG (testng-VersionedReplStateTransferTest:) [CacheImpl] Stopping cache ___defaultcache on NodeA-3607 18:06:44,767 TRACE (testng-VersionedReplStateTransferTest:) [ComponentsJmxRegistration] Unregistering jmx resources.. 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute rollbacks [r=true,w=false,is=false,type=long] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute commits [r=true,w=false,is=false,type=long] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute prepares [r=true,w=false,is=false,type=long] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void resetStatistics 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forget 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forget 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String showInDoubtTransactions 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute joinComplete [r=true,w=false,is=true,type=boolean] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute stateTransferInProgress [r=true,w=false,is=true,type=boolean] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute numberOfLocksHeld [r=true,w=false,is=false,type=int] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute concurrencyLevel [r=true,w=false,is=false,type=int] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute numberOfLocksAvailable [r=true,w=false,is=false,type=int] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void recordKnownGlobalKeyset 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute replicationFailures [r=true,w=false,is=false,type=long] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute replicationCount [r=true,w=false,is=false,type=long] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute committedViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute averageReplicationTime [r=true,w=false,is=false,type=long] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute successRatio [r=true,w=false,is=false,type=java.lang.String] 18:06:44,768 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute pendingViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:44,769 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute successRatioFloatingPoint [r=true,w=false,is=false,type=double] 18:06:44,769 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void resetStatistics 18:06:44,740 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 307 to NodeA-3607 18:06:44,773 TRACE (asyncTransportThread-2,NodeA:) [CommandAwareRpcDispatcher] Responses: [sender=NodeC-45489, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:06:44,774 TRACE (OOB-1,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[], topology=CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeA-3607, NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,774 DEBUG (testng-VersionedReplStateTransferTest:) [TransactionTable] Wait for on-going transactions to finish for 30 seconds. 18:06:44,808 DEBUG (testng-VersionedReplStateTransferTest:) [TransactionTable] All transactions terminated 18:06:44,808 TRACE (testng-VersionedReplStateTransferTest:) [StateConsumerImpl] Shutting down StateConsumer of cache ___defaultcache on node NodeA-3607 18:06:44,808 TRACE (testng-VersionedReplStateTransferTest:) [StateProviderImpl] Shutting down StateProvider of cache ___defaultcache on node NodeA-3607 18:06:44,808 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferManagerImpl] Shutting down StateTransferManager of cache ___defaultcache on node NodeA-3607 18:06:44,808 DEBUG (testng-VersionedReplStateTransferTest:) [LocalTopologyManagerImpl] Node NodeA-3607 leaving cache ___defaultcache 18:06:44,809 TRACE (testng-VersionedReplStateTransferTest:) [ComponentsJmxRegistration] Unregistering jmx resources.. 18:06:44,809 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute cacheManagerStatus [r=true,w=false,is=false,type=java.lang.String] 18:06:44,809 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [ClusterTopologyManagerImpl] Cache ___defaultcache topology updated: CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,809 DEBUG (asyncTransportThread-1,NodeA:___defaultcache) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,809 TRACE (asyncTransportThread-3,NodeA:___defaultcache) [LocalTopologyManagerImpl] Ignoring consistent hash update 4 for cache ___defaultcache that doesn't exist locally 18:06:44,809 TRACE (asyncTransportThread-0,NodeA:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:44,818 TRACE (asyncTransportThread-0,NodeA:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2} to addresses null with response mode GET_ALL 18:06:44,819 TRACE (asyncTransportThread-0,NodeA:) [CommandAwareRpcDispatcher] real_dests=[NodeC-45489, NodeB-43205] 18:06:44,809 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterMembers [r=true,w=false,is=false,type=java.lang.String] 18:06:44,819 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute nodeAddress [r=true,w=false,is=false,type=java.lang.String] 18:06:44,819 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:44,819 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute physicalAddresses [r=true,w=false,is=false,type=java.lang.String] 18:06:44,819 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute createdCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:44,819 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 314 18:06:44,819 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2} [sender=NodeA-3607] 18:06:44,819 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,820 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,820 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StaleTransactionCleanupService] Saw 1 leavers - kicking off a lock breaking task 18:06:44,820 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]} 18:06:44,820 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,820 TRACE (LockBreakingService,___defaultcache,NodeB-43205:) [TransactionTable] No global transactions pertain to originator(s) [NodeA-3607] who have left the cluster. 18:06:44,808 TRACE (OOB-1,ISPN,NodeA-3607:___defaultcache) [DefaultRebalancePolicy] Not triggering rebalance for cache ___defaultcache, no joiners and the current consistent hash is already balanced 18:06:44,820 TRACE (LockBreakingService,___defaultcache,NodeB-43205:) [TransactionTable] Completed cleaning transactions originating on leavers 18:06:44,820 DEBUG (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,819 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute version [r=true,w=false,is=false,type=java.lang.String] 18:06:44,819 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 314 18:06:44,848 TRACE (OOB-2,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeA-3607, joinInfo=null, topologyId=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null, throwable=null, viewId=2} [sender=NodeA-3607] 18:06:44,848 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,848 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,848 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StaleTransactionCleanupService] Saw 1 leavers - kicking off a lock breaking task 18:06:44,858 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]} 18:06:44,858 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:44,858 DEBUG (OOB-2,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:44,858 TRACE (OOB-2,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Minimum view ID still is 4; nothing to change 18:06:44,859 TRACE (OOB-2,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 314 to NodeA-3607 18:06:44,862 TRACE (LockBreakingService,___defaultcache,NodeC-45489:) [TransactionTable] No global transactions pertain to originator(s) [NodeA-3607] who have left the cluster. 18:06:44,847 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheNames [r=true,w=false,is=false,type=java.lang.String] 18:06:44,877 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute runningCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:06:44,877 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterName [r=true,w=false,is=false,type=java.lang.String] 18:06:44,877 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute name [r=true,w=false,is=false,type=java.lang.String] 18:06:44,877 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterSize [r=true,w=false,is=false,type=int] 18:06:44,881 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:44,881 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:06:44,881 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan75:type=CacheManager,name="DefaultCacheManager",component=CacheManager 18:06:44,881 INFO (testng-VersionedReplStateTransferTest:) [JGroupsTransport] ISPN000080: Disconnecting and closing JGroups Channel 18:06:44,843 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Minimum view ID still is 2; nothing to change 18:06:44,891 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 314 to NodeA-3607 18:06:44,893 TRACE (asyncTransportThread-0,NodeA:) [CommandAwareRpcDispatcher] Responses: [sender=NodeC-45489, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] [sender=NodeB-43205, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:06:44,862 TRACE (LockBreakingService,___defaultcache,NodeC-45489:) [TransactionTable] Completed cleaning transactions originating on leavers 18:06:44,897 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [DefaultRebalancePolicy] Cache ___defaultcache status changed: joiners=[], topology=CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null} 18:06:44,958 TRACE (asyncTransportThread-1,NodeA:___defaultcache) [DefaultRebalancePolicy] Not triggering rebalance for cache ___defaultcache, no joiners and the current consistent hash is already balanced 18:06:44,924 DEBUG (testng-VersionedReplStateTransferTest:) [TCPConnectionMap] connection table is not running, discarding message to 127.0.0.1:9501 18:06:44,922 DEBUG (Incoming-1,ISPN,NodeC-45489:) [GMS] NodeC-45489: installing view [NodeB-43205|3] [NodeB-43205, NodeC-45489] 18:06:44,922 DEBUG (Incoming-1,ISPN,NodeB-43205:) [GMS] NodeB-43205: installing view [NodeB-43205|3] [NodeB-43205, NodeC-45489] 18:06:44,971 DEBUG (Incoming-1,ISPN,NodeB-43205:) [STABLE] resuming message garbage collection 18:06:44,972 DEBUG (Incoming-1,ISPN,NodeB-43205:) [NAKACK2] removed NodeA-3607 from xmit_table (not member anymore) 18:06:44,973 DEBUG (Incoming-1,ISPN,NodeB-43205:) [STABLE] [ergonomics] setting max_bytes to 800KB (2 members) 18:06:45,043 DEBUG (Incoming-1,ISPN,NodeB-43205:) [JGroupsTransport] New view accepted: [NodeB-43205|3] [NodeB-43205, NodeC-45489] 18:06:45,044 INFO (Incoming-1,ISPN,NodeB-43205:) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeB-43205|3] [NodeB-43205, NodeC-45489] 18:06:44,961 DEBUG (Incoming-1,ISPN,NodeC-45489:) [NAKACK2] removed NodeA-3607 from xmit_table (not member anymore) 18:06:44,960 DEBUG (testng-VersionedReplStateTransferTest:) [TCPConnectionMap] connection table is not running, discarding message to 127.0.0.1:9502 18:06:45,047 DEBUG (testng-VersionedReplStateTransferTest:) [TEST_PING] Stop discovery for NodeA-3607 18:06:45,047 DEBUG (testng-VersionedReplStateTransferTest:) [TCP] closing sockets and stopping threads 18:06:45,052 INFO (testng-VersionedReplStateTransferTest:) [JGroupsTransport] ISPN000082: Stopping the RpcDispatcher 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Implicit transaction started! 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Marked tx as implicit. 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@259f7c89] 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}, originLocal=true 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [TransactionTable] Created a new local transaction: LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, viewId=4} org.infinispan.transaction.synchronization.SyncLocalTransaction@f5 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [EntryFactoryImpl] Retrieved from container ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}} 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}. 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [LocalTransaction] Adding modification PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}. Mod list is null 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Committing transaction as it was implicit: TransactionImple < ac, BasicAction: 0:ffff7f000001:9ca1:5079837d:f9 status: ActionStatus.RUNNING > 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:) [SynchronizationAdapter] beforeCompletion called for LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=null, viewId=4} org.infinispan.transaction.synchronization.SyncLocalTransaction@f5 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:) [TransactionCoordinator] Sending prepare command through the chain: VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'} 18:06:45,054 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@9a919ab] 18:06:45,055 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen=null, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'}, originLocal=true 18:06:45,055 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [OptimisticLockingInterceptor] Not using lock reordering as we have a single key. 18:06:45,055 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Suppressing invocation of method handlePrepareCommand. 18:06:45,055 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] NodeC-45489 broadcasting call VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'} to recipient list null 18:06:45,055 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=null, command=VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'}, mode=SYNCHRONOUS, timeout=15000 18:06:45,055 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'} to addresses null with response mode GET_ALL 18:06:45,056 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] real_dests=[NodeA-3607, NodeB-43205] 18:06:45,057 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 318 18:06:45,051 DEBUG (Incoming-1,ISPN,NodeC-45489:) [STABLE] [ergonomics] setting max_bytes to 800KB (2 members) 18:06:45,058 DEBUG (Incoming-1,ISPN,NodeC-45489:) [JGroupsTransport] New view accepted: [NodeB-43205|3] [NodeB-43205, NodeC-45489] 18:06:45,058 INFO (Incoming-1,ISPN,NodeC-45489:) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeB-43205|3] [NodeB-43205, NodeC-45489] 18:06:45,058 TRACE (Incoming-1,ISPN,NodeC-45489:) [ClusterIdGenerator] Calculated rank based on view EventImpl{type=VIEW_CHANGED, newMembers=[NodeB-43205, NodeC-45489], oldMembers=[NodeA-3607, NodeB-43205, NodeC-45489], localAddress=NodeC-45489, viewId=3, subgroupsMerged=null, mergeView=false} and result was 844433520066560 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:local, cacheName='___defaultcache'} [sender=NodeC-45489] 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:remote, cacheName='___defaultcache'} 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:) [TransactionTable] Created and registered remote transaction RemoteTransaction{modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], lookedUpEntries={}, lockedKeys= null, backupKeyLocks null, missingLookedUpEntries false, tx=GlobalTransaction::245:remote} 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:) [PrepareCommand] Invoking remotely originated prepare: VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:remote, cacheName='___defaultcache'} with invocation context: org.infinispan.context.impl.RemoteTxInvocationContext@cd9d6c1 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [InvocationContextInterceptor] Invoked with command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:remote, cacheName='___defaultcache'} and InvocationContext [org.infinispan.context.impl.RemoteTxInvocationContext@cd9d6c1] 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command VersionedPrepareCommand {modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], onePhaseCommit=false, versionsSeen={hello=SimpleClusteredVersion{viewId=1, version=1}}, gtx=GlobalTransaction::245:remote, cacheName='___defaultcache'}, originLocal=false 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [OptimisticLockingInterceptor] Not using lock reordering as we have a single key. 18:06:45,059 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [OptimisticLockingInterceptor] Checking for pending locks and then locking key hello 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [AbstractCacheTransaction] Transaction gtx=GlobalTransaction::197:local potentially locks key hello? false 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [AbstractCacheTransaction] Transaction gtx=GlobalTransaction::245:remote potentially locks key hello? false 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [OptimisticLockingInterceptor] Finished waiting for other potential lockers, trying to acquire the lock on hello 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [LockManagerImpl] Attempting to lock hello with acquisition timeout of 10000 millis 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [LockManagerImpl] Successfully acquired lock hello! 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [AbstractCacheTransaction] Registering locked key: hello 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [EntryFactoryImpl] Exists in context? null 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [EntryFactoryImpl] Retrieved from container ImmortalCacheEntry{key=hello, value=VersionedImmortalCacheValue {value=world, version=SimpleClusteredVersion{viewId=1, version=1}}} 18:06:45,060 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [RemoteTransaction] Adding key hello to tx GlobalTransaction::245:remote 18:06:45,044 TRACE (Incoming-1,ISPN,NodeB-43205:) [ClusterIdGenerator] Calculated rank based on view EventImpl{type=VIEW_CHANGED, newMembers=[NodeB-43205, NodeC-45489], oldMembers=[NodeA-3607, NodeB-43205, NodeC-45489], localAddress=NodeB-43205, viewId=3, subgroupsMerged=null, mergeView=false} and result was 844429225099264 18:06:45,065 TRACE (notification-thread-0,NodeB:) [ClusterTopologyManagerImpl] Received new cluster view: 3 18:06:45,152 DEBUG (notification-thread-0,NodeB:) [ClusterTopologyManagerImpl] Recovering running caches in the cluster 18:06:45,062 TRACE (notification-thread-0,NodeC:) [ClusterTopologyManagerImpl] Received new cluster view: 3 18:06:45,061 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CallInterceptor] Executing command: PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}. 18:06:45,156 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [CallInterceptor] Suppressing invocation of method handlePrepareCommand. 18:06:45,157 TRACE (asyncTransportThread-2,NodeB:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=null, type=GET_STATUS, sender=NodeB-43205, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=2}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:45,157 TRACE (OOB-1,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] CommandTopologyId=4, localTopologyId=4 18:06:45,157 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] sending rsp for 318 to NodeC-45489 18:06:45,160 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Responses: [sender=NodeA-3607, received=false, suspected=true] [sender=NodeB-43205, retval=SuccessfulResponse{responseValue={hello=SimpleClusteredVersion{viewId=3, version=2}}} , received=true, suspected=false] 18:06:45,247 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] replication exception: org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: NodeA-3607 at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:102) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:540) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:177) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:199) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:256) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:243) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:238) at org.infinispan.interceptors.VersionedReplicationInterceptor.broadcastPrepare(VersionedReplicationInterceptor.java:59) at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:125) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.VersionedEntryWrappingInterceptor.visitPrepareCommand(VersionedEntryWrappingInterceptor.java:66) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:118) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:107) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:207) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:187) at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:93) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:129) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:93) at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:127) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:347) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1009) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 18:06:45,248 ERROR (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] ISPN000136: Execution error org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: NodeA-3607 at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:102) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:540) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:177) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:199) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:256) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:243) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:238) at org.infinispan.interceptors.VersionedReplicationInterceptor.broadcastPrepare(VersionedReplicationInterceptor.java:59) at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:125) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.VersionedEntryWrappingInterceptor.visitPrepareCommand(VersionedEntryWrappingInterceptor.java:66) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:118) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:107) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:207) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:187) at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:93) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:129) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:93) at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:127) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:347) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1009) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 18:06:45,249 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Transaction marked for rollback as exception was received. 18:06:45,249 ERROR (testng-VersionedReplStateTransferTest:) [TransactionCoordinator] Error while processing prepare org.infinispan.remoting.transport.jgroups.SuspectException: Suspected member: NodeA-3607 at org.infinispan.remoting.transport.AbstractTransport.parseResponseAndAddToResponseList(AbstractTransport.java:102) at org.infinispan.remoting.transport.jgroups.JGroupsTransport.invokeRemotely(JGroupsTransport.java:540) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:177) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:199) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:256) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:243) at org.infinispan.remoting.rpc.RpcManagerImpl.invokeRemotely(RpcManagerImpl.java:238) at org.infinispan.interceptors.VersionedReplicationInterceptor.broadcastPrepare(VersionedReplicationInterceptor.java:59) at org.infinispan.interceptors.ReplicationInterceptor.visitPrepareCommand(ReplicationInterceptor.java:125) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.VersionedEntryWrappingInterceptor.visitPrepareCommand(VersionedEntryWrappingInterceptor.java:66) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.invokeNextAndCommitIf1Pc(AbstractTxLockingInterceptor.java:118) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.NotificationInterceptor.visitPrepareCommand(NotificationInterceptor.java:58) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.TxInterceptor.visitPrepareCommand(TxInterceptor.java:107) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.statetransfer.StateTransferInterceptor.handleTopologyAffectedCommand(StateTransferInterceptor.java:207) at org.infinispan.statetransfer.StateTransferInterceptor.handleTxCommand(StateTransferInterceptor.java:187) at org.infinispan.statetransfer.StateTransferInterceptor.visitPrepareCommand(StateTransferInterceptor.java:93) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.base.CommandInterceptor.invokeNextInterceptor(CommandInterceptor.java:118) at org.infinispan.interceptors.InvocationContextInterceptor.handleAll(InvocationContextInterceptor.java:129) at org.infinispan.interceptors.InvocationContextInterceptor.handleDefault(InvocationContextInterceptor.java:93) at org.infinispan.commands.AbstractVisitor.visitPrepareCommand(AbstractVisitor.java:127) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:132) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:347) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:141) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1009) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 18:06:45,157 TRACE (asyncTransportThread-2,NodeB:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=null, type=GET_STATUS, sender=NodeB-43205, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=2} to addresses null with response mode GET_ALL 18:06:45,251 TRACE (testng-VersionedReplStateTransferTest:) [TransactionCoordinator] rollback transaction GlobalTransaction::245:local 18:06:45,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@15ad241f] 18:06:45,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=-1}, originLocal=true 18:06:45,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] NodeC-45489 broadcasting call RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=-1} to recipient list null 18:06:45,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=null, command=RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4}, mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=15000 18:06:45,251 TRACE (asyncTransportThread-2,NodeB:) [CommandAwareRpcDispatcher] real_dests=[NodeC-45489] 18:06:45,269 TRACE (OOB-1,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 319 18:06:45,270 TRACE (OOB-1,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=null, type=GET_STATUS, sender=NodeB-43205, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=2} [sender=NodeB-43205] 18:06:45,270 TRACE (OOB-1,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 319 to NodeB-43205 18:06:45,251 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4} to single recipient NodeB-43205 with response mode GET_NONE 18:06:45,290 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:45,290 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] Response(s) to RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4} is {} 18:06:45,290 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Suppressing invocation of method handleRollbackCommand. 18:06:45,290 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractCacheTransaction] Clearing locked keys: null 18:06:45,290 TRACE (testng-VersionedReplStateTransferTest:) [TransactionTable] Removed LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=[hello], viewId=4} org.infinispan.transaction.synchronization.SyncLocalTransaction@f5 from transaction table. 18:06:45,290 TRACE (testng-VersionedReplStateTransferTest:) [AbstractCacheTransaction] Transaction GlobalTransaction::245:local has completed, notifying listening threads. 18:06:45,291 WARN (testng-VersionedReplStateTransferTest:) [arjuna] ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff7f000001:9ca1:5079837d:fa, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=[hello], viewId=4} org.infinispan.transaction.synchronization.SyncLocalTransaction@f5} org.infinispan.transaction.synchronization.SynchronizationAdapter@114 > org.infinispan.CacheException: Could not prepare. at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1009) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) ... 31 more 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:) [SynchronizationAdapter] afterCompletion(4) called for LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=null, backupKeyLocks=[hello], viewId=4} org.infinispan.transaction.synchronization.SyncLocalTransaction@f5. 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:) [TransactionCoordinator] rollback transaction GlobalTransaction::245:local 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=-1} and InvocationContext [org.infinispan.context.impl.LocalTxInvocationContext@7f8388eb] 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=-1}, originLocal=true 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] NodeC-45489 broadcasting call RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=-1} to recipient list null 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [JGroupsTransport] dests=null, command=RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4}, mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=15000 18:06:45,276 TRACE (asyncTransportThread-2,NodeB:) [CommandAwareRpcDispatcher] Responses: [sender=NodeC-45489, retval=SuccessfulResponse{responseValue={___defaultcache=[Ljava.lang.Object;@5775be14}} , received=true, suspected=false] 18:06:45,301 TRACE (notification-thread-0,NodeB:) [ClusterTopologyManagerImpl] Initializing rebalance policy for cache ___defaultcache, pre-existing partitions are [CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null}, CacheTopology{id=4, currentCH=ReplicatedConsistentHash{members=[NodeB-43205, NodeC-45489]}, pendingCH=null}] 18:06:45,301 DEBUG (notification-thread-0,NodeB:) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null} 18:06:45,301 TRACE (asyncTransportThread-0,NodeB:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeB-43205, joinInfo=null, topologyId=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null, throwable=null, viewId=3}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:06:45,301 TRACE (asyncTransportThread-0,NodeB:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeB-43205, joinInfo=null, topologyId=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null, throwable=null, viewId=3} to addresses null with response mode GET_ALL 18:06:45,301 DEBUG (asyncTransportThread-1,NodeB:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null} 18:06:45,301 TRACE (asyncTransportThread-1,NodeB:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null} 18:06:45,302 TRACE (asyncTransportThread-1,NodeB:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]} 18:06:45,302 TRACE (asyncTransportThread-1,NodeB:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:45,302 DEBUG (asyncTransportThread-1,NodeB:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:45,302 TRACE (asyncTransportThread-1,NodeB:___defaultcache) [TransactionTable] Minimum view ID still is 2; nothing to change 18:06:45,295 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4} to single recipient NodeB-43205 with response mode GET_NONE 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [RpcManagerImpl] Response(s) to RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4} is {} 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Suppressing invocation of method handleRollbackCommand. 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [AbstractCacheTransaction] Clearing locked keys: null 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:) [AbstractEnlistmentAdapter] About to invoke tx completion notification on nodes null 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:) [RpcManagerImpl] NodeC-45489 broadcasting call TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=0, gtx=GlobalTransaction::245:local, cacheName=___defaultcache} to recipient list null 18:06:45,307 TRACE (testng-VersionedReplStateTransferTest:) [JGroupsTransport] dests=null, command=TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=4, gtx=GlobalTransaction::245:local, cacheName=___defaultcache} , mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=15000 18:06:45,309 TRACE (OOB-2,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 326 18:06:45,310 TRACE (OOB-2,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4} [sender=NodeC-45489] 18:06:45,310 TRACE (OOB-2,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=4} 18:06:45,310 TRACE (OOB-2,ISPN,NodeB-43205:) [AbstractTransactionBoundaryCommand] About to execute tx command RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=4} 18:06:45,310 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [InvocationContextInterceptor] Invoked with command RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=4} and InvocationContext [org.infinispan.context.impl.RemoteTxInvocationContext@cd9d6c1] 18:06:45,310 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [StateTransferInterceptor] handleTopologyAffectedCommand for command RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=4}, originLocal=false 18:06:45,290 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 323 18:06:45,313 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=4} [sender=NodeC-45489] 18:06:45,313 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=4} 18:06:45,313 TRACE (OOB-1,ISPN,NodeB-43205:) [AbstractTransactionBoundaryCommand] Did not find a RemoteTransaction for GlobalTransaction::245:remote 18:06:45,310 TRACE (testng-VersionedReplStateTransferTest:) [CommandAwareRpcDispatcher] Replication task sending TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=4, gtx=GlobalTransaction::245:local, cacheName=___defaultcache} to single recipient NodeB-43205 with response mode GET_NONE 18:06:45,314 TRACE (testng-VersionedReplStateTransferTest:) [CommandAwareRpcDispatcher] Response: null 18:06:45,314 TRACE (testng-VersionedReplStateTransferTest:) [RpcManagerImpl] Response(s) to TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=4, gtx=GlobalTransaction::245:local, cacheName=___defaultcache} is {} 18:06:45,314 TRACE (OOB-1,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 327 18:06:45,302 TRACE (asyncTransportThread-0,NodeB:) [CommandAwareRpcDispatcher] real_dests=[NodeC-45489] 18:06:45,314 TRACE (OOB-1,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute command: TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=4, gtx=GlobalTransaction::245:local, cacheName=___defaultcache} [sender=NodeC-45489] 18:06:45,328 TRACE (OOB-1,ISPN,NodeB-43205:) [InboundInvocationHandlerImpl] Calling perform() on TxCompletionNotificationCommand{ xid=null, internalId=0, topologyId=4, gtx=GlobalTransaction::245:local, cacheName=___defaultcache} 18:06:45,331 TRACE (OOB-1,ISPN,NodeB-43205:) [TxCompletionNotificationCommand] Processing completed transaction GlobalTransaction::245:local 18:06:45,332 TRACE (OOB-1,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 328 18:06:45,314 WARN (testng-VersionedReplStateTransferTest:) [CacheImpl] ISPN000160: Could not complete injected transaction. javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1009) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: org.infinispan.CacheException: Could not prepare. at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) ... 26 more Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) ... 31 more 18:06:45,332 TRACE (OOB-1,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeB-43205, joinInfo=null, topologyId=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null, throwable=null, viewId=3} [sender=NodeB-43205] 18:06:45,334 TRACE (testng-VersionedReplStateTransferTest:) [CacheImpl] Could not rollback java.lang.IllegalStateException: BaseTransaction.rollback - ARJUNA016074: no transaction! at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.rollback(BaseTransaction.java:130) at org.infinispan.CacheImpl.tryRollback(CacheImpl.java:1022) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1012) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) 18:06:45,334 ERROR (testng-VersionedReplStateTransferTest:) [UnitTestTestNGListener] Test testStateTransfer(org.infinispan.container.versioning.VersionedReplStateTransferTest) failed. org.infinispan.CacheException: Could not commit implicit transaction at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1013) at org.infinispan.CacheImpl.put(CacheImpl.java:695) at org.infinispan.CacheImpl.put(CacheImpl.java:687) at org.infinispan.CacheSupport.put(CacheSupport.java:53) at org.infinispan.container.versioning.VersionedReplStateTransferTest.testStateTransfer(VersionedReplStateTransferTest.java:85) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:74) at org.testng.internal.Invoker.invokeMethod(Invoker.java:673) at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:846) at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1170) at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125) at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109) at org.testng.TestRunner.runWorkers(TestRunner.java:1147) at org.testng.TestRunner.privateRun(TestRunner.java:749) at org.testng.TestRunner.run(TestRunner.java:600) at org.testng.SuiteRunner.runTest(SuiteRunner.java:317) at org.testng.SuiteRunner.access$000(SuiteRunner.java:34) at org.testng.SuiteRunner$SuiteWorker.run(SuiteRunner.java:351) at org.testng.internal.thread.ThreadUtil$CountDownLatchedRunnable.run(ThreadUtil.java:147) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1177) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:117) at org.infinispan.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1009) ... 24 more Caused by: org.infinispan.CacheException: Could not prepare. at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:70) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:273) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:93) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:164) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1165) ... 26 more Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:161) at org.infinispan.transaction.TransactionCoordinator.prepare(TransactionCoordinator.java:123) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:68) ... 31 more 18:06:45,336 INFO (testng-VersionedReplStateTransferTest:) [UnitTestTestNGListener] Test suite progress: tests succeeded: 145, failed: 1, skipped: 0. 18:06:45,325 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Removed RemoteTransaction{modifications=[PutKeyValueCommand{key=hello, value=new world, flags=null, putIfAbsent=false, lifespanMillis=-1, maxIdleTimeMillis=-1}], lookedUpEntries={hello=ClusteredRepeatableReadEntry(5559277e){key=hello, value=new world, oldValue=world, isCreated=false, isChanged=true, isRemoved=false, isValid=true}}, lockedKeys= [hello], backupKeyLocks null, missingLookedUpEntries false, tx=GlobalTransaction::245:remote} from transaction table. 18:06:45,341 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [AbstractCacheTransaction] Transaction GlobalTransaction::245:remote has completed, notifying listening threads. 18:06:45,342 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [TransactionTable] Removed local transaction GlobalTransaction::245:remote? true 18:06:45,342 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [CallInterceptor] Suppressing invocation of method handleRollbackCommand. 18:06:45,407 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [LockManagerImpl] Attempting to unlock hello 18:06:45,407 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [AbstractCacheTransaction] Clearing locked keys: [hello] 18:06:45,407 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] CommandTopologyId=4, localTopologyId=6 18:06:45,408 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [StateTransferManagerImpl] Forwarding command RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=6} to new targets [NodeC-45489] 18:06:45,408 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [RpcManagerImpl] NodeB-43205 broadcasting call RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=6} to recipient list [NodeC-45489] 18:06:45,334 DEBUG (OOB-1,ISPN,NodeC-45489:___defaultcache) [LocalTopologyManagerImpl] Updating local consistent hash(es) for cache ___defaultcache: new topology = CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null} 18:06:45,446 TRACE (OOB-1,ISPN,NodeC-45489:___defaultcache) [StateTransferManagerImpl] Installing new cache topology CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]}, pendingCH=null} 18:06:45,446 TRACE (OOB-1,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Received new CH: ReplicatedConsistentHash{members=[NodeC-45489, NodeB-43205]} 18:06:45,447 TRACE (OOB-1,ISPN,NodeC-45489:___defaultcache) [StateConsumerImpl] Discarding removed segments: []; new segments: [0]; old segments: [0] 18:06:45,447 DEBUG (OOB-1,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Topology changed, recalculating minViewId 18:06:45,447 TRACE (OOB-1,ISPN,NodeC-45489:___defaultcache) [TransactionTable] Changing minimum view ID from 4 to 6 18:06:45,447 TRACE (OOB-1,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 328 to NodeB-43205 18:06:45,448 TRACE (asyncTransportThread-0,NodeB:) [CommandAwareRpcDispatcher] Responses: [sender=NodeC-45489, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:06:45,420 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command EntrySetCommand{set=1 elements} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@1c6ec155] 18:06:45,453 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: EntrySetCommand{set=1 elements}. 18:06:45,454 TRACE (testng-VersionedReplStateTransferTest:) [TestingUtil] Cache contents before stopping: [hello=world] 18:06:45,454 DEBUG (testng-VersionedReplStateTransferTest:) [CacheImpl] Stopping cache ___defaultcache on NodeB-43205 18:06:45,454 TRACE (testng-VersionedReplStateTransferTest:) [ComponentsJmxRegistration] Unregistering jmx resources.. 18:06:45,454 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:45,408 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [JGroupsTransport] dests=[NodeC-45489], command=RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=6}, mode=SYNCHRONOUS, timeout=15000 18:06:45,454 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:45,455 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Replication task sending RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=6} to single recipient NodeC-45489 with response mode GET_ALL 18:06:45,458 TRACE (OOB-1,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 332 18:06:45,458 TRACE (OOB-1,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute command: RollbackCommand {gtx=GlobalTransaction::245:local, cacheName='___defaultcache', topologyId=6} [sender=NodeB-43205] 18:06:45,458 TRACE (OOB-1,ISPN,NodeC-45489:) [InboundInvocationHandlerImpl] Calling perform() on RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=6} 18:06:45,455 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forget 18:06:45,449 TRACE (asyncTransportThread-3,NodeB:) [ClusterTopologyManagerImpl] Rebalancing consistent hash for cache ___defaultcache, members are [NodeC-45489, NodeB-43205] 18:06:45,467 TRACE (asyncTransportThread-3,NodeB:) [ClusterTopologyManagerImpl] The balanced CH is the same as the current CH, not rebalancing 18:06:45,466 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String showInDoubtTransactions 18:06:45,466 TRACE (OOB-1,ISPN,NodeC-45489:) [AbstractTransactionBoundaryCommand] Did not find a RemoteTransaction for GlobalTransaction::245:remote 18:06:45,491 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceRollback 18:06:45,491 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceCommit 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forget 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute prepares [r=true,w=false,is=false,type=long] 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute rollbacks [r=true,w=false,is=false,type=long] 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute commits [r=true,w=false,is=false,type=long] 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void resetStatistics 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute joinComplete [r=true,w=false,is=true,type=boolean] 18:06:45,510 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute stateTransferInProgress [r=true,w=false,is=true,type=boolean] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void recordKnownGlobalKeyset 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute concurrencyLevel [r=true,w=false,is=false,type=int] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute numberOfLocksAvailable [r=true,w=false,is=false,type=int] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute numberOfLocksHeld [r=true,w=false,is=false,type=int] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute committedViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute replicationCount [r=true,w=false,is=false,type=long] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute successRatioFloatingPoint [r=true,w=false,is=false,type=double] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute replicationFailures [r=true,w=false,is=false,type=long] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute successRatio [r=true,w=false,is=false,type=java.lang.String] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute averageReplicationTime [r=true,w=false,is=false,type=long] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute pendingViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void resetStatistics 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RecoveryAdmin 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Transactions 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=StateTransferManager 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RollingUpgradeManager 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=LockManager 18:06:45,511 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RpcManager 18:06:45,491 TRACE (OOB-1,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 332 to NodeB-43205 18:06:45,514 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [CommandAwareRpcDispatcher] Response: null 18:06:45,514 TRACE (OOB-2,ISPN,NodeB-43205:___defaultcache) [RpcManagerImpl] Response(s) to RollbackCommand {gtx=GlobalTransaction::245:remote, cacheName='___defaultcache', topologyId=6} is {NodeC-45489=null} 18:06:45,516 DEBUG (testng-VersionedReplStateTransferTest:) [TransactionTable] Wait for on-going transactions to finish for 30 seconds. 18:07:15,531 WARN (testng-VersionedReplStateTransferTest:) [TransactionTable] ISPN000100: Stopping, but there are 1 local transactions and 0 remote transactions that did not finish in time. 18:07:15,532 TRACE (testng-VersionedReplStateTransferTest:) [StateConsumerImpl] Shutting down StateConsumer of cache ___defaultcache on node NodeB-43205 18:07:15,532 TRACE (testng-VersionedReplStateTransferTest:) [StateProviderImpl] Shutting down StateProvider of cache ___defaultcache on node NodeB-43205 18:07:15,532 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferManagerImpl] Shutting down StateTransferManager of cache ___defaultcache on node NodeB-43205 18:07:15,532 DEBUG (testng-VersionedReplStateTransferTest:) [LocalTopologyManagerImpl] Node NodeB-43205 leaving cache ___defaultcache 18:07:15,532 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [InvocationContextInterceptor] Invoked with command EntrySetCommand{set=1 elements} and InvocationContext [org.infinispan.context.impl.NonTxInvocationContext@4226758d] 18:07:15,532 TRACE (testng-VersionedReplStateTransferTest:___defaultcache) [CallInterceptor] Executing command: EntrySetCommand{set=1 elements}. 18:07:15,532 TRACE (testng-VersionedReplStateTransferTest:) [TestingUtil] Cache contents before stopping: [hello=world] 18:07:15,532 DEBUG (testng-VersionedReplStateTransferTest:) [CacheImpl] Stopping cache ___defaultcache on NodeC-45489 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ComponentsJmxRegistration] Unregistering jmx resources.. 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String showInDoubtTransactions 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceCommit 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceCommit 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forget 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceRollback 18:07:15,533 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forget 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation java.lang.String forceRollback 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute joinComplete [r=true,w=false,is=true,type=boolean] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute stateTransferInProgress [r=true,w=false,is=true,type=boolean] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute pendingViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute replicationFailures [r=true,w=false,is=false,type=long] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute averageReplicationTime [r=true,w=false,is=false,type=long] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute committedViewAsString [r=true,w=false,is=false,type=java.lang.String] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute successRatioFloatingPoint [r=true,w=false,is=false,type=double] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute replicationCount [r=true,w=false,is=false,type=long] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute successRatio [r=true,w=false,is=false,type=java.lang.String] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void resetStatistics 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute numberOfLocksHeld [r=true,w=false,is=false,type=int] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute numberOfLocksAvailable [r=true,w=false,is=false,type=int] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute concurrencyLevel [r=true,w=false,is=false,type=int] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute prepares [r=true,w=false,is=false,type=long] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute rollbacks [r=true,w=false,is=false,type=long] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute statisticsEnabled [r=true,w=true,is=false,type=boolean] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute commits [r=true,w=false,is=false,type=long] 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void resetStatistics 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void recordKnownGlobalKeyset 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RecoveryAdmin 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=StateTransferManager 18:07:15,534 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RpcManager 18:07:15,535 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=LockManager 18:07:15,535 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=Transactions 18:07:15,535 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=Cache,name="___defaultcache(repl_sync)",manager="DefaultCacheManager",component=RollingUpgradeManager 18:07:15,533 TRACE (asyncTransportThread-2,NodeB:___defaultcache) [ClusterTopologyManagerImpl] Cache ___defaultcache topology updated: CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489]}, pendingCH=null} 18:07:15,552 DEBUG (testng-VersionedReplStateTransferTest:) [TransactionTable] Wait for on-going transactions to finish for 30 seconds. 18:07:15,552 DEBUG (testng-VersionedReplStateTransferTest:) [TransactionTable] All transactions terminated 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [StateConsumerImpl] Shutting down StateConsumer of cache ___defaultcache on node NodeC-45489 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [StateProviderImpl] Shutting down StateProvider of cache ___defaultcache on node NodeC-45489 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [StateTransferManagerImpl] Shutting down StateTransferManager of cache ___defaultcache on node NodeC-45489 18:07:15,552 DEBUG (testng-VersionedReplStateTransferTest:) [LocalTopologyManagerImpl] Node NodeC-45489 leaving cache ___defaultcache 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [JGroupsTransport] dests=[NodeB-43205], command=CacheTopologyControlCommand{cache=___defaultcache, type=LEAVE, sender=NodeC-45489, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=3}, mode=ASYNCHRONOUS_WITH_SYNC_MARSHALLING, timeout=0 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=LEAVE, sender=NodeC-45489, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=3} to single recipient NodeB-43205 with response mode GET_NONE 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [CommandAwareRpcDispatcher] Response: null 18:07:15,552 TRACE (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Ignore call to stop as the cache manager is stopping 18:07:15,552 DEBUG (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Stopping cache manager ISPN on NodeB-43205 18:07:15,553 DEBUG (testng-VersionedReplStateTransferTest:) [CacheImpl] Stopping cache ___defaultcache on NodeB-43205 18:07:15,553 DEBUG (testng-VersionedReplStateTransferTest:) [ComponentRegistry] Ignoring call to stop() as current state is org.infinispan.factories.ComponentRegistry@6397a190 18:07:15,553 TRACE (testng-VersionedReplStateTransferTest:) [ComponentsJmxRegistration] Unregistering jmx resources.. 18:07:15,553 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterName [r=true,w=false,is=false,type=java.lang.String] 18:07:15,553 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute createdCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:07:15,553 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute cacheManagerStatus [r=true,w=false,is=false,type=java.lang.String] 18:07:15,553 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute runningCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterSize [r=true,w=false,is=false,type=int] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute nodeAddress [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterMembers [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheNames [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute name [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute version [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute physicalAddresses [r=true,w=false,is=false,type=java.lang.String] 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:07:15,554 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:07:15,555 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan77:type=CacheManager,name="DefaultCacheManager",component=CacheManager 18:07:15,555 INFO (testng-VersionedReplStateTransferTest:) [JGroupsTransport] ISPN000080: Disconnecting and closing JGroups Channel 18:07:15,555 TRACE (OOB-6,ISPN,NodeB-43205:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 2641 18:07:15,551 DEBUG (asyncTransportThread-2,NodeB:___defaultcache) [ClusterTopologyManagerImpl] Updating cluster-wide consistent hash for cache ___defaultcache, topology = CacheTopology{id=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489]}, pendingCH=null} 18:07:15,555 TRACE (OOB-6,ISPN,NodeB-43205:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=LEAVE, sender=NodeC-45489, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=3} [sender=NodeC-45489] 18:07:15,555 TRACE (asyncTransportThread-1,NodeB:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeB-43205, joinInfo=null, topologyId=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489]}, pendingCH=null, throwable=null, viewId=3}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:07:15,556 TRACE (asyncTransportThread-1,NodeB:) [CommandAwareRpcDispatcher] Replication task sending CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeB-43205, joinInfo=null, topologyId=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489]}, pendingCH=null, throwable=null, viewId=3} to addresses null with response mode GET_ALL 18:07:15,556 TRACE (asyncTransportThread-1,NodeB:) [CommandAwareRpcDispatcher] real_dests=[NodeC-45489] 18:07:15,557 TRACE (OOB-4,ISPN,NodeC-45489:) [RequestCorrelator] calling (org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher) with request 2642 18:07:15,557 TRACE (OOB-4,ISPN,NodeC-45489:) [CommandAwareRpcDispatcher] Attempting to execute non-CacheRpcCommand command: CacheTopologyControlCommand{cache=___defaultcache, type=CH_UPDATE, sender=NodeB-43205, joinInfo=null, topologyId=6, currentCH=ReplicatedConsistentHash{members=[NodeC-45489]}, pendingCH=null, throwable=null, viewId=3} [sender=NodeB-43205] 18:07:15,557 TRACE (OOB-4,ISPN,NodeC-45489:___defaultcache) [LocalTopologyManagerImpl] Ignoring consistent hash update 6 for cache ___defaultcache that doesn't exist locally 18:07:15,557 TRACE (OOB-4,ISPN,NodeC-45489:) [RequestCorrelator] sending rsp for 2642 to NodeB-43205 18:07:15,562 TRACE (asyncTransportThread-1,NodeB:) [CommandAwareRpcDispatcher] Responses: [sender=NodeC-45489, retval=SuccessfulResponse{responseValue=null} , received=true, suspected=false] 18:07:15,634 DEBUG (Incoming-1,ISPN,NodeC-45489:) [GMS] NodeC-45489: installing view [NodeC-45489|4] [NodeC-45489] 18:07:15,634 DEBUG (Incoming-1,ISPN,NodeC-45489:) [STABLE] resuming message garbage collection 18:07:15,634 DEBUG (Incoming-1,ISPN,NodeC-45489:) [NAKACK2] removed NodeB-43205 from xmit_table (not member anymore) 18:07:15,634 DEBUG (Incoming-1,ISPN,NodeC-45489:) [STABLE] [ergonomics] setting max_bytes to 400KB (1 members) 18:07:15,642 DEBUG (Incoming-1,ISPN,NodeC-45489:) [JGroupsTransport] New view accepted: [NodeC-45489|4] [NodeC-45489] 18:07:15,642 INFO (Incoming-1,ISPN,NodeC-45489:) [JGroupsTransport] ISPN000094: Received new cluster view: [NodeC-45489|4] [NodeC-45489] 18:07:15,642 TRACE (Incoming-1,ISPN,NodeC-45489:) [ClusterIdGenerator] Calculated rank based on view EventImpl{type=VIEW_CHANGED, newMembers=[NodeC-45489], oldMembers=[NodeB-43205, NodeC-45489], localAddress=NodeC-45489, viewId=4, subgroupsMerged=null, mergeView=false} and result was 1125904201809920 18:07:15,643 TRACE (notification-thread-0,NodeC:) [ClusterTopologyManagerImpl] Received new cluster view: 4 18:07:15,643 DEBUG (notification-thread-0,NodeC:) [ClusterTopologyManagerImpl] Recovering running caches in the cluster 18:07:15,659 DEBUG (testng-VersionedReplStateTransferTest:) [TCPConnectionMap] connection table is not running, discarding message to 127.0.0.1:9502 18:07:15,659 DEBUG (testng-VersionedReplStateTransferTest:) [TEST_PING] Stop discovery for NodeB-43205 18:07:15,659 DEBUG (testng-VersionedReplStateTransferTest:) [TCP] closing sockets and stopping threads 18:07:15,653 TRACE (asyncTransportThread-0,NodeC:) [JGroupsTransport] dests=null, command=CacheTopologyControlCommand{cache=null, type=GET_STATUS, sender=NodeC-45489, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=3}, mode=SYNCHRONOUS_IGNORE_LEAVERS, timeout=240000 18:07:18,663 INFO (testng-VersionedReplStateTransferTest:) [JGroupsTransport] ISPN000082: Stopping the RpcDispatcher 18:07:18,672 DEBUG (testng-VersionedReplStateTransferTest:) [DefaultCacheManager] Stopping cache manager ISPN on NodeC-45489 18:07:18,672 TRACE (asyncTransportThread-2,NodeB:___defaultcache) [CacheTopologyControlCommand] Command execution CacheTopologyControlCommand{cache=___defaultcache, type=LEAVE, sender=NodeB-43205, joinInfo=null, topologyId=0, currentCH=null, pendingCH=null, throwable=null, viewId=3} was interrupted because the cache manager is shutting down 18:07:18,672 DEBUG (asyncTransportThread-0,NodeB:) [AbstractJBossMarshaller] Using JBoss Marshalling 18:07:18,672 DEBUG (asyncTransportThread-0,NodeB:___defaultcache) [LocalTopologyManagerImpl] Ignoring consistent hash update 6 for cache ___defaultcache, the local cache manager is not running 18:07:18,674 DEBUG (testng-VersionedReplStateTransferTest:) [CacheImpl] Stopping cache ___defaultcache on NodeC-45489 18:07:18,674 DEBUG (testng-VersionedReplStateTransferTest:) [ComponentRegistry] Ignoring call to stop() as current state is org.infinispan.factories.ComponentRegistry@4a203af0 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ComponentsJmxRegistration] Unregistering jmx resources.. 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheNames [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute createdCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute runningCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute version [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute definedCacheCount [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterSize [r=true,w=false,is=false,type=int] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute cacheManagerStatus [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute name [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterMembers [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute physicalAddresses [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute clusterName [r=true,w=false,is=false,type=java.lang.String] 18:07:18,674 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Attribute nodeAddress [r=true,w=false,is=false,type=java.lang.String] 18:07:18,687 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:07:18,687 TRACE (testng-VersionedReplStateTransferTest:) [ResourceDMBean] Operation void startCache 18:07:18,687 TRACE (testng-VersionedReplStateTransferTest:) [JmxUtil] Unregistered infinispan99:type=CacheManager,name="DefaultCacheManager",component=CacheManager 18:07:18,687 INFO (testng-VersionedReplStateTransferTest:) [JGroupsTransport] ISPN000080: Disconnecting and closing JGroups Channel 18:07:18,719 DEBUG (testng-VersionedReplStateTransferTest:) [TEST_PING] Stop discovery for NodeC-45489 18:07:18,727 DEBUG (testng-VersionedReplStateTransferTest:) [TCP] closing sockets and stopping threads 18:07:18,728 INFO (testng-VersionedReplStateTransferTest:) [JGroupsTransport] ISPN000082: Stopping the RpcDispatcher 18:07:43,669 TRACE (Transaction Reaper Worker 0:) [TransactionCoordinator] rollback transaction GlobalTransaction::197:local 18:07:43,679 TRACE (Transaction Reaper Worker 0:) [AbstractCacheTransaction] Transaction GlobalTransaction::197:local has completed, notifying listening threads.