Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Critical
    • Resolution: Out of Date
    • Affects Version/s: 9.2.0.CR1
    • Fix Version/s: 11.0.0.Alpha2
    • Component/s: None
    • Labels:
      None

      Description

      This was observed during the execution of test HotRodRemoteStreamingIT, Line 499 on method stopServer().

      The surefire process waits forever for the two servers to stop.

      One of the servers is waiting for the initial state transfer to complete:

      "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fdaf48f69f0 nid=0x590a waiting on condition [0x00007fdae1a26000]
         java.lang.Thread.State: TIMED_WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00000000ee8cfc00> (a java.util.concurrent.CountDownLatch$Sync)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
              at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
              at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
              at org.infinispan.commons.util.SecurityActions$$Lambda$170/1509149284.run(Unknown Source)
              at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
              at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
              at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
              at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
              at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
              at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1225674479.run(Unknown Source)
              at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
              at org.infinispan.factories.SecurityActions$$Lambda$176/1784470267.run(Unknown Source)
              at org.infinispan.security.Security.doPrivileged(Security.java:76)
              at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
              at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
              at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
              at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
              at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
              at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
              at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
              at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
              at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
              at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
              at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
              at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
              at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
              at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/218224883.run(Unknown Source)
              at java.lang.Thread.run(Thread.java:748)
      

      The other server is running a series of operations after the server stops, such
      as EmbeddedCacheManager#undefineConfiguration, this can be seen on 3 threads:
      MSC service thread 1-1, MSC service thread 1-2 and MSC service thread 1-3

      "MSC service thread 1-1" #12 prio=5 os_prio=0 tid=0x00007fd4e86720a0 nid=0x56f1 waiting on condition [0x00007fd4bc8f2000]
         java.lang.Thread.State: WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00000000fb922a18> (a java.util.concurrent.CompletableFuture$Signaller)
              at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
              at java.util.concurrent.CompletableFuture$Signaller.block(CompletableFuture.java:1693)
              at java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3323)
              at java.util.concurrent.CompletableFuture.waitingGet(CompletableFuture.java:1729)
              at java.util.concurrent.CompletableFuture.join(CompletableFuture.java:1934)
              at org.infinispan.manager.DefaultCacheManager.undefineConfiguration(DefaultCacheManager.java:394)
              at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:25)
              at org.infinispan.security.actions.UndefineConfigurationAction.run(UndefineConfigurationAction.java:13)
              at org.infinispan.security.Security.doPrivileged(Security.java:76)
              at org.infinispan.registry.impl.SecurityActions.undefineConfiguration(SecurityActions.java:39)
              at org.infinispan.registry.impl.InternalCacheRegistryImpl.unregisterInternalCache(InternalCacheRegistryImpl.java:77)
              - locked <0x00000000ee2e1420> (a org.infinispan.registry.impl.InternalCacheRegistryImpl)
              at org.infinispan.server.hotrod.HotRodServer.stop(HotRodServer.java:486)
              at org.infinispan.server.endpoint.subsystem.ProtocolServerService.doStop(ProtocolServerService.java:216)
              at org.infinispan.server.endpoint.subsystem.ProtocolServerService.stop(ProtocolServerService.java:206)
              - locked <0x00000000edd0eae0> (a org.infinispan.server.endpoint.subsystem.ProtocolServerService)
              at org.jboss.msc.service.ServiceControllerImpl$StopTask.stopService(ServiceControllerImpl.java:2056)
              at org.jboss.msc.service.ServiceControllerImpl$StopTask.run(ServiceControllerImpl.java:2017)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      

      For some reason, also in the other server, the CounterModule lifecycle is reacting to a
      cache manager start event, and it's hanged as well:

      "Thread-48" #95 prio=5 os_prio=0 tid=0x00007fd4986a2090 nid=0x575b waiting on condition [0x00007fd48f2f4000]
         java.lang.Thread.State: TIMED_WAITING (parking)
              at sun.misc.Unsafe.park(Native Method)
              - parking to wait for  <0x00000000ee91b048> (a java.util.concurrent.CountDownLatch$Sync)
              at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
              at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
              at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
              at org.infinispan.statetransfer.StateTransferManagerImpl.waitForInitialStateTransferToComplete(StateTransferManagerImpl.java:231)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:498)
              at org.infinispan.commons.util.SecurityActions.lambda$invokeAccessibly$0(SecurityActions.java:79)
              at org.infinispan.commons.util.SecurityActions$$Lambda$170/1758042552.run(Unknown Source)
              at org.infinispan.commons.util.SecurityActions.doPrivileged(SecurityActions.java:71)
              at org.infinispan.commons.util.SecurityActions.invokeAccessibly(SecurityActions.java:76)
              at org.infinispan.commons.util.ReflectionUtil.invokeAccessibly(ReflectionUtil.java:185)
              at org.infinispan.factories.AbstractComponentRegistry$PrioritizedMethod.invoke(AbstractComponentRegistry.java:968)
              at org.infinispan.factories.AbstractComponentRegistry.lambda$invokePrioritizedMethods$6(AbstractComponentRegistry.java:703)
              at org.infinispan.factories.AbstractComponentRegistry$$Lambda$175/1042194083.run(Unknown Source)
              at org.infinispan.factories.SecurityActions.lambda$run$1(SecurityActions.java:72)
              at org.infinispan.factories.SecurityActions$$Lambda$176/2118421355.run(Unknown Source)
              at org.infinispan.security.Security.doPrivileged(Security.java:76)
              at org.infinispan.factories.SecurityActions.run(SecurityActions.java:71)
              at org.infinispan.factories.AbstractComponentRegistry.invokePrioritizedMethods(AbstractComponentRegistry.java:696)
              at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:689)
              at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:607)
              at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:228)
              at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:962)
              at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:411)
              at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:637)
              at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:582)
              at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:468)
              at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:454)
              at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:440)
              at org.infinispan.counter.impl.CounterModuleLifecycle.lambda$startCaches$0(CounterModuleLifecycle.java:132)
              at org.infinispan.counter.impl.CounterModuleLifecycle$$Lambda$194/1023465788.run(Unknown Source)
              at java.lang.Thread.run(Thread.java:748)
      

        Gliffy Diagrams

          Attachments

          1. s1
            95 kB
          2. s2
            102 kB
          3. test
            722 kB

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                gustavonalle Gustavo Fernandes
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: