Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-12374

shutdown command stops the Infinispan Server but leaves the JVM process running

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 11.0.4.Final, 12.0.0.Dev05
    • 11.0.3.Final, 12.0.0.Dev04
    • Server
    • None

    Description

      Simple steps to reproduce:

      [rhusar@ribera infinispan-server-11.0.3.Final]$ ./bin/server.sh
      ...
      [rhusar@ribera infinispan-server-11.0.3.Final]$ ./bin/cli.sh
      [disconnected]> connect --username=admin --password=secret
      [ribera-54749@cluster//containers/default]> shutdown server ribera-54749

      Ensuring it's shutdown:

      [ribera-54749@cluster//containers/default]> shutdown cluster
      Could not connect to server: java.net.ConnectException: Failed to connect to localhost/0:0:0:0:0:0:0:1:11222

      Check the log:

      19:36:45,922 INFO (pool-6-thread-1) [org.infinispan.SERVER] ISPN080002: Infinispan Server stopping
      19:36:46,050 INFO (pool-6-thread-1) [org.infinispan.manager.DefaultCacheManager] Stopping cache manager on ribera-54749
      19:36:46,115 INFO (pool-6-thread-1) [org.infinispan.CLUSTER] ISPN000080: Disconnecting JGroups channel cluster
      19:36:46,124 INFO (pool-6-thread-1) [org.infinispan.CONTAINER] ISPN000390: Persisted state, version=11.0.3.Final timestamp=2020-09-30T17:36:46.122Z
      19:36:46,131 INFO (pool-6-thread-1) [org.infinispan.SERVER] ISPN080003: Infinispan Server stopped

      Check ps:

      43335 ttys001    0:13.50 /Library/Java/JavaVirtualMachines/adoptopenjdk-8.jdk/Contents/Home/bin/java -server -Xms64m -Xmx512m -XX:MetaspaceSize=64M -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Dvisualvm.display.name=infinispan-server -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dinfinispan.server.home.path=/Users/rhusar/git/wildfly/testsuite/integration/clustering/target/infinispan-server-11.0.3.Final -classpath :/Users/rhusar/git/wildfly/testsuite/integration/clustering/target/infinispan-server-11.0.3.Final/boot/infinispan-server-runtime-11.0.3.Final-loader.jar org.infinispan.server.loader.Loader org.infinispan.server.Bootstrap
      

      check jstack:

      [rhusar@ribera infinispan-server-11.0.3.Final]$ jstack 43335
      2020-09-30 19:46:57
      Full thread dump OpenJDK 64-Bit Server VM (25.265-b01 mixed mode):
      
      "Attach Listener" #107 daemon prio=9 os_prio=31 tid=0x00007f984e988800 nid=0xbb07 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "DestroyJavaVM" #106 prio=5 os_prio=31 tid=0x00007f984e910000 nid=0x2903 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "pool-5-thread-1" #76 prio=5 os_prio=31 tid=0x00007f984cb10800 nid=0xc003 waiting on condition [0x0000700013dc0000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007bed3c5d0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      
      "RxCachedWorkerPoolEvictor-1" #15 daemon prio=5 os_prio=31 tid=0x00007f9845a36000 nid=0xa503 waiting on condition [0x000070001010c000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007a1820548> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      
      "RxSchedulerPurge-1" #14 daemon prio=5 os_prio=31 tid=0x00007f9845a35000 nid=0x5703 waiting on condition [0x0000700010009000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007a18c3f08> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      
      "Log4j2-TF-6-Scheduled-1" #12 daemon prio=5 os_prio=31 tid=0x00007f9849052000 nid=0xa803 waiting on condition [0x000070000ff06000]
         java.lang.Thread.State: TIMED_WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x00000007a103fb30> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
      	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
      	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      
      "Service Thread" #9 daemon prio=9 os_prio=31 tid=0x00007f984580d800 nid=0x4003 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C1 CompilerThread3" #8 daemon prio=9 os_prio=31 tid=0x00007f9844024000 nid=0x4203 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread2" #7 daemon prio=9 os_prio=31 tid=0x00007f984382b800 nid=0x4303 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread1" #6 daemon prio=9 os_prio=31 tid=0x00007f984382a800 nid=0x3d03 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "C2 CompilerThread0" #5 daemon prio=9 os_prio=31 tid=0x00007f9844842000 nid=0x3b03 waiting on condition [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Signal Dispatcher" #4 daemon prio=9 os_prio=31 tid=0x00007f984484e000 nid=0x4403 runnable [0x0000000000000000]
         java.lang.Thread.State: RUNNABLE
      
      "Finalizer" #3 daemon prio=8 os_prio=31 tid=0x00007f9843015000 nid=0x3403 in Object.wait() [0x000070000f5e8000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007a08693d8> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:144)
      	- locked <0x00000007a08693d8> (a java.lang.ref.ReferenceQueue$Lock)
      	at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:165)
      	at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:216)
      
      "Reference Handler" #2 daemon prio=10 os_prio=31 tid=0x00007f9843012000 nid=0x3303 in Object.wait() [0x000070000f4e5000]
         java.lang.Thread.State: WAITING (on object monitor)
      	at java.lang.Object.wait(Native Method)
      	- waiting on <0x00000007a08b7420> (a java.lang.ref.Reference$Lock)
      	at java.lang.Object.wait(Object.java:502)
      	at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
      	- locked <0x00000007a08b7420> (a java.lang.ref.Reference$Lock)
      	at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
      
      "VM Thread" os_prio=31 tid=0x00007f9844838000 nid=0x3103 runnable
      
      "GC task thread#0 (ParallelGC)" os_prio=31 tid=0x00007f984400f000 nid=0x1d07 runnable
      
      "GC task thread#1 (ParallelGC)" os_prio=31 tid=0x00007f9844809000 nid=0x1f03 runnable
      
      "GC task thread#2 (ParallelGC)" os_prio=31 tid=0x00007f9844809800 nid=0x5403 runnable
      
      "GC task thread#3 (ParallelGC)" os_prio=31 tid=0x00007f9845808800 nid=0x2b03 runnable
      
      "GC task thread#4 (ParallelGC)" os_prio=31 tid=0x00007f9845809000 nid=0x5103 runnable
      
      "GC task thread#5 (ParallelGC)" os_prio=31 tid=0x00007f984300b000 nid=0x2c03 runnable
      
      "GC task thread#6 (ParallelGC)" os_prio=31 tid=0x00007f984580a000 nid=0x4f03 runnable
      
      "GC task thread#7 (ParallelGC)" os_prio=31 tid=0x00007f9843809800 nid=0x2f03 runnable
      
      "VM Periodic Task Thread" os_prio=31 tid=0x00007f984580e800 nid=0x5603 waiting on condition
      
      JNI global references: 2019
      

      This logic is use by the forked test driver, if its for some reason meant to keep running, this becomes a driver issue.

      Attachments

        Issue Links

          Activity

            People

              ttarrant@redhat.com Tristan Tarrant
              rhn-engineering-rhusar Radoslav Husar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: