Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-9123

Very slow server shutdown with active MDB

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • None
    • 7.1.0.DR12, 7.1.0.DR13, 7.1.0.DR17, 7.1.0.ER2
    • ActiveMQ
    • None
    • Hide

      Steps to reproduce:

      git clone git://git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git
cd eap-tests-hornetq/scripts/
git checkout master

groovy -DEAP_VERSION=7.1.0.ER2 PrepareServers7.groovy
export WORKSPACE=$PWD
export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap
export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap
export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap
export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap

cd ../jboss-hornetq-testsuite/
mvn clean test -Dtest=RemoteJcaTestCase#testShutdownOfMdbWhenJmsServerIsDown -DfailIfNoTests=false -Deap=7x  | tee log
      Show
      Steps to reproduce: git clone git: //git.app.eng.bos.redhat.com/jbossqe/eap-tests-hornetq.git cd eap-tests-hornetq/scripts/ git checkout master groovy -DEAP_VERSION=7.1.0.ER2 PrepareServers7.groovy export WORKSPACE=$PWD export JBOSS_HOME_1=$WORKSPACE/server1/jboss-eap export JBOSS_HOME_2=$WORKSPACE/server2/jboss-eap export JBOSS_HOME_3=$WORKSPACE/server3/jboss-eap export JBOSS_HOME_4=$WORKSPACE/server4/jboss-eap cd ../jboss-hornetq-testsuite/ mvn clean test -Dtest=RemoteJcaTestCase#testShutdownOfMdbWhenJmsServerIsDown -DfailIfNoTests= false -Deap=7x | tee log

      If server with active MDB is cleanly shutdown then it takes very long minutes. Thread dump shows that there are threads waiting on:

      "ServerService Thread Pool -- 81" #222 prio=5 os_prio=0 tid=0x00007fc5f40e2800 nid=0x1fbd waiting on condition [0x00007fc64ce2f000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000000bf536478> (a java.util.concurrent.CountDownLatch$Sync)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
        at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
        at org.apache.activemq.artemis.utils.FutureLatch.await(FutureLatch.java:38)
        at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation.teardown(ActiveMQActivation.java:387)
        - locked <0x00000000bce42678> (a org.apache.activemq.artemis.ra.inflow.ActiveMQActivation)
        at org.apache.activemq.artemis.ra.inflow.ActiveMQActivation.stop(ActiveMQActivation.java:291)
        at org.apache.activemq.artemis.ra.ActiveMQResourceAdapter.endpointDeactivation(ActiveMQResourceAdapter.java:196)
        at org.jboss.jca.core.rar.EndpointImpl.deactivate(EndpointImpl.java:255)
        at org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponent.deactivate(MessageDrivenComponent.java:282)
        at org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponent.done(MessageDrivenComponent.java:252)
        - locked <0x00000000bcde4598> (a org.jboss.as.ejb3.component.messagedriven.MessageDrivenComponent)
        at org.jboss.as.ejb3.component.EJBComponent.stop(EJBComponent.java:600)
        at org.jboss.as.ee.component.ComponentStartService$2.run(ComponentStartService.java:78)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
        at org.jboss.threads.JBossThread.run(JBossThread.java:320)

      Customer impact: Long shutdowns will lead to longer maintenance updates when server must be restarted. This will affect customer's SLA.

        1. thread-dump5.txt
          72 kB
        2. thread-dump4.txt
          79 kB
        3. thread-dump3.txt
          80 kB
        4. thread-dump2.txt
          80 kB
        5. thread-dump1.txt
          80 kB
        6. thread-dump.txt
          153 kB
        7. mdb-node-4-thread-dump.txt
          109 kB

              mtaylor1@redhat.com Martyn Taylor (Inactive)
              mnovak1@redhat.com Miroslav Novak
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: