Uploaded image for project: 'JBoss Enterprise Application Platform 4 and 5'
  1. JBoss Enterprise Application Platform 4 and 5
  2. JBPAPP-8124

Deployment service freezing in soak tests


    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Critical Critical
    • EAP No Release
    • EAP 6.0.0 DR 13, EAP 6.0.0 ER 2, EAP 6.0.0 ER 3_Beta1
    • System
    • None
    • NEW

      07:33:13,783 INFO [org.jboss.as.server.deployment] (MSC service thread 1-6) JBAS015876: Starting deployment of "deployment.ear5066419734"
      07:33:14,395 INFO [org.jboss.as.server] (management-handler-threads - 201) JBAS018559: Deployed "deployment.ear5066419734"
      07:33:14,436 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015877: Stopped deployment deployment.ear5066419734 in 22ms
      07:33:14,455 INFO [org.jboss.as.server] (management-handler-threads - 202) JBAS018558: Undeployed "deployment.ear5066419734"
      07:33:15,471 INFO [org.jboss.as.server.deployment] (MSC service thread 1-11) JBAS015876: Starting deployment of "deployment.ear287054427"
      07:33:16,157 INFO [org.jboss.as.server] (management-handler-threads - 201) JBAS018559: Deployed "deployment.ear287054427"
      07:33:16,197 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) JBAS015877: Stopped deployment deployment.ear287054427 in 21ms
      07:33:16,263 INFO [org.jboss.as.server] (management-handler-threads - 202) JBAS018558: Undeployed "deployment.ear287054427"
      07:33:17,278 INFO [org.jboss.as.server.deployment] (MSC service thread 1-10) JBAS015876: Starting deployment of "deployment.ear829511841624"
      07:33:17,876 INFO [org.jboss.as.server] (management-handler-threads - 201) JBAS018559: Deployed "deployment.ear829511841624"
      07:33:17,914 INFO [org.jboss.as.server.deployment] (MSC service thread 1-6) JBAS015877: Stopped deployment deployment.ear829511841624 in 20ms
      07:33:17,935 INFO [org.jboss.as.server] (management-handler-threads - 202) JBAS018558: Undeployed "deployment.ear829511841624"
      07:33:18,950 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) JBAS015876: Starting deployment of "deployment.ear498231454"

      ... and then nothing happened anymore.

      Deployment service freezes after multiple deployments and undeployments of the same file (the names in this log are different because it is always generated randomly). It happened on Sun JDK 6 (both 32bit and 64bit) and OpenJDK 6. Tried with different deployment files, from the most basic EJB3 jar to a 58 megabyte EAR containing 5 different applications.

      The server keeps running and doesn't complain about any memory issues, but trying to deploy anything will get you stuck. For example the 'deploy' operation in CLI will never return, only a "JBAS014900: Content added at location......." will appear in server.log (and not every time).

      https://hudson.qa.jboss.com/hudson/job/eap6-deployment-soak-test-memory-leaks-32bit/ - see the console logs of runs #13 to #19.
      This test runs a server and keeps deploying+undeploying an application while always waiting for a log in server.log that confirms successful deployment.

      For bigger files, this happens sooner - after about 12 deploy/undeploy operations, for smaller applications it happens after about 20.

      If you try to deploy a file by copying it into "deployments" directory, a .isdeploying marker will be created, and after a while this log appears in server.log:

      02:45:40,527 ERROR [org.jboss.as.server.deployment.scanner] (DeploymentScanner-threads - 1) JBAS015052: Did not receive a response to the deployment operation within the allowed timeout period [60 seconds]. Check the server configuration file and the server logs to find more about the status of the deployment.

        1. jaxrpc-samples-swa.war
          8 kB
        2. jbpapp8124-reproducer.zip
          1.20 MB
        3. threaddump
          37 kB

            jperkins-rhn James Perkins
            jmartisk@redhat.com Jan Martiska
            0 Vote for this issue
            4 Start watching this issue