Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-11603

Intermittent NPE from MP config SubsystemDeploymentProcessor

    XMLWordPrintable

Details

    • Hide

      As said in description, we don't have any reliable reproducer yet. The NPE is thrown during the server startup after the reload operation. It goes like this in our tests:

      1. EAP is started (standalone.sh, standalone-full.xml config)
      2. simple deployment war is deployed on server via CLI deploy command
      3. simple config change is done, like:
        • ./subsystem=webservices:write-attribute(name=wsdl-port,value=8084)
      4. reload
      5. and now, during the server startup the NPE is thrown in the server.log
      Show
      As said in description, we don't have any reliable reproducer yet. The NPE is thrown during the server startup after the reload operation. It goes like this in our tests: EAP is started (standalone.sh, standalone-full.xml config) simple deployment war is deployed on server via CLI deploy command simple config change is done, like: ./subsystem=webservices:write-attribute(name=wsdl-port,value=8084) reload and now, during the server startup the NPE is thrown in the server.log

    Description

      In our automation tests, we can see intermittent NPE with following stack trace:

      20:01:38,985 ERROR [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0043: Deployment unit processor org.wildfly.extension.microprofile.config.smallrye.deployment.SubsystemDeploymentProcessor@54fd99c3 unexpectedly threw an exception during undeploy phase POST_MODULE of deployment "org.jboss.qa.management.ws.WebservicesWsdlHostIT.war": java.lang.NullPointerException
	at org.wildfly.extension.microprofile.config-smallrye@7.2.0.GA-redhat-00004//org.wildfly.extension.microprofile.config.smallrye.deployment.SubsystemDeploymentProcessor.undeploy(SubsystemDeploymentProcessor.java:102)
	at org.jboss.as.server@6.0.11.Final-redhat-00001//org.jboss.as.server.deployment.DeploymentUnitPhaseService.safeUndeploy(DeploymentUnitPhaseService.java:211)
	at org.jboss.as.server@6.0.11.Final-redhat-00001//org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:149)
	at org.jboss.msc@1.4.5.Final-redhat-00001//org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1738)
	at org.jboss.msc@1.4.5.Final-redhat-00001//org.jboss.msc.service.ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1700)
	at org.jboss.msc@1.4.5.Final-redhat-00001//org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1558)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1364)
	at java.base/java.lang.Thread.run(Thread.java:834)

20:01:38,986 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-1) MSC000001: Failed to start service jboss.deployment.unit."org.jboss.qa.management.ws.WebservicesWsdlHostIT.war".POST_MODULE: org.jboss.msc.service.StartException in service jboss.deployment.unit."org.jboss.qa.management.ws.WebservicesWsdlHostIT.war".POST_MODULE: WFLYSRV0153: Failed to process phase POST_MODULE of deployment "org.jboss.qa.management.ws.WebservicesWsdlHostIT.war"
	at org.jboss.as.server@6.0.11.Final-redhat-00001//org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:151)
	at org.jboss.msc@1.4.5.Final-redhat-00001//org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1738)
	at org.jboss.msc@1.4.5.Final-redhat-00001//org.jboss.msc.service.ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1700)
	at org.jboss.msc@1.4.5.Final-redhat-00001//org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1558)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487)
	at org.jboss.threads@2.3.2.Final-redhat-1//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1364)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.IllegalStateException: No ConfigProviderResolver implementation found!
	at org.eclipse.microprofile.config.api@1.3.0.redhat-00001//org.eclipse.microprofile.config.spi.ConfigProviderResolver.instance(ConfigProviderResolver.java:122)
	at org.wildfly.extension.microprofile.config-smallrye@7.2.0.GA-redhat-00004//org.wildfly.extension.microprofile.config.smallrye.deployment.SubsystemDeploymentProcessor.deploy(SubsystemDeploymentProcessor.java:67)
	at org.jboss.as.server@6.0.11.Final-redhat-00001//org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:144)
	... 8 more

      further on, there is also following error in the server.log (it has been dealt in WFWIP-64, although in different circumstances, I guess):

      20:01:39,164 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "org.jboss.qa.management.ws.WebservicesWsdlHostIT.war")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"org.jboss.qa.management.ws.WebservicesWsdlHostIT.war\".POST_MODULE" => "WFLYSRV0153: Failed to process phase POST_MODULE of deployment \"org.jboss.qa.management.ws.WebservicesWsdlHostIT.war\"
    Caused by: java.lang.IllegalStateException: No ConfigProviderResolver implementation found!"}}
20:01:39,169 INFO  [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0010: Deployed "org.jboss.qa.management.ws.WebservicesWsdlHostIT.war" (runtime-name : "org.jboss.qa.management.ws.WebservicesWsdlHostIT.war")
20:01:39,171 INFO  [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0183: Service status report
WFLYCTL0186:   Services which failed to start:      service jboss.deployment.unit."org.jboss.qa.management.ws.WebservicesWsdlHostIT.war".POST_MODULE: WFLYSRV0153: Failed to process phase POST_MODULE of deployment "org.jboss.qa.management.ws.WebservicesWsdlHostIT.war"

      The NPE itself goes from the 'undeploy' method in SubsystemDeploymentProcessor. Although, according to the surrounding log messages (longer test log is attached [^NPE-server.log]), the server is still in the boot process by that time.

      This NPE happens quite rarely so it looks like some kind of a race-condition is in place. So far we have seen this only on Windows with JDK-11 combination but cannot say it does not affect other platforms for sure.

      Also, we have not seen it with latest WildFly nightly builds too (WildFly revision 353b95ef201c18dbd465087f520392c8525a2213) (yet?).

      Attachments

        Issue Links

          clones

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-16197 Intermittent NPE from MP config SubsystemDeploymentProcessor

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Bug - A problem that impairs or prevents the functions of the product. WFCORE-4357 Support new MSC Service API for deployment dependency

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved

          Activity

            People

              jmesnil1@redhat.com Jeff Mesnil
              jmesnil1@redhat.com Jeff Mesnil
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: