Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-20552

Unable to start a pod with AB_PROMETHEUS_ENABLE=true env var

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 7.4.0.CD21
    • 7.4.0.CD21
    • OpenShift
    • None

      Test using prometheus is failing. After investigation it seems problem is Deployment config AB_PROMETHEUS_ENABLE=true which leads to pod failure

      java.lang.RuntimeException: WFLYCTL0079: Failed initializing module org.jboss.as.logging
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.extension.ParallelExtensionAddHandler$1.execute(ParallelExtensionAddHandler.java:115)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:1005)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:744)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:468)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1415)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.ModelControllerImpl.boot(ModelControllerImpl.java:503)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:515)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.AbstractControllerService.boot(AbstractControllerService.java:477)
	at org.jboss.as.server@13.0.2.Final-redhat-00001//org.jboss.as.server.ServerService.boot(ServerService.java:451)
	at org.jboss.as.server@13.0.2.Final-redhat-00001//org.jboss.as.server.ServerService.boot(ServerService.java:404)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:416)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: WFLYLOG0078: The logging subsystem requires the log manager to be org.jboss.logmanager.LogManager. The subsystem has not be initialized and cannot be used. To use JBoss Log Manager you must add the system property "java.util.logging.manager" and set it to "org.jboss.logmanager.LogManager"
	at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
	at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.extension.ParallelExtensionAddHandler$1.execute(ParallelExtensionAddHandler.java:107)
	... 11 more
Caused by: java.lang.IllegalStateException: WFLYLOG0078: The logging subsystem requires the log manager to be org.jboss.logmanager.LogManager. The subsystem has not be initialized and cannot be used. To use JBoss Log Manager you must add the system property "java.util.logging.manager" and set it to "org.jboss.logmanager.LogManager"
	at org.jboss.as.logging@13.0.2.Final-redhat-00001//org.jboss.as.logging.LoggingExtension.initialize(LoggingExtension.java:195)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.extension.ExtensionAddHandler.initializeExtension(ExtensionAddHandler.java:131)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.extension.ExtensionAddHandler.initializeExtension(ExtensionAddHandler.java:103)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.extension.ParallelExtensionAddHandler$ExtensionInitializeTask.call(ParallelExtensionAddHandler.java:144)
	at org.jboss.as.controller@13.0.2.Final-redhat-00001//org.jboss.as.controller.extension.ParallelExtensionAddHandler$ExtensionInitializeTask.call(ParallelExtensionAddHandler.java:127)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.jboss.threads@2.4.0.Final-redhat-00001//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads@2.4.0.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
	at org.jboss.threads@2.4.0.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads@2.4.0.Final-redhat-00001//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.base/java.lang.Thread.run(Thread.java:834)
	at org.jboss.threads@2.4.0.Final-redhat-00001//org.jboss.threads.JBossThread.run(JBossThread.java:513)

      When I manually set deployment config environment to AB_PROMETHEUS_ENABLE=false pod start successfully. I have compared both pod logs (in attachment) and I see AB_PROMETHEUS_ENABLE=true add this JAVA_OPTS: -javaagent:/opt/jboss/container/prometheus/jmx_prometheus_javaagent.jar=9799:/opt/jboss/container/prometheus/etc/jmx-exporter-config.yaml compared to AB_PROMETHEUS_ENABLE=false

      We have seen already that exception during investigation of https://issues.redhat.com/browse/CLOUD-3857 during CD21.
      I see there were some changes in CD21 regarding java agent opts handling [1].

      [1] standalone.conf of https://master-jenkins-csb-eap-qe.cloud.paas.psi.redhat.com/job/eap-7.x-openshift-image-diff/182/artifact/output-old-against-new/dist-diff2-output.html

        1. example-app-prometheus-off.log
          8 kB
        2. example-app-prometheus-on.log
          29 kB
        3. log(2)
          8 kB

            kwills@redhat.com Ken Wills
            mchoma@redhat.com Martin Choma
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: