Uploaded image for project: 'JBoss Enterprise Application Platform'
  1. JBoss Enterprise Application Platform
  2. JBEAP-19826

[GSS](7.3.z) Exception while exporting metrics during WildFly initialization

XMLWordPrintable

    • +
    • Hide

      Deploy WFLY-13188-call-metrics-http.jar and WFLY-13188-stall.jar and start EAP

      1. Enable MP Metrics subsystem
      2. Start WildFly
      3. While WildFly is still in the startup process, call /metrics endpoint
      Show
      Deploy WFLY-13188 -call-metrics-http.jar and WFLY-13188 -stall.jar and start EAP Enable MP Metrics subsystem Start WildFly While WildFly is still in the startup process, call /metrics endpoint

      When the open tracing system is enabled and some metrics are exposed, if when the WildFly is in the startup process, the Prometheus server calls the /metrics endpoint, WildFly prints a stacktrace for each metrics that is has detected (i.e. more than thousands) with the following message:

      *WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available
      *

      In some cases the error persists, even after the server startup is done and we need to restart the WildFly; but we couldn't find the exact scenario for that.

      Here is an example for wildfly_jpa_second_level_cache_size_in_memory metric:

      2020-03-02 09:38:06,739 WARN                          [management I/O-2] [metrics] Unable to export metric wildfly_jpa_second_level_cache_size_in_memory: java.lang.IllegalStateException: WFLYMETRICS0003: Unable to read attribute second-level-cache-size-in-memory on [
    ("deployment" => "application.ear"),
    ("subdeployment" => "application-persistence.jar"),
    ("subsystem" => "jpa"),
    ("hibernate-persistence-unit" => "application.ear/application-persistence.jar#app"),
    ("entity-cache" => "com.app.persistence.entity.Country")
]: "WFLYCTL0379: System boot is in process; execution of remote management operations is not currently available".
        at org.wildfly.extension.microprofile.metrics.MetricCollector.readAttributeValue(MetricCollector.java:303)
        at org.wildfly.extension.microprofile.metrics.MetricCollector.access(MetricCollector.java:70)
        at org.wildfly.extension.microprofile.metrics.MetricCollector.getValue(MetricCollector.java:174)
        at org.wildfly.extension.microprofile.metrics.MetricCollector.getValue(MetricCollector.java:171)
        at io.smallrye.metrics.exporters.OpenMetricsExporter.createSimpleValueLine(OpenMetricsExporter.java:445)
        at io.smallrye.metrics.exporters.OpenMetricsExporter.exposeEntries(OpenMetricsExporter.java:178)
        at io.smallrye.metrics.exporters.OpenMetricsExporter.getEntriesForScope(OpenMetricsExporter.java:150)
        at io.smallrye.metrics.exporters.OpenMetricsExporter.exportAllScopes(OpenMetricsExporter.java:101)
        at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:116)
        at io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:73)
        at org.wildfly.extension.microprofile.metrics.MetricsContextService.handleRequest(MetricsContextService.java:81)
        at org.jboss.as.domain.http.server.security.RealmReadinessHandler.handleRequest(RealmReadinessHandler.java:51)
        at org.jboss.as.domain.http.server.security.ServerErrorReadinessHandler.handleRequest(ServerErrorReadinessHandler.java:35)
        at io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:91)
        at io.undertow.server.handlers.ChannelUpgradeHandler.handleRequest(ChannelUpgradeHandler.java:211)
        at io.undertow.server.handlers.cache.CacheHandler.handleRequest(CacheHandler.java:92)
        at io.undertow.server.handlers.error.SimpleErrorPageHandler.handleRequest(SimpleErrorPageHandler.java:78)
        at io.undertow.server.handlers.CanonicalPathHandler.handleRequest(CanonicalPathHandler.java:49)
        at org.jboss.as.domain.http.server.ManagementHttpRequestHandler.handleRequest(ManagementHttpRequestHandler.java:57)
        at org.jboss.as.domain.http.server.cors.CorsHttpHandler.handleRequest(CorsHttpHandler.java:75)
        at org.jboss.as.domain.http.server.ManagementHttpServer$UpgradeFixHandler.handleRequest(ManagementHttpServer.java:666)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:376)
        at io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
        at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
        at io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:59)
        at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
        at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
        at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
        at org.xnio.nio.WorkerThread.run(WorkerThread.java:591)

        1. WFLY-13188-call-metrics-http.jar
          10 kB
        2. WFLY-13188-stall.jar
          2 kB

              bstansbe@redhat.com Brian Stansberry
              rhn-support-bmaxwell Brad Maxwell
              Votes:
              4 Vote for this issue
              Watchers:
              13 Start watching this issue

                Created:
                Updated:
                Resolved: