Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-14524

Error due to IllegalStateException while exporting metrics during server stop with standalone-microprofile.xml

    XMLWordPrintable

Details

    • Hide

      run a long loop making 10000 requests to /metrics endpoint.

      for i in {1..10000}
      do
         curl http://localhost:9990/metrics/
      done
      

      As long as the loop doesn't end, shut down a stable server for a few (3-4) times, I can see that error with WFLY master.

      Show
      run a long loop making 10000 requests to /metrics endpoint. for i in {1..10000} do curl http://localhost:9990/metrics/ done As long as the loop doesn't end, shut down a stable server for a few (3-4) times, I can see that error with WFLY master.
    • ---
    • ---

    Description

      Intermittent error due to IllegalStateException by hitting http://localhost:9990/metrics/ in stable server shutdown (started with standalone-microprofile.xml, with no deployment).

      17:47:40,087 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 23.0.0.Final-SNAPSHOT (WildFly Core 15.0.0.Beta1) started in 3237ms - Started 237 of 385 services (218 services are lazy, passive or on-demand)
      17:47:40,089 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management
      17:47:40,090 INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0051: Admin console listening on http://127.0.0.1:9990
      ^C17:47:44,343 INFO  [org.jboss.as.server] (Thread-1) WFLYSRV0272: Suspending server
      17:47:44,350 ERROR [io.undertow.request] (management I/O-1) UT005071: Undertow request failed HttpServerExchange{ GET /metrics/}: java.lang.IllegalStateException: No entry for wildfly_datasources_pool_xarecover_average_time found
      	at io.smallrye.metrics@3.0.1//io.smallrye.metrics.exporters.OpenMetricsExporter.exposeEntries(OpenMetricsExporter.java:173)
      	at io.smallrye.metrics@3.0.1//io.smallrye.metrics.exporters.OpenMetricsExporter.getEntriesForScope(OpenMetricsExporter.java:162)
      	at io.smallrye.metrics@3.0.1//io.smallrye.metrics.exporters.OpenMetricsExporter.exportAllScopes(OpenMetricsExporter.java:113)
      	at io.smallrye.metrics@3.0.1//io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:122)
      	at io.smallrye.metrics@3.0.1//io.smallrye.metrics.MetricsRequestHandler.handleRequest(MetricsRequestHandler.java:79)
      	at org.wildfly.extension.microprofile.metrics-smallrye@23.0.0.Final-SNAPSHOT//org.wildfly.extension.microprofile.metrics.MicroProfileMetricsContextService$1.handleRequest(MicroProfileMetricsContextService.java:76)
      	at org.wildfly.extension.metrics@23.0.0.Final-SNAPSHOT//org.wildfly.extension.metrics.MetricsContextService$2.handleRequest(MetricsContextService.java:92)
      	at org.jboss.as.domain-http-interface@15.0.0.Beta1//org.jboss.as.domain.http.server.security.RealmReadinessHandler.handleRequest(RealmReadinessHandler.java:51)
      	at org.jboss.as.domain-http-interface@15.0.0.Beta1//org.jboss.as.domain.http.server.security.ServerErrorReadinessHandler.handleRequest(ServerErrorReadinessHandler.java:35)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.handlers.PathHandler.handleRequest(PathHandler.java:104)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.handlers.ChannelUpgradeHandler.handleRequest(ChannelUpgradeHandler.java:211)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.handlers.cache.CacheHandler.handleRequest(CacheHandler.java:92)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.handlers.error.SimpleErrorPageHandler.handleRequest(SimpleErrorPageHandler.java:78)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.handlers.CanonicalPathHandler.handleRequest(CanonicalPathHandler.java:49)
      	at org.jboss.as.domain-http-interface@15.0.0.Beta1//org.jboss.as.domain.http.server.ManagementHttpRequestHandler.handleRequest(ManagementHttpRequestHandler.java:57)
      	at org.jboss.as.domain-http-interface@15.0.0.Beta1//org.jboss.as.domain.http.server.cors.CorsHttpHandler.handleRequest(CorsHttpHandler.java:75)
      	at org.jboss.as.domain-http-interface@15.0.0.Beta1//org.jboss.as.domain.http.server.ManagementHttpServer$UpgradeFixHandler.handleRequest(ManagementHttpServer.java:717)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.protocol.http.HttpReadListener.handleEventWithNoRunningRequest(HttpReadListener.java:255)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.protocol.http.HttpReadListener.handleEvent(HttpReadListener.java:136)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:162)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:100)
      	at io.undertow.core@2.2.4.Final//io.undertow.server.protocol.http.HttpOpenListener.handleEvent(HttpOpenListener.java:57)
      	at org.jboss.xnio@3.8.4.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      	at org.jboss.xnio@3.8.4.Final//org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:291)
      	at org.jboss.xnio@3.8.4.Final//org.xnio.ChannelListeners$10.handleEvent(ChannelListeners.java:286)
      	at org.jboss.xnio@3.8.4.Final//org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      	at org.jboss.xnio.nio@3.8.4.Final//org.xnio.nio.QueuedNioTcpServer2.acceptTask(QueuedNioTcpServer2.java:178)
      	at org.jboss.xnio.nio@3.8.4.Final//org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612)
      	at org.jboss.xnio.nio@3.8.4.Final//org.xnio.nio.WorkerThread.run(WorkerThread.java:479)
      
      17:47:44,351 INFO  [org.jboss.as.server] (Thread-1) WFLYSRV0220: Server shutdown has been requested via an OS signal
      17:47:44,365 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0008: Undertow HTTP listener default suspending
      17:47:44,365 INFO  [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS]
      17:47:44,366 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0008: Undertow HTTPS listener https suspending
      17:47:44,367 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 127.0.0.1:8080
      17:47:44,368 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-4) WFLYUT0007: Undertow HTTPS listener https stopped, was bound to 127.0.0.1:8443
      17:47:44,372 INFO  [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = h2
      17:47:44,371 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0019: Host default-host stopping
      17:47:44,375 INFO  [org.wildfly.extension.undertow] (MSC service thread 1-5) WFLYUT0004: Undertow 2.2.4.Final stopping
      17:47:44,387 INFO  [org.jboss.as] (MSC service thread 1-1) WFLYSRV0050: WildFly Full 23.0.0.Final-SNAPSHOT (WildFly Core 15.0.0.Beta1) stopped in 33ms
      

      I run a long loop making 10000 requests to /metrics endpoint.

      for i in {1..10000}
      do
         curl http://localhost:9990/metrics/
      done
      

      As long as the loop doesn't end, shut down a stable server for a few (3-4) times, I can see that error with WFLY master.

      Attachments

        Activity

          People

            bstansbe@redhat.com Brian Stansberry
            chaowan@redhat.com Chao Wang
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: