Uploaded image for project: 'WildFly'
  1. WildFly
  2. WFLY-14656

Particular HTTP/2 Ajax requests to JSF resources yield "default task" threads stuck within AbstractFramedStreamSinkChannel.awaitWritable()

    XMLWordPrintable

Details

    • Hide

      1. Deploy the example project to a vanilla Wildfly server.
      2. Visit https://localhost:8443/wftest/index.xhtml using Firefox. Must be https so that HTTP/2 is used.
      3. Click "Auto inc".
      4. Watch the number of stuck threads increase: watch 'jstack <pid> | grep 'AbstractFramedStreamSinkChannel.awaitWritable' | wc -l'
      5. Repeat 2.-4. with Chrome, but number of stuck threads won't increase.

      JSF page

      index.xhtml
      <html xmlns="http://www.w3.org/1999/xhtml"
        xmlns:f="http://xmlns.jcp.org/jsf/core"
        xmlns:h="http://xmlns.jcp.org/jsf/html"
        xmlns:p="http://primefaces.org/ui">
      
      <h:head>
      </h:head>
      
      <h:body>
       <h:form>
        <p:commandButton value="Inc">
         <p:ajax listener="#{foo.inc}" update="@all"/>
        </p:commandButton>
      
        <p:commandButton value="Auto inc" onclick="inc()" style="margin-left: 0.5rem"/>
        <p:remoteCommand name="inc" actionListener="#{foo.inc}" update="@all" oncomplete="setTimeout(inc, 0);"/>
      
        <p>
         <h:outputText id="label" value="#{foo.value}"/>
        </p>
       </h:form>
      </h:body>
      </html>
      

      Backing bean

      Foo.java
      package com.example;
      
      import javax.faces.view.ViewScoped;
      import javax.inject.Named;
      import java.io.Serializable;
      
      
      @Named
      @ViewScoped
      public class Foo implements Serializable {
       private int value;
      
      
       public int getValue() {
        return value;
       }
      
      
       public void setValue(int value) {
        this.value=value;
       }
      
      
       public void inc() {
        this.value++;
       }
      }
      
      Show
      1. Deploy the example project to a vanilla Wildfly server. 2. Visit https://localhost:8443/wftest/index.xhtml using Firefox. Must be https so that HTTP/2 is used. 3. Click "Auto inc". 4. Watch the number of stuck threads increase: watch 'jstack <pid> | grep 'AbstractFramedStreamSinkChannel.awaitWritable' | wc -l' 5. Repeat 2.-4. with Chrome, but number of stuck threads won't increase. JSF page index.xhtml <html xmlns= "http://www.w3.org/1999/xhtml" xmlns:f = "http://xmlns.jcp.org/jsf/core" xmlns:h = "http://xmlns.jcp.org/jsf/html" xmlns:p = "http://primefaces.org/ui" > <h:head> </h:head> <h:body> <h:form> <p:commandButton value= "Inc" > <p:ajax listener= "#{foo.inc}" update= "@all" /> </p:commandButton> <p:commandButton value= "Auto inc" onclick= "inc()" style= "margin-left: 0.5rem" /> <p:remoteCommand name= "inc" actionListener= "#{foo.inc}" update= "@all" oncomplete= "setTimeout(inc, 0);" /> <p> <h:outputText id= "label" value= "#{foo.value}" /> </p> </h:form> </h:body> </html> Backing bean Foo.java package com.example; import javax.faces.view.ViewScoped; import javax.inject.Named; import java.io.Serializable; @Named @ViewScoped public class Foo implements Serializable { private int value; public int getValue() { return value; } public void setValue( int value) { this .value=value; } public void inc() { this .value++; } }

    Description

      Particular HTTP/2 Ajax requests to JSF resources yield "default task" threads stuck within AbstractFramedStreamSinkChannel.awaitWritable().

      The requests are initiated by PrimeFaces, typically when updating @all components or opening global dialogs. They cause some .js or .css resources to be (re)loaded.

      It only happens when using Firefox and HTTP/2.

      In production, too many of those threads sometimes caused all "default I/O" threads to consume 100% CPU and Wildfly to become unresponsive. I tried setting various timeouts for the listener, but it didn't help. The threads disappear after a while, though. I noticed this first with Wildfly 20.0.1.Final and can still reproduce it with 23.0.0.Final.

      I attached a minimal example project to reproduce the issue (see "Steps to Reproduce").

      Stuck thread

      "default task-207" #1724 prio=5 os_prio=0 cpu=33,17ms elapsed=705,57s tid=0x000055d9e5b89000 nid=0x1f898 in Object.wait()  [0x00007f59bd2e0000]
         java.lang.Thread.State: WAITING (on object monitor)
       at java.lang.Object.wait(java.base@11.0.10/Native Method)
       - waiting on <no object reference available>
       at java.lang.Object.wait(java.base@11.0.10/Object.java:328)
       at io.undertow.server.protocol.framed.AbstractFramedStreamSinkChannel.awaitWritable(AbstractFramedStreamSinkChannel.java:289)
       - waiting to re-lock in wait() <0x000000043add1860> (a java.lang.Object)
       at org.xnio.conduits.StreamSinkChannelWrappingConduit.awaitWritable(StreamSinkChannelWrappingConduit.java:99)
       at org.xnio.conduits.ConduitStreamSinkChannel.awaitWritable(ConduitStreamSinkChannel.java:134)
       at io.undertow.channels.DetachableStreamSinkChannel.awaitWritable(DetachableStreamSinkChannel.java:87)
       at io.undertow.server.HttpServerExchange$WriteDispatchChannel.awaitWritable(HttpServerExchange.java:2115)
       at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:578)
       at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:151)
       at java.nio.channels.Channels$WritableByteChannelImpl.write(java.base@11.0.10/Channels.java:464)
       - locked <0x000000043adda930> (a java.lang.Object)
       at com.sun.faces.application.resource.ResourceHandlerImpl.handleResourceRequest(ResourceHandlerImpl.java:318)
       at javax.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:171)
       at org.primefaces.application.resource.PrimeResourceHandler.handleResourceRequest(PrimeResourceHandler.java:87)
       at javax.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:704)
       at javax.faces.webapp.FacesServlet.service(FacesServlet.java:451)
       at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74)
       at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:129)
       at io.opentracing.contrib.jaxrs2.server.SpanFinishingFilter.doFilter(SpanFinishingFilter.java:52)
       at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61)
       at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131)
       at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84)
       at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62)
       at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68)
       at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36)
       at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78)
       at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
       at io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68)
       at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:117)
       at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57)
       at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
       at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46)
       at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64)
       at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60)
       at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77)
       at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50)
       at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43)
       at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
       at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61)
       at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
       at org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68)
       at io.undertow.servlet.handlers.SendErrorPageHandler.handleRequest(SendErrorPageHandler.java:52)
       at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43)
       at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269)
       at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78)
       at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133)
       at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130)
       at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
       at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
       at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
       at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction$$Lambda$1336/0x0000000841087040.call(Unknown Source)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$1337/0x0000000841087440.call(Unknown Source)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$1337/0x0000000841087440.call(Unknown Source)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$1337/0x0000000841087440.call(Unknown Source)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1530)
       at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction$$Lambda$1337/0x0000000841087440.call(Unknown Source)
       at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249)
       at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78)
       at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99)
       at io.undertow.server.Connectors.executeRootHandler(Connectors.java:387)
       at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:841)
       at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
       at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
       at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
       at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
       at org.xnio.XnioWorker$WorkerThreadFactory$1$1.run(XnioWorker.java:1280)
       at java.lang.Thread.run(java.base@11.0.10/Thread.java:834)
      
         Locked ownable synchronizers:
       - None
      

      Attachments

        Issue Links

          Activity

            People

              flaviarnn Flavia Rainone
              foobartender S E (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: