When multiple clients all authenticating with client certs make enough requests in parallel Undertow worker task pool gets saturated and all threads in the pool are stuck at:
"XNIO-2 task-64" #256 prio=5 os_prio=0 tid=0x00007f537c024800 nid=0x1005 in Object.wait() [0x00007f5366883000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at io.undertow.protocols.ssl.SslConduit.awaitReadable(SslConduit.java:320)
- locked <0x00000001ce9dc0b0> (a io.undertow.protocols.ssl.SslConduit)
at org.xnio.conduits.ConduitStreamSourceChannel.awaitReadable(ConduitStreamSourceChannel.java:155)
at io.undertow.server.ConnectionSSLSessionInfo.renegotiateNoRequest(ConnectionSSLSessionInfo.java:216)
at io.undertow.server.ConnectionSSLSessionInfo.renegotiate(ConnectionSSLSessionInfo.java:130)
at io.undertow.security.impl.ClientCertAuthenticationMechanism.getPeerCertificates(ClientCertAuthenticationMechanism.java:125)
at io.undertow.security.impl.ClientCertAuthenticationMechanism.authenticate(ClientCertAuthenticationMechanism.java:92)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:245)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:268)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.access$100(SecurityContextImpl.java:231)
at io.undertow.security.impl.SecurityContextImpl.attemptAuthentication(SecurityContextImpl.java:125)
at io.undertow.security.impl.SecurityContextImpl.authTransition(SecurityContextImpl.java:99)
at io.undertow.security.impl.SecurityContextImpl.authenticate(SecurityContextImpl.java:92)
at io.undertow.security.handlers.AuthenticationCallHandler.handleRequest(AuthenticationCallHandler.java:50)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:332)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Then little by little SSL handshakes start falling of with:
java.lang.IllegalStateException: UT000124: renegotiation timed out
at io.undertow.server.ConnectionSSLSessionInfo.renegotiateNoRequest(ConnectionSSLSessionInfo.java:222)
at io.undertow.server.ConnectionSSLSessionInfo.renegotiate(ConnectionSSLSessionInfo.java:130)
at io.undertow.security.impl.ClientCertAuthenticationMechanism.getPeerCertificates(ClientCertAuthenticationMechanism.java:127)
at io.undertow.security.impl.ClientCertAuthenticationMechanism.authenticate(ClientCertAuthenticationMechanism.java:94)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:245)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:268)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.transition(SecurityContextImpl.java:268)
at io.undertow.security.impl.SecurityContextImpl$AuthAttempter.access$100(SecurityContextImpl.java:231)
at io.undertow.security.impl.SecurityContextImpl.attemptAuthentication(SecurityContextImpl.java:125)
at io.undertow.security.impl.SecurityContextImpl.authTransition(SecurityContextImpl.java:99)
at io.undertow.security.impl.SecurityContextImpl.authenticate(SecurityContextImpl.java:92)
at io.undertow.security.handlers.AuthenticationCallHandler.handleRequest(AuthenticationCallHandler.java:50)
at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360)
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
By the looks of it the place from which notifyAll() happens (SslConduit.runTasks()) has to be executed in a thread from the same saturated pool which will obviously not have any free threads to allocate.
This causes vicious cycle where the app becomes barely responsive.