Uploaded image for project: 'XNIO'
  1. XNIO
  2. XNIO-350

xnio.nio.alt-queued-server breaks undertow ssl under load

    XMLWordPrintable

Details

    Description

      I can locally reproduce this failure using a jmh benchmark where the client is configured not to use persistent connections:

      ERROR [2019-10-04T14:21:22.321Z] io.undertow.request: Closing SSLConduit after exception on handshake
javax.net.ssl.SSLProtocolException: Handshake message sequence violation, 1
	at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1530)
	at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:528)
	at sun.security.ssl.SSLEngineImpl.readNetRecord(SSLEngineImpl.java:802)
	at sun.security.ssl.SSLEngineImpl.unwrap(SSLEngineImpl.java:766)
	at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:624)
	at com.palantir.tritium.metrics.InstrumentedSslEngine.unwrap(InstrumentedSslEngine.java:135)
	at io.undertow.protocols.ssl.SslConduit.doUnwrap(SslConduit.java:757)
	at io.undertow.protocols.ssl.SslConduit.doHandshake(SslConduit.java:648)
	at io.undertow.protocols.ssl.SslConduit.access$900(SslConduit.java:63)
	at io.undertow.protocols.ssl.SslConduit$5$1.run(SslConduit.java:1084)
	at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:479)
Caused by: javax.net.ssl.SSLProtocolException: Handshake message sequence violation, 1
	at sun.security.ssl.HandshakeStateManager.check(HandshakeStateManager.java:362)
	at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:212)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
	at sun.security.ssl.Handshaker$1.run(Handshaker.java:970)
	at sun.security.ssl.Handshaker$1.run(Handshaker.java:967)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1459)
	at io.undertow.protocols.ssl.SslConduit$5.run(SslConduit.java:1072)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1871)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1451)
	at java.lang.Thread.run(Thread.java:748)

      This error is from a running service, and began to appear after upgrading xnio. Downgrading XNIO resolved the issue. In my synthetic reproducer I was able to resolve hte issue by setting the xnio.nio.alt-queued-server jvm property to false (unfortunately baked into an internal code-base, I can put together a smaller public reproducer this weekend or next week if that's helpful)

      java.io.IOException: overflow
	at io.undertow.protocols.ssl.SslConduit.doWrap(SslConduit.java:911)
	at io.undertow.protocols.ssl.SslConduit.doHandshake(SslConduit.java:649)
	at io.undertow.protocols.ssl.SslConduit.access$900(SslConduit.java:63)
	at io.undertow.protocols.ssl.SslConduit$5$1.run(SslConduit.java:1084)
	at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612)
	at org.xnio.nio.WorkerThread.run(WorkerThread.java:479)

      Attachments

        Issue Links

          is caused by

          Bug - A problem that impairs or prevents the functions of the product. XNIO-351 Queued executor (v2) can accept into the wrong thread

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Resolved
          is incorporated by

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. JBEAP-17689 Upgrade XNIO to 3.7.6.Final

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Verified

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. JBEAP-17665 [GSS](7.2.z) Upgrade XNIO from 3.7.3.Final-redhat-00001 to 3.7.6.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Component Upgrade - A task tracking an update to a bundled component or revision of component upstream of the project. WFCORE-4691 Upgrade XNIO to 3.7.5.Final

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              dlloyd@redhat.com David Lloyd
              carterkozak Carter Kozak
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: