Uploaded image for project: 'WildFly OpenSSL'
  1. WildFly OpenSSL
  2. WFSSL-73

OpenSSLEngine shuts down too early

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 2.1.4.Final
    • 1.0.12.Final
    • None

    Description

      Using wildfly-openssl 1.0.12.Final with with Netty 4.1.51.Final and 4.1.63.Final.

      OpenSSLEngine shuts itself down in unwrap() after receiving a close_notify message from the client here:

      	at org.wildfly.openssl.OpenSSLEngine.shutdown(OpenSSLEngine.java:205)
	at org.wildfly.openssl.OpenSSLEngine.closeInbound(OpenSSLEngine.java:692)
	at org.wildfly.openssl.OpenSSLEngine.unwrap(OpenSSLEngine.java:657)
	at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637)
	at io.netty.handler.ssl.JdkSslEngine.unwrap(JdkSslEngine.java:92)
	at io.netty.handler.ssl.JdkAlpnSslEngine.unwrap(JdkAlpnSslEngine.java:143)
	at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1380)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1275)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1322)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)

      RFC 2246 section 7.2.1 says both sides need to send a close_notify message for the connection to be closed, so I believe the server could send more data, and OpenSSLEngine should stay available.

      In our case, Netty's output buffer is empty, but it still goes through the motions of flushing the connection before closing it, and it fails because the engine is shut down:

      java.lang.IllegalStateException: ssl is null
	at org.wildfly.openssl.SSLImpl.getSessionId0(Native Method)
	at org.wildfly.openssl.SSLImpl.getSessionId(SSLImpl.java:494)
	at org.wildfly.openssl.OpenSSLEngine.getSession(OpenSSLEngine.java:977)
	at io.netty.handler.ssl.JdkSslEngine.getSession(JdkSslEngine.java:48)
	at io.netty.handler.ssl.SslHandler$SslEngineType$3.allocateWrapBuffer(SslHandler.java:312)
	at io.netty.handler.ssl.SslHandler.allocateOutNetBuf(SslHandler.java:2207)
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:840)
	at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:811)
	at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:792)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:189)
	at io.netty.handler.codec.http2.Http2MultiplexCodec.flush0(Http2MultiplexCodec.java:282)
	at io.netty.handler.codec.http2.Http2MultiplexCodec.processPendingReadCompleteQueue(Http2MultiplexCodec.java:261)
	at io.netty.handler.codec.http2.Http2MultiplexCodec.channelReadComplete(Http2MultiplexCodec.java:240)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:397)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:390)
	at io.netty.handler.ssl.SslHandler.channelReadComplete0(SslHandler.java:1341)
	at io.netty.handler.ssl.SslHandler.channelReadComplete(SslHandler.java:1330)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:397)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:390)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1415)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:397)
	at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:925)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:812)

      The comment in Http2MultiplexCodec is not very reassuring, but from what I can tell the code is correct.

      We have a Netty handler that automatically closes the connection on errors, which triggers another flush, and another IllegalStateException, and so on until the thread runs out of stack. We have added a crude workaround in our code to ignore IllegalStateExceptions with message ssl is null (ISPN-12558, and now ISPN-12996).

      Attachments

        Issue Links

          causes

          Bug - A problem that impairs or prevents the functions of the product. ISPN-12558 StackOverflowError during REST connection shutdown

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. ISPN-12996 StackOverflowError during Hot Rod connection shutdown

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is related to

          Task - Represents a small unit of work that is not end-user facing. WFSSL-74 Update the error message that occurs when OpenSSLEngine#closeInbound is called before receiving a close_notify message from the peer

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved
          relates to

          Bug - A problem that impairs or prevents the functions of the product. ISPN-13016 HotRodSniFunctionalTest.testServerWithNotMatchingDefaultAndClientWithSNI random hangs

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Code Review
          links to

          Web Link Github issue #93

          Activity

            People

              fjuma1@redhat.com Farah Juma
              dberinde@redhat.com Dan Berindei (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: