Loading...

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: 2.1.4.Final
Affects Version/s: 1.0.12.Final
Labels:
None

Blocked:
False
Ready:
False
Release Note Text:
Undefined
Git Pull Request:
https://github.com/wildfly-security/wildfly-openssl/pull/103
Market:

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Using wildfly-openssl 1.0.12.Final with with Netty 4.1.51.Final and 4.1.63.Final.

OpenSSLEngine shuts itself down in unwrap() after receiving a close_notify message from the client here:

	at org.wildfly.openssl.OpenSSLEngine.shutdown(OpenSSLEngine.java:205)
	at org.wildfly.openssl.OpenSSLEngine.closeInbound(OpenSSLEngine.java:692)
	at org.wildfly.openssl.OpenSSLEngine.unwrap(OpenSSLEngine.java:657)
	at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637)
	at io.netty.handler.ssl.JdkSslEngine.unwrap(JdkSslEngine.java:92)
	at io.netty.handler.ssl.JdkAlpnSslEngine.unwrap(JdkAlpnSslEngine.java:143)
	at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:282)
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1380)
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1275)
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1322)
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:93)
	at org.infinispan.server.core.transport.StatsChannelHandler.channelRead(StatsChannelHandler.java:26)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
	at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)

RFC 2246 section 7.2.1 says both sides need to send a close_notify message for the connection to be closed, so I believe the server could send more data, and OpenSSLEngine should stay available.

In our case, Netty's output buffer is empty, but it still goes through the motions of flushing the connection before closing it, and it fails because the engine is shut down:

java.lang.IllegalStateException: ssl is null
	at org.wildfly.openssl.SSLImpl.getSessionId0(Native Method)
	at org.wildfly.openssl.SSLImpl.getSessionId(SSLImpl.java:494)
	at org.wildfly.openssl.OpenSSLEngine.getSession(OpenSSLEngine.java:977)
	at io.netty.handler.ssl.JdkSslEngine.getSession(JdkSslEngine.java:48)
	at io.netty.handler.ssl.SslHandler$SslEngineType$3.allocateWrapBuffer(SslHandler.java:312)
	at io.netty.handler.ssl.SslHandler.allocateOutNetBuf(SslHandler.java:2207)
	at io.netty.handler.ssl.SslHandler.wrap(SslHandler.java:840)
	at io.netty.handler.ssl.SslHandler.wrapAndFlush(SslHandler.java:811)
	at io.netty.handler.ssl.SslHandler.flush(SslHandler.java:792)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:750)
	at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:742)
	at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:728)
	at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:189)
	at io.netty.handler.codec.http2.Http2MultiplexCodec.flush0(Http2MultiplexCodec.java:282)
	at io.netty.handler.codec.http2.Http2MultiplexCodec.processPendingReadCompleteQueue(Http2MultiplexCodec.java:261)
	at io.netty.handler.codec.http2.Http2MultiplexCodec.channelReadComplete(Http2MultiplexCodec.java:240)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:397)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:390)
	at io.netty.handler.ssl.SslHandler.channelReadComplete0(SslHandler.java:1341)
	at io.netty.handler.ssl.SslHandler.channelReadComplete(SslHandler.java:1330)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:397)
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelReadComplete(AbstractChannelHandlerContext.java:390)
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelReadComplete(DefaultChannelPipeline.java:1415)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:410)
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelReadComplete(AbstractChannelHandlerContext.java:397)
	at io.netty.channel.DefaultChannelPipeline.fireChannelReadComplete(DefaultChannelPipeline.java:925)
	at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:812)

The comment in Http2MultiplexCodec is not very reassuring, but from what I can tell the code is correct.

We have a Netty handler that automatically closes the connection on errors, which triggers another flush, and another IllegalStateException, and so on until the thread runs out of stack. We have added a crude workaround in our code to ignore IllegalStateExceptions with message ssl is null (~~ISPN-12558~~, and now ~~ISPN-12996~~).

causes

ISPN-12558 StackOverflowError during REST connection shutdown

Closed

ISPN-12996 StackOverflowError during Hot Rod connection shutdown

Closed

is related to

WFSSL-74 Update the error message that occurs when OpenSSLEngine#closeInbound is called before receiving a close_notify message from the peer

Resolved

relates to

ISPN-13016 HotRodSniFunctionalTest.testServerWithNotMatchingDefaultAndClientWithSNI random hangs

Code Review

links to

Github issue #93

Details

Description

Attachments

Issue Links

Activity

People

Dates

Hide