Uploaded image for project: 'WildFly OpenSSL'
  1. WildFly OpenSSL
  2. WFSSL-74

Update the error message that occurs when OpenSSLEngine#closeInbound is called before receiving a close_notify message from the peer

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major Major
    • 2.1.4.Final
    • None
    • None

      Infinispan has a test case where the client doesn't trust the server's certificate:

      07:22:27,164 DEBUG (HotRod-Test-ServerIO-229-1:[]) [OpenSSLEngine] WFOPENSSL0042 Setting pre-TLS 1.3 cipher suites to ECDHE-ECDSA-AES256-GCM-SHA384:...
      07:22:27,165 DEBUG (HotRod-Test-ServerIO-229-1:[]) [OpenSSLEngine] WFOPENSSL0043 Setting TLS 1.3 cipher suites to TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256:TLS_AES_128_GCM_SHA256:TLS_AES_128_CCM_SHA256
      07:22:27,192 DEBUG (Test-Client-231-1:[]) [OpenSSLContextSPI] Certificate verification failed
      sun.security.validator.ValidatorException: Certificate signature validation failed
      	at sun.security.validator.SimpleValidator.engineValidate(SimpleValidator.java:216) ~[?:?]
      	at sun.security.validator.Validator.validate(Validator.java:264) ~[?:?]
      	at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313) ~[?:?]
      	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:233) ~[?:?]
      	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:110) ~[?:?]
      	at org.wildfly.openssl.OpenSSLContextSPI.lambda$init$0(OpenSSLContextSPI.java:241) ~[wildfly-openssl-2.1.3.Final.jar:2.1.3.Final]
      	at org.wildfly.openssl.SSLImpl.readFromSSL0(Native Method) ~[wildfly-openssl-2.1.3.Final.jar:2.1.3.Final]
      	at org.wildfly.openssl.SSLImpl.readFromSSL(SSLImpl.java:153) [wildfly-openssl-2.1.3.Final.jar:?]
      	at org.wildfly.openssl.OpenSSLEngine.unwrap(OpenSSLEngine.java:612) [wildfly-openssl-2.1.3.Final.jar:2.1.3.Final]
      	at javax.net.ssl.SSLEngine.unwrap(SSLEngine.java:637) [?:?]
      	at io.netty.handler.ssl.SslHandler$SslEngineType$3.unwrap(SslHandler.java:284) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1358) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1253) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1300) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:508) [netty-codec-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:447) [netty-codec-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at java.lang.Thread.run(Thread.java:834) [?:?]
      Caused by: java.security.SignatureException: Signature does not match.
      	at sun.security.x509.X509CertImpl.verify(X509CertImpl.java:456) ~[?:?]
      	at sun.security.x509.X509CertImpl.verify(X509CertImpl.java:390) ~[?:?]
      	at org.wildfly.openssl.OpenSslX509Certificate.verify(OpenSslX509Certificate.java:139) ~[wildfly-openssl-2.1.3.Final.jar:2.1.3.Final]
      	at sun.security.validator.SimpleValidator.engineValidate(SimpleValidator.java:214) ~[?:?]
      	... 32 more
      07:22:27,196 DEBUG (Test-Client-231-1:[]) [OpenSSLEngine] WFOPENSSL0008 Read from SSL failed error: (337047686) read result:(-1) error string: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed
      07:22:27,197 DEBUG (HotRod-Test-ServerIO-229-1:[]) [SslHandler] [id: 0x38f32a8c, L:/127.0.0.1:12521 ! R:/127.0.0.1:47730] SSLEngine.closeInbound() raised an exception.
      javax.net.ssl.SSLException: WFOPENSSL0009 Inbound is closed
      	at org.wildfly.openssl.OpenSSLEngine.closeInbound(OpenSSLEngine.java:716) ~[wildfly-openssl-2.1.3.Final.jar:2.1.3.Final]
      	at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1815) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.handler.ssl.SslHandler.channelInactive(SslHandler.java:1086) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.fireChannelInactive(AbstractChannelHandlerContext.java:241) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelInactive(DefaultChannelPipeline.java:1405) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:262) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelInactive(AbstractChannelHandlerContext.java:248) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.DefaultChannelPipeline.fireChannelInactive(DefaultChannelPipeline.java:901) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.AbstractChannel$AbstractUnsafe$8.run(AbstractChannel.java:831) [netty-transport-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384) [netty-transport-native-epoll-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.63.Final.jar:4.1.63.Final]
      	at java.lang.Thread.run(Thread.java:834) [?:?]
      

      Their Netty handler on the server side then ends up attempting to call OpenSSLEngine#closeInbound and this results in the following SSLException being thrown:

      javax.net.ssl.SSLException: WFOPENSSL0009 Inbound is closed
              at org.wildfly.openssl.OpenSSLEngine.closeInbound(OpenSSLEngine.java:716) ~[wildfly-openssl-2.1.4.CR1-SNAPSHOT.jar:2.1.4.CR1-SNAPSHOT]
              at io.netty.handler.ssl.SslHandler.setHandshakeFailure(SslHandler.java:1815) [netty-handler-4.1.63.Final.jar:4.1.63.Final]
      

      It turns out that Netty's SSLHandler#setHandshakeFailure can swallow the SSLException if closeInbound is called before receiving the close_notify message since the connection is about to be closed anyway. However, Netty's SSLHandler#setHandshakeFailure is looking for the specific string that the JDK SSLEngine throws in this case: "closing inbound before receiving peer's close_notify".

      We need to update our error message to be the same so that Netty's SSLHandler#setHandshakeFailure method will swallow the SSLException when closeInbound is called before receiving the close_notify message.

      See https://issues.redhat.com/browse/WFSSL-73?focusedCommentId=16169487&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16169487 for more details.

            fjuma1@redhat.com Farah Juma
            fjuma1@redhat.com Farah Juma
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: