Uploaded image for project: 'Undertow'
  1. Undertow
  2. UNDERTOW-642

NPE with HTTP2 and specific clients

XMLWordPrintable

    • Hide

      We can only reproduce this with one client: an Android phone running Chrome 48.0.2564.95 and Android 5.1.1 with the User-Agent string:

      Mozilla/5.0 (Linux; Android 5.1.1; LG-VS985 Build/LMY48Y) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.95 Mobile Safari/537.36

      Other phones running the same Chrome build do not trigger the error. We know other phones in the wild also trigger this error because we do see the error in our logs periodically.

      Show
      We can only reproduce this with one client: an Android phone running Chrome 48.0.2564.95 and Android 5.1.1 with the User-Agent string: Mozilla/5.0 (Linux; Android 5.1.1; LG-VS985 Build/LMY48Y) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.95 Mobile Safari/537.36 Other phones running the same Chrome build do not trigger the error. We know other phones in the wild also trigger this error because we do see the error in our logs periodically.

      Thank you for your amazing work.

      We see the following error in the logs when a specific "type" of client hits our Undertow server:

      2016-02-22 19:28:18,338 XNIO001007: A channel event listener threw an exception
      java.lang.NullPointerException
      at io.undertow.protocols.http2.Http2PriorityTree$Http2PriorityNode.addDependent(Http2PriorityTree.java:248)
      at io.undertow.protocols.http2.Http2PriorityTree$Http2PriorityNode.exclusive(Http2PriorityTree.java:258)
      at io.undertow.protocols.http2.Http2PriorityTree.registerStream(Http2PriorityTree.java:65)
      at io.undertow.protocols.http2.Http2Channel.createChannel(Http2Channel.java:310)
      at io.undertow.protocols.http2.Http2Channel.createChannel(Http2Channel.java:60)
      at io.undertow.server.protocol.framed.AbstractFramedChannel.receive(AbstractFramedChannel.java:433)
      at io.undertow.server.protocol.http2.Http2ReceiveListener.handleEvent(Http2ReceiveListener.java:103)
      at io.undertow.server.protocol.http2.Http2ReceiveListener.handleEvent(Http2ReceiveListener.java:56)
      at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      at io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:872)
      at io.undertow.server.protocol.framed.AbstractFramedChannel$FrameReadListener.handleEvent(AbstractFramedChannel.java:853)
      at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
      at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
      at io.undertow.protocols.ssl.SslConduit$SslReadReadyHandler.readReady(SslConduit.java:1059)
      at io.undertow.protocols.ssl.SslConduit$1.run(SslConduit.java:229)
      at org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:580)
      at org.xnio.nio.WorkerThread.run(WorkerThread.java:464)

      We are embedding Undertow. The configuration looks like this:

      			// Only allow secure ciphers
      			Sequence<String> ciphers = Sequence.of(
      				"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384",
      				"TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384","TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
      				"TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256","TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384",
      				"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384","TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA",
      				"TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384","TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384","TLS_ECDH_RSA_WITH_AES_256_CBC_SHA",
      				"TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA","TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256","TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256",
      				"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA","TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA","TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256",
      				"TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256","TLS_ECDH_RSA_WITH_AES_128_CBC_SHA","TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA",
      				"TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA","TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA","TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA",
      				"TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA");
      			
      			// Explicitly disable SSLv3 even though it's implied by the allowed cipher list
      			Sequence<String> sslProtocols = Sequence.of("TLSv1","TLSv1.1","TLSv1.2");
      									
                  try {
      	        	SSLContext sslContext = createSSLContext(loadKeyStore("server.keystore"), loadKeyStore("server.truststore"));
      	        	System.out.println("Keystore found.  HTTP/HTTPS.");
      
      	        	server = Undertow.builder()
      			        	.setServerOption(UndertowOptions.ENABLE_HTTP2, true)
      			        	.setBufferSize(16 * 1024)
      			        	
      				        .setSocketOption(Options.BACKLOG, 50000)
      				        .setSocketOption(Options.SSL_ENABLED_CIPHER_SUITES, ciphers)
      				        .setSocketOption(Options.SSL_ENABLED_PROTOCOLS, sslProtocols)
      
      				        .setServerOption(UndertowOptions.MAX_ENTITY_SIZE, (long)(8 * 1024 * 1024)) 	// max upload size
      				        .setServerOption(UndertowOptions.ALWAYS_SET_KEEP_ALIVE, false) //don't send a keep-alive header for HTTP/1.1 requests, as it is not required
      				        .setServerOption(UndertowOptions.ALWAYS_SET_DATE, true)
      				        .setServerOption(UndertowOptions.ENABLE_CONNECTOR_STATISTICS, false)
      				        .setServerOption(UndertowOptions.RECORD_REQUEST_START_TIME, false)	
      
      				        .setWorkerOption(Options.CONNECTION_HIGH_WATER, 1000000)
      			        	.setWorkerOption(Options.CONNECTION_LOW_WATER, 1000000)
      			        	.setWorkerOption(Options.TCP_NODELAY, true)
      			        	.setWorkerOption(Options.CORK, true)
      
      			        	.setWorkerOption(Options.WORKER_IO_THREADS, Runtime.getRuntime().availableProcessors() - 2)
      			        	.setWorkerOption(Options.WORKER_TASK_CORE_THREADS, 2)
      			        	.setWorkerOption(Options.WORKER_TASK_MAX_THREADS, Runtime.getRuntime().availableProcessors())
      
      			        	.addHttpListener(port, host)
      			        	.addHttpsListener(securePort, host, sslContext)	        	
      			        	.setHandler(router)
      			        	.build();
      	        	
      	        } catch (Exception e) {
      

      We traced the bug to the the "exclusive=true" case in Http2PriorityTree. When exclusive is true and the node has no dependents the "exclusive()" method of Http2PriorityNode will always throw an NPE. Wrapping...

           node.addDependent(i);
      

      ...with...

           if (i != null) {
                node.addDependent(i);
           }
      

      ..."fixes" the problem in our case. However, not understanding the code or the HTTP2 protocol, this naive fix may have other implications.

      Please take a look at the code and see if a better fix is in order.

      For, now we've switched back to HTTP1.1 and the server is stable.

              sdouglas1@redhat.com Stuart Douglas (Inactive)
              mgrabinski Michael Grabinski (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: