Details
-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
28.0.0.Final
-
None
-
---
-
---
Description
We upgraded from 27.0.1 to 28.0.0 at the start of May. Although the upgrade/testing went smoothly, in production we saw a small number of NPE during multi-part uploads, and had to revert to 27.0.1.
The start of the traceback is:
2023-05-01 15:37:41,611 ERROR [stderr] (default task-2) java.lang.NullPointerException: Cannot invoke "io.undertow.connector.PooledByteBuffer.getBuffer()" because "this.pooled" is null
2023-05-01 15:37:41,611 ERROR [stderr] (default task-2) at io.undertow.servlet@2.3.5.Final//io.undertow.servlet.spec.ServletInputStreamImpl.readIntoBuffer(ServletInputStreamImpl.java:202)
2023-05-01 15:37:41,611 ERROR [stderr] (default task-2) at io.undertow.servlet@2.3.5.Final//io.undertow.servlet.spec.ServletInputStreamImpl.read(ServletInputStreamImpl.java:176)
2023-05-01 15:37:41,611 ERROR [stderr] (default task-2) at io.undertow.core@2.3.5.Final//io.undertow.server.handlers.form.MultiPartParserDefinition$MultiPartUploadHandler.parseBlocking(MultiPartParserDefinition.java:226)
2023-05-01 15:37:41,611 ERROR [stderr] (default task-2) at io.undertow.servlet@2.3.5.Final//io.undertow.servlet.spec.HttpServletRequestImpl.parseFormData(HttpServletRequestImpl.java:877)
2023-05-01 15:37:41,611 ERROR [stderr] (default task-2) at io.undertow.servlet@2.3.5.Final//io.undertow.servlet.spec.HttpServletRequestImpl.getParameterMap(HttpServletRequestImpl.java:829)
The uploads were 1-5 MB in size. The application servlet responds to the HTTP POST by extracting some values from the request, then calling getParameterMap().
HAProxy logs suggest that the requests may have been in progress for a little over 90 seconds.It is hard to tell if slow or unreliable uplinks from the clients may be aggravating the problem.
The application code was not changed across the upgrade apart from being recompiled. Other supporting non-wildfly libraries were not changed. The wildfly configuration was identical.
Unfortunately we are unable to replicate this at will, and are still no nearer being able to provide any more information. I do suspect it is timing-dependent and potentially related to interrupted uploads. It's also possible that haproxy is somehow contributing. With a frequency in production of about one per day it is too frequent to leave live, but too infrequent to know how to diagnose. Despite numerous attempts I've been unable to provoke it myself which makes trying to create a simple reproducer very hard.
Attachments
Issue Links
- is caused by
-
UNDERTOW-2298 Possible NPE on concurrent read & close in ServletInputStreamImpl
- Pull Request Sent