We have a client application which does ejb calls to the server to retrieve detailed data for user selected item. The call is performed in a separate thread. When user choses different item, before the previous result is returned, we interrupt the previous thread and run a new ejb call. After over then 80 (output messages limit) of such calls the communication hangs. No more ejb calls can be processed for that session.

It happens only with big responses from the server, as when the server fills in the output window, it waits for ACK or CLOSE from the client. The client side, when interrupted, never sends such message and the server side thread is blocked until the whole channel is closed. Those thread are blocked in org.jboss.remoting3.remote.OutboundMessage$1.accept(OutboundMessage.java:114)

I've been able to create a test case, demonstrating the issue - in attached file and patch (4.0.12.Final).

The patch contains also a proposal for fix - close the message on exception: in /org/jboss/ejb/protocol/remote/EJBClientChannel.java:1227 the discardResult();

} catch (IOException | ClassNotFoundException ex) {
   discardResult();
   throw new EJBException("Failed to read response", ex);
}

The same scenario fails on master.