There are situations in which a Database performs a rollback and Agroal is flushing the connections too agressively, leading to unnecessary performance degradation. This could be handled more gracefully.

A Resource Manager like a Database may unilaterally decide to rollback. This is an optimization/behaviour that DB2i does in case of an xa_end() with flag TMFAIL. In this case a rollback is being initiated by the application and when xa_end() is called with TMFAIL the return code is XA_RBBASE, which leads to an XAException with errorCode 100 (the value of XA_RBBASE). When this Exception is thrown Agroal decides in BaseXaResource#end() that the transactionAware (ConnectionHandler) should be set to flushOnly. In this case, this is overkill and the connection could be kept alive.

Excerpt from the XA Specification:

Rollback-only
An RM need not wait for global transaction completion to report an error. The RM
can return rollback-only as the result of any xa_start() or xa_end() call. The TM can
use this knowledge to avoid starting additional work on behalf of the global
transaction. An RM can also unilaterally roll back and forget a transaction branch
any time before it prepares it. A TM detects this when an RM subsequently
indicates that it does not recognise the XID.

I propose that Agroal be made more tolerant to at least this specific situation and handle rollback-only from the RM more gracefully when already rolling back. But this would also apply to commits. The RM is allowed to mark the transaction rollback-only before an xa_prepare, regardless of whether or not an application is going to commit or rollback. There is no reason to flush a connection in these cases, because there is nothing wrong with the connection.