In the case where XATerminatorImple.prepare is called and the tx has previously been set rollbackOnly, we throw XA_RBROLLBACK without actually rolling back. To make matters worse we explicitly remove the tx handle, making subsequent cleanup via commit/rollback impossible.
The JCA 1.5 spec fails to specify the expected behaviour, but following the XA spec this would be a violation, as the XA_RBROLLBACK code indicates the RM has rolled back the branch and needs/expects no further contact.
However, actually rolling back can cause heuristics outcomes, since the subordinate tx may contain several branches. These can't be reported from prepare, which does not support throwing the heuristic exceptions or using XAException.XA_HE* error codes. That would seem to imply a necessity for either violating the XATerminator API interface spec or returning e.g. XAER_RMERR instead of XA_RBROLLBACK, and expecting to have rollback called explicitly to allow clean up. Or we can clean up in prepare, bury any heuristic and report it as a XA_RMERR.