XAException of error code XAException.XA_RETRY is wrongly wrapped under XAException with error code XAException.XAER_RMERR if transient error happens when committing imported transaction.

ImportedTransaction.doCommit returns false if there was a transient error that implies that a caller could call the commit once again in a while to finish the commit. For the caller to try that it has to be informed with correct XAException error code. (Caller is e.g. some EIS system which imported an inflow transaction and now commands it to commit.)
Returning the XAException.XA_RETRY is right as you define that caller should return back and try to commit. Returning XAException.XAER_RMERR is wrong as it says there was some unspecified error and the caller can just say - there is something wrong and give up to retry with the commit. This is how I understand and Narayana implements the XA spec.