The XA specification states the following about the RBROLLBACK error code for xa_commit():

[XA_RB∗]
The resource manager did not commit the work done on behalf of the transaction branch. Upon return, the resource manager has rolled back the branch’s work and has released all held resources. These values may be returned only if TMONEPHASE is set in flags:
[XA_RBROLLBACK]
The resource manager rolled back the transaction branch for an unspecified reason

It is clear that the RBROLLBACK code may only be returned if the branch is known to have rolled back (and indeed if the onePhase flag is set to true).

To cater for scenarios where the XAResource is not able to determine the outcome of the resource managers commit. In those scenarios the XA specification is clear that the only legal error code to return is:

[XAER_RMFAIL]
An error occurred that makes the resource manager unavailable.