The only surefire way to ensure that Narayana recovers in-flight subordinates in the event of a master crash is to persist the transaction as soon as the subordinate is enrolled. We specifically want to avoid:

• An ever-growing set of URIs to recover (that is, if we know that a given node is not a subordinate, there is no need to contact it)
• Any situation where a node has a subordinate transaction but the master doesn't know about it

In order to resolve this, we must register a recovery helper which utilizes a file store which is populated with the active outflowed transactions.

Keeping a data directory which maintains multiple files seems reasonable: one file per transaction with subordinates. Enlisting a subordinate would add one line to the file, with each line consisting of a recovery URI. The name of the file would be a base-16 or base-32 (NOT base-64) encoding of the XID or UID of the transaction. The file would be created before attempting to enlist the first subordinate (and appended to thereafter), and flushed after the write operation is complete (see FileChannel.force(true)), and a failure to create or append to the file would result in an XA resource error of some sort. On commit/rollback/forget as you say we can just unlink the file. We could consider removing entries that did not turn out to require enlistment, but that may add some complexity or slowdown. Because the file may be concurrently accessed, it should be created in a loop exclusively or opened with no flags; if either fails, re-check the file (use a method on Files for this purpose) and try again. The file channel must be locked exclusively after the open but before any access. Using the append option should ensure that new items are always added at the end once the lock is acquired.

The recovery helper would traverse this collection and create a SubordinateXAResource for each destination/parent name. This can be done by calling org.wildfly.transaction.client.RemoteTransactionContext#outflowTransaction and calling verifyEnlistment() on the resultant handle.

The appropriate location for this code is within or alongside the JBoss transaction provider. It could alternatively be factored out into a utility which is used by the JBoss transaction provider (and any other provider that needs similar behavior).