A customer reported that 'There isn't much documentation about it on the red hat portal, but with support was able to see it fast' in relation to an issue they had recently. Escalation: https://gss--rh-escalimpl.visualforce.com/apex/Escalation_View?id=a3n2K00001MjZ9eQAF&sfdc.override=1  Here is the issue:

Original Issue:

What problem/issue/behavior are you having trouble with? What do you expect to see?
Hi, the application team is running pega in JBOSS-EAP in Azure. We have defined 3 datasources in our standalone-full.xml, which are as follows:-
[] jndi-name="java:/jdbc/AdminPegaRULES"  non-XA
[] jndi-name="java:/jdbc/PegaRULES"  non-XA
[] jndi-name="java:/jdbc/PegaFCMR"  XA

A Pega application running on these configuration is reporting the following exception in the logs with respect to the PegaFCMR datasource.
Users have reported that these issues are intermittent (only occurs the first time, then it works ok). Once they get this error, upon retrying they are able to proceed with the task.

The Error from log details will be attached.

Where are you experiencing the behavior? What environment?
QAT

When does the behavior occur? Frequency? Repeatedly? At certain times?
User has reported is intermittent, but the application has determined that occurs the first time they are doing the transaction. Then on the second time they can do same or another transaction, an is ok.

What information can you provide around timeframes and the business impact?
The business impact is high and urgent. Can you join a brigde we have with the PEGA application team?

Original Solution:

Looking at the attached log, we can track the connection object involved in the exception and see that it is in fact used several times prior to the issue inside and outside of a transaction, but eventually it fails when being used inside a transaction.

The following shows the cause of the issue:

2020-05-28 12:44:05,483 WARN [com.arjuna.ats.jta] (default task-60) ARJUNA016061: TransactionImple.enlistResource - XAResource.start returned: XAException.XAER_OUTSIDE for < formatId=131077, gtrid_length=29, bqual_length=36, tx_uid=0:ffff0a707566:66e79cd2:5ecfe513:13d, node_name=1, branch_uid=0:ffff0a707566:66e79cd2:5ecfe513:13f, subordinatenodename=null, eis_name=java:/jdbc/PegaFCMR >: com.ibm.db2.jcc.am.XaException: [jcc][t4][10401][12066][4.11.77] XA exception: XAER_OUTSIDE ERRORCODE=-4228, SQLSTATE=null

Some database vendors XA implementations don't like using an XA connection with and without a JTA transaction.

The <no-tx-separate-pools> property indicates that two connection pools are needed to isolate connections used with JTA transactions from those used without a JTA transaction.

Please add the no-tx-separate-pools property to your xa-pool configuration:

<xa-pool>
<min-pool-size>100</min-pool-size>
<max-pool-size>2000</max-pool-size>
<no-tx-separate-pools>true</no-tx-separate-pools>
</xa-pool>

Let me know if this clears up the issue. If the issue persists please provide the updated configuration for review, and new logs. At this point we do not need TRACE logging enabled anymore, so please revert before generating new logs (if the issue persists).