Details
-
Bug
-
Resolution: Done
-
Major
-
7.4.0.GA, 7.4.1.GA
-
False
-
False
-
+
-
undefined
Description
A transaction timeout during XA connection enlistment results in a connection leak. The behavior is verifiable with with CP6 - CP8 and also with the 7.4 GA and with CP1 1 but it is not reproducible with 7.3 as late as CP5.
The scenario is this ...
1. Start a transaction with a 2 second transaction timeout.
2. Request a connection from an XA pool (reproduced with both Oracle and H2) using a Byteman rule to inject a sleep during the "enlistment" ...
# Rule for H2
RULE org.h2.jdbcx.JdbcXAConnection.start
CLASS ^org.h2.jdbcx.JdbcXAConnection
METHOD start
IF true
DO traceStack("[BMAN] sleeping during XAResource.start ...", 10);
Thread.sleep(5000);
ENDRULE
# Rule for Oracle
RULE oracle.jdbc.xa.client.OracleXAResource.start
CLASS oracle.jdbc.xa.client.OracleXAResource
METHOD start
IF true
DO traceStack("[BMAN] sleeping during XAResource.start ...", 10);
Thread.sleep(5000);
ENDRULE
This results in a timeout (during the sleep). The application code subsequently closes the connection (implicitly using try-with-resources though explicit closure also tested and behavior is the same).
I replaced the ExampleDS with the below to convert to an XA pool
<xa-datasource jndi-name="java:jboss/datasources/ExampleDS" pool-name="ExampleDS" enabled="true" use-java-context="true" statistics-enabled="${wildfly.datasources.statistics-enabled:${wildfly.statistics-enabled:false}}">
<xa-datasource-property name="URL">
jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
</xa-datasource-property>
<driver>h2</driver>
<security>
<user-name>sa</user-name>
<password>sa</password>
</security>
</xa-datasource>
3. Rollback the failed transaction.
Subsequently, statistics show a connection in-use (e.g. during a getConnection the stats show this in the log). And, during shutdown I see
... IJ000615: Destroying active connection in pool: ExampleDS (org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@74a2f7bb)
The difference in 7.3 CP5 and earlier seems to be that enlistment actually fails after the XAResource.start but before the connection is returned
... javax.transaction.SystemException: Error enlisting resource in transaction=Local transaction (delegate=TransactionImple < ... status: ActionStatus.ABORTED >, owner=Local transaction context for provider JBoss JTA transaction provider) at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.checkEnlisted(TxConnectionListener.java:957) at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:394) at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.tx.TxConnectionManagerImpl.managedConnectionReconnected(TxConnectionManagerImpl.java:564) at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:977) at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:792) at org.jboss.ironjacamar.jdbcadapters@1.4.22.Final-redhat-00001//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151) at org.jboss.as.connector@7.3.5.GA-redhat-00001//org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64) ... Caused by: java.lang.Throwable: Failed to enlist at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.listener.TxConnectionListener$TransactionSynchronization.enlist(TxConnectionListener.java:1001) at org.jboss.ironjacamar.impl@1.4.22.Final-redhat-00001//org.jboss.jca.core.connectionmanager.listener.TxConnectionListener.enlist(TxConnectionListener.java:379)
And, because of the enlistment failure, the connection is returned to the pool with a stack like this:
2021-08-20 14:39:03,793 INFO [stdout] (ServerService Thread Pool -- 78) [BMAN] org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreConcurrentLinkedDequeManagedConnectionPool.returnConnection(SemaphoreConcurrentLinkedDequeManagedConnectionPool.java:-1)
... org.jboss.jca.core.connectionmanager.pool.AbstractPool.returnConnection(AbstractPool.java:847)
... org.jboss.jca.core.connectionmanager.AbstractConnectionManager.returnManagedConnection(AbstractConnectionManager.java:725)
... org.jboss.jca.core.connectionmanager.AbstractConnectionManager.managedConnectionDisconnected(AbstractConnectionManager.java:1113)
... org.jboss.jca.core.connectionmanager.AbstractConnectionManager.disconnectManagedConnection(AbstractConnectionManager.java:1069)
... org.jboss.jca.core.connectionmanager.AbstractConnectionManager.reconnectManagedConnection(AbstractConnectionManager.java:1046)
... org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:792)
... org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:151)
... org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
The enlistment failure does not occur with newer releases but it looks like (based on Byteman rules) enlist occurs (without failure) after the timeout with 7.3 CP6 and later and with 7.4 GA and CP1.
Attachments
Issue Links
- clones
-
-
- Verified
-
- incorporates
-
-
- Resolved
-
- is caused by
-
-
- Verified
-
- is incorporated by
-
JBEAP-24132 [GSS](7.4.z) Upgrade Ironjacamar from 1.5.3.SP2-redhat-00001 to 1.5.10.Final-redhat-00001
-
- Closed
-
