Uploaded image for project: 'Red Hat Process Automation Manager'
  1. Red Hat Process Automation Manager
  2. RHPAM-2441

KIE Server is not recovering database transactions after OOM error

    XMLWordPrintable

Details

    • Bug
    • Resolution: Cannot Reproduce
    • Major
    • None
    • 7.4.1.GA
    • Cloud
    • None
    • OpenShift 3.11, RHPAM Authoring template

    • 2019 Week 41-43 (from Okt 7), 2019 Week 44-46 (from Okt 28), 2019 Week 47-49 (from Nov 18), 2019 Week 50-52 (from Dec 9), 2020 Week 01-03 (from Dec 30), 2020 Week 04-06 (from Jan 20), 2020 Week 07-09 (from Feb 10), 2020 Week 10-12 (from Mar 2), 2020 Week 25-27 (from Jun 15), 2020 Week 28-30 (from Jul 6), 2020 Week 31-33 (from Jul 27), 2020 Week 34-36 (from Aug 17), 2020 Week 37-39 (from Sep 7), 2020 Week 40-42 (from Sep 28), 2020 Week 43-45 (from Okt 19), 2020 Week 46-48 (from Nov 9)

    Description

      When is KIE Server run Out Of Memory in OpenShift, the pod is restarted. When the pod with OOM error killed, the H2 database can't recover the transactions from the previous state. Newly started pod cannot access the DB because before it was closed.

      In the new KIE Server pod log was this exception:

      08:29:35,324 WARN  [com.arjuna.ats.jta] (ServerService Thread Pool -- 78) ARJUNA016045: attempted rollback of < formatId=131077, gtrid_length=51, bqual_length=36, tx_uid=0:ffff0a82067f:-6cdaea1a:5d9eebbd:10, node_name=m-sso-kieserver-9-2xf7x, branch_uid=0:ffff0a82067f:-6cdaea1a:5d9eebbd:1a, subordinatenodename=null, eis_name=java:/jboss/datasources/rhpam_EJBTimer > (XAResourceWrapperImpl@7915a213[xaResource=org.jboss.jca.adapters.jdbc.xa.XAManagedConnection@1e1ecdd2 pad=false overrideRmValue=null productName=H2 productVersion=@PROJECT_VERSION@ (2016-10-31) jndiName=java:/jboss/datasources/rhpam_EJBTimer]) failed with exception code ARJUNA016099: Unknown error code:0: javax.transaction.xa.XAException: Database is already closed (to disable automatic closing at VM shutdown, add ";DB_CLOSE_ON_EXIT=FALSE" to the db URL) [90121-193]
      
      Caused by: org.h2.jdbc.JdbcSQLException: Database is already closed (to disable automatic closing at VM shutdown, add ";DB_CLOSE_ON_EXIT=FALSE" to the db URL) [90121-193]
      

      This error is happening on MySQL as well:

      Caused by: java.sql.SQLException: javax.resource.ResourceException: IJ000470: You are trying to use a connection factory that has been shut down: java:/jboss/datasources/rhpam
      	at org.jboss.ironjacamar.jdbcadapters@1.4.15.Final-redhat-00001//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:159)
      	at org.jboss.as.connector@7.2.1.GA-redhat-00004//org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:64)
      	at deployment.ROOT.war//org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl.getConnection(DatasourceConnectionProviderImpl.java:122)
      	at deployment.ROOT.war//org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:35)
      	at deployment.ROOT.war//org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:106)
      	... 100 more
      

      This is not a file lock problem, but a connection/transaction state that can't recover from OOM.

      The error description might differ because it's related to the database driver, not from EAP nor KIE.

      Attachments

        Activity

          People

            mdessi-1 Massimiliano Dessi
            jakubschwan Jakub Schwan
            Jakub Schwan Jakub Schwan
            Jakub Schwan Jakub Schwan
            Xiaofeng Bai
            Votes:
            0 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: