INFO [Tue Jan 21 12:26:14 UTC 2020] Finished migration check cycle, pausing for 30 seconds before resuming CRITICAL:osquery:Cannot query OpenShift API for "https://openshift.default.svc/api/v1/namespaces/pkremens-namespace-beta/pods/xa-load-migration-1-nt88c/log?timestamps=true&tailLines=1" Traceback (most recent call last): File "/opt/partition/queryosapi.py", line 120, in queryResult = getLog(podName, sinceTime, tailLine) File "/opt/partition/queryosapi.py", line 83, in getLog .format(osquery.OpenShiftQuery.getNameSpace(), podName, sinceTimeParam, tailLineParam)) File "/opt/partition/osquery.py", line 55, in queryApi return urllib2.urlopen(request, cafile = OpenShiftQuery.CERT_FILE_PATH).read() File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/usr/lib64/python2.7/urllib2.py", line 437, in open response = meth(req, response) File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib64/python2.7/urllib2.py", line 475, in error return self._call_chain(*args) File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 403: Forbidden INFO [Tue Jan 21 12:26:45 UTC 2020] Examining existence of living pod 'xa-load-1' INFO [Tue Jan 21 12:26:48 UTC 2020] Examining existence of living pod 'xa-load-2' INFO [Tue Jan 21 12:26:54 UTC 2020] Examining existence of living pod 'xa-load-3' INFO [Tue Jan 21 12:26:59 UTC 2020] Finished migration check cycle, pausing for 30 seconds before resuming INFO [Tue Jan 21 12:27:30 UTC 2020] Examining existence of living pod 'xa-load-1' INFO [Tue Jan 21 12:27:30 UTC 2020] Examining existence of living pod 'xa-load-2' INFO [Tue Jan 21 12:27:32 UTC 2020] Examining existence of living pod 'xa-load-3' INFO [Tue Jan 21 12:27:32 UTC 2020] Finished migration check cycle, pausing for 30 seconds before resuming INFO [Tue Jan 21 12:28:02 UTC 2020] Examining existence of living pod 'xa-load-1' INFO [Tue Jan 21 12:28:02 UTC 2020] Examining existence of living pod 'xa-load-2' INFO [Tue Jan 21 12:28:03 UTC 2020] Examining existence of living pod 'xa-load-3' INFO Processing broker(0): jbamqa-amq=JBAMQA INFO Generating resource adapter configuration for service: jbamqa-amq (amq) generating object config for queue/xaQueue1 INFO Processing broker(1): jbamqb-amq=JBAMQB INFO Generating resource adapter configuration for service: jbamqb-amq (amq) generating object config for queue/xaQueue2 WARN No password defined for JGroups cluster. AUTH protocol will be disabled. Please define JGROUPS_CLUSTER_PASSWORD. WARN Environment variable OPENSHIFT_KUBE_PING_NAMESPACE undefined. Clustering will be unavailable. Please refer to the documentation for configuration. INFO Configuring JGroups discovery protocol to kubernetes.KUBE_PING INFO Configuring JGroups cluster traffic encryption protocol to SYM_ENCRYPT. WARN Detected missing JGroups encryption configuration, the communication within the cluster WILL NOT be encrypted. Using PicketBox SSL configuration. WARN Missing SSO_URL. Unable to properly configure SSO-enabled applications INFO Access log is disabled, ignoring configuration. Running jboss-eap-7-tech-preview/eap73-openjdk8-openshift-rhel7 image, version 7.3.0.Beta -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=100m -XX:+ExitOnOutOfMemoryError ========================================================================= JBoss Bootstrap Environment JBOSS_HOME: /opt/eap JAVA: /usr/lib/jvm/java-1.8.0/bin/java JAVA_OPTS: -server -verbose:gc -Xloggc:"/opt/eap/standalone/log/gc.log" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,jdk.nashorn.api,com.sun.crypto.provider -Djava.awt.headless=true -javaagent:/opt/jboss/container/jolokia/jolokia.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -Xbootclasspath/p:/opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.11.Final-redhat-00001.jar:/opt/eap/modules/system/layers/base/org/glassfish/javax/json/main/javax.json-1.1.2.redhat-1.jar:/opt/eap/modules/system/layers/base/javax/json/api/main/javax.json-api-1.1.2.redhat-1.jar:/opt/eap/modules/system/layers/base/org/wildfly/common/main/wildfly-common-1.5.1.Final-redhat-00001.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dsun.util.logging.disableCallerCheck=true -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -Dorg.apache.activemq.SERIALIZABLE_PACKAGES=org.jboss.xpaas.broker.model ========================================================================= OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N 12:28:04,709 INFO [org.jboss.modules] (main) JBoss Modules version 1.9.1.Final-redhat-00001 12:28:05,016 INFO [org.jboss.msc] (main) JBoss MSC version 1.4.8.Final-redhat-00001 12:28:05,027 INFO [org.jboss.threads] (main) JBoss Threads version 2.3.3.Final-redhat-00001 12:28:05,155 INFO [org.jboss.as] (MSC service thread 1-1) WFLYSRV0049: JBoss EAP 7.3.0.Beta (WildFly Core 9.0.2.Final-redhat-00001) starting 12:28:05,156 DEBUG [org.jboss.as.config] (MSC service thread 1-1) Configured system properties: [Standalone] = awt.toolkit = sun.awt.X11.XToolkit com.arjuna.ats.arjuna.common.RecoveryEnvironmentBean.periodicRecoveryPeriod = 1 com.arjuna.ats.arjuna.common.RecoveryEnvironmentBean.recoveryBackoffPeriod = 1 com.arjuna.ats.jta.common.JTAEnvironmentBean.orphanSafetyInterval = 1 ejb.resource-adapter-name = activemq-rar.rar file.encoding = ANSI_X3.4-1968 file.encoding.pkg = sun.io file.separator = / java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment java.awt.headless = true java.awt.printerjob = sun.print.PSPrinterJob java.class.path = /opt/eap/jboss-modules.jar:/opt/jboss/container/jolokia/jolokia.jar java.class.version = 52.0 java.endorsed.dirs = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/endorsed java.ext.dirs = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/ext:/usr/java/packages/lib/ext java.home = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre java.io.tmpdir = /tmp java.library.path = /usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib java.net.preferIPv4Stack = true java.runtime.name = OpenJDK Runtime Environment java.runtime.version = 1.8.0_222-b10 java.security.egd = file:/dev/./urandom java.specification.name = Java Platform API Specification java.specification.vendor = Oracle Corporation java.specification.version = 1.8 java.util.logging.manager = org.jboss.logmanager.LogManager java.vendor = Oracle Corporation java.vendor.url = http://java.oracle.com/ java.vendor.url.bug = http://bugreport.sun.com/bugreport/ java.version = 1.8.0_222 java.vm.info = mixed mode java.vm.name = OpenJDK 64-Bit Server VM java.vm.specification.name = Java Virtual Machine Specification java.vm.specification.vendor = Oracle Corporation java.vm.specification.version = 1.8 java.vm.vendor = Oracle Corporation java.vm.version = 25.222-b10 javax.management.builder.initial = org.jboss.as.jmx.PluggableMBeanServerBuilder jboss.bind.address = 10.128.1.124 jboss.bind.address.management = 127.0.0.1 jboss.bind.address.private = 10.128.1.124 jboss.home.dir = /opt/eap jboss.host.name = xa-load-migration-1-nt88c jboss.messaging.host = 10.128.1.124 jboss.modules.dir = /opt/eap/modules jboss.modules.system.pkgs = org.jboss.logmanager,jdk.nashorn.api,com.sun.crypto.provider jboss.node.name = xa-load-3 jboss.qualified.host.name = xa-load-migration-1-nt88c jboss.server.base.dir = /opt/eap/standalone jboss.server.config.dir = /opt/eap/standalone/configuration jboss.server.data.dir = /opt/eap/standalone/partitioned_data/xa-load-3/serverData jboss.server.deploy.dir = /opt/eap/standalone/partitioned_data/xa-load-3/serverData/content jboss.server.log.dir = /opt/eap/standalone/log jboss.server.name = xa-load-migration-1-nt88c jboss.server.persist.config = true jboss.server.temp.dir = /opt/eap/standalone/tmp line.separator = logging.configuration = file:/opt/eap/standalone/configuration/logging.properties module.path = /opt/eap/modules org.apache.activemq.SERIALIZABLE_PACKAGES = org.jboss.xpaas.broker.model org.jboss.boot.log.file = /opt/eap/standalone/log/server.log org.jboss.resolver.warning = true os.arch = amd64 os.name = Linux os.version = 4.18.0-147.3.1.el8_1.x86_64 path.separator = : sun.arch.data.model = 64 sun.boot.class.path = /opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.11.Final-redhat-00001.jar:/opt/eap/modules/system/layers/base/org/glassfish/javax/json/main/javax.json-1.1.2.redhat-1.jar:/opt/eap/modules/system/layers/base/javax/json/api/main/javax.json-api-1.1.2.redhat-1.jar:/opt/eap/modules/system/layers/base/org/wildfly/common/main/wildfly-common-1.5.1.Final-redhat-00001.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/resources.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/rt.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/sunrsasign.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/jsse.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/jce.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/charsets.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/jfr.jar:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/classes sun.boot.library.path = /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b10-0.el7_6.x86_64/jre/lib/amd64 sun.cpu.endian = little sun.cpu.isalist = sun.io.unicode.encoding = UnicodeLittle sun.java.command = /opt/eap/jboss-modules.jar -mp /opt/eap/modules org.jboss.as.standalone -Djboss.home.dir=/opt/eap -Djboss.server.base.dir=/opt/eap/standalone -c standalone-openshift.xml -bmanagement 127.0.0.1 -Djboss.server.data.dir=/opt/eap/standalone/partitioned_data/xa-load-3/serverData -Dcom.arjuna.ats.arjuna.common.RecoveryEnvironmentBean.recoveryBackoffPeriod=1 -Dcom.arjuna.ats.arjuna.common.RecoveryEnvironmentBean.periodicRecoveryPeriod=1 -Dcom.arjuna.ats.jta.common.JTAEnvironmentBean.orphanSafetyInterval=1 -b 10.128.1.124 -bprivate 10.128.1.124 -Djboss.node.name=xa-load-3 -Djboss.messaging.host=10.128.1.124 -Dejb.resource-adapter-name=activemq-rar.rar sun.java.launcher = SUN_STANDARD sun.jnu.encoding = ANSI_X3.4-1968 sun.management.compiler = HotSpot 64-Bit Tiered Compilers sun.os.patch.level = unknown sun.util.logging.disableCallerCheck = true user.country = US user.dir = /home/jboss user.home = /home/jboss user.language = en user.name = jboss user.timezone = UTC 12:28:05,157 DEBUG [org.jboss.as.config] (MSC service thread 1-1) VM Arguments: -D[Standalone] -verbose:gc -Xloggc:/opt/eap/standalone/log/gc.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=3M -XX:-TraceClassUnloading -Xms1303m -Xmx1303m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,jdk.nashorn.api,com.sun.crypto.provider -Djava.awt.headless=true -javaagent:/opt/jboss/container/jolokia/jolokia.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -Xbootclasspath/p:/opt/eap/jboss-modules.jar:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.11.Final-redhat-00001.jar:/opt/eap/modules/system/layers/base/org/glassfish/javax/json/main/javax.json-1.1.2.redhat-1.jar:/opt/eap/modules/system/layers/base/javax/json/api/main/javax.json-api-1.1.2.redhat-1.jar:/opt/eap/modules/system/layers/base/org/wildfly/common/main/wildfly-common-1.5.1.Final-redhat-00001.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dsun.util.logging.disableCallerCheck=true -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -Dorg.apache.activemq.SERIALIZABLE_PACKAGES=org.jboss.xpaas.broker.model -Dorg.jboss.boot.log.file=/opt/eap/standalone/log/server.log -Dlogging.configuration=file:/opt/eap/standalone/configuration/logging.properties 12:28:06,373 INFO [stdout] (JolokiaStart) I> No access restrictor found, access to any MBean is allowed 12:28:06,408 INFO [stdout] (JolokiaStart) Jolokia: Agent started with URL https://10.128.1.124:8778/jolokia/ 12:28:06,453 INFO [org.wildfly.security] (ServerService Thread Pool -- 22) ELY00001: WildFly Elytron version 1.9.1.Final-redhat-00001 12:28:08,120 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http) 12:28:08,127 WARN [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0035: No security realm or http server authentication defined for http management service; all access will be unrestricted. 12:28:08,138 INFO [org.xnio] (MSC service thread 1-2) XNIO version 3.7.2.Final-redhat-00001 12:28:08,149 INFO [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.7.2.Final-redhat-00001 12:28:08,235 INFO [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 5.0.12.Final-redhat-00001 12:28:08,358 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 52) WFLYCLINF0001: Activating Infinispan subsystem. 12:28:08,365 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 53) WFLYIO001: Worker 'default' has auto-configured to 2 core threads with 16 task threads based on your 1 available processors 12:28:08,390 INFO [org.wildfly.extension.microprofile.config.smallrye._private] (ServerService Thread Pool -- 63) WFLYCONF0001: Activating WildFly MicroProfile Config Subsystem 12:28:08,395 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 57) WFLYCLJG0001: Activating JGroups subsystem. JGroups version 4.0.19 12:28:08,397 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 44) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.4) 12:28:08,406 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 67) WFLYNAM0001: Activating Naming Subsystem 12:28:08,442 INFO [org.jboss.as.jaxrs] (ServerService Thread Pool -- 54) WFLYRS0016: RESTEasy version 3.7.0.Final-redhat-00001 12:28:08,463 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 60) WFLYJSF0007: Activated the following JSF Implementations: [main] 12:28:08,515 INFO [org.wildfly.extension.microprofile.health.smallrye] (ServerService Thread Pool -- 64) WFLYHEALTH0001: Activating Eclipse MicroProfile Health Subsystem 12:28:08,533 INFO [org.wildfly.extension.microprofile.metrics.smallrye] (ServerService Thread Pool -- 65) WFLYMETRICS0001: Activating Eclipse MicroProfile Metrics Subsystem 12:28:08,563 INFO [org.jboss.as.connector] (MSC service thread 1-1) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.4.16.Final-redhat-00001) 12:28:08,578 INFO [org.jboss.as.security] (ServerService Thread Pool -- 73) WFLYSEC0002: Activating Security Subsystem 12:28:08,620 INFO [org.jboss.as.webservices] (ServerService Thread Pool -- 78) WFLYWS0002: Activating WebServices Extension 12:28:08,669 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 44) WFLYJCA0005: Deploying non-JDBC-compliant driver class com.mysql.cj.jdbc.Driver (version 8.0) 12:28:08,812 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 44) WFLYJCA0005: Deploying non-JDBC-compliant driver class org.postgresql.Driver (version 42.2) 12:28:08,879 INFO [org.jboss.as.security] (MSC service thread 1-1) WFLYSEC0001: Current PicketBox version=5.0.3.Final-redhat-3 12:28:09,099 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0014: Creating file handler for path '/opt/eap/welcome-content' with options [directory-listing: 'false', follow-symlink: 'false', case-sensitive: 'true', safe-symlink-paths: '[]'] 12:28:09,104 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0003: Undertow 2.0.21.Final-redhat-00001 starting 12:28:09,157 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0018: Started Driver service with driver-name = h2 12:28:09,178 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0018: Started Driver service with driver-name = mysql 12:28:09,259 INFO [io.smallrye.metrics] (MSC service thread 1-2) Converted [2] config entries and added [4] replacements 12:28:09,260 INFO [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0482: Strict pool mdb-strict-max-pool is using a max instance size of 4 (per class), which is derived from the number of CPUs on this host. 12:28:09,261 INFO [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0481: Strict pool slsb-strict-max-pool is using a max instance size of 16 (per class), which is derived from thread worker pool sizing. 12:28:09,266 INFO [io.smallrye.metrics] (MSC service thread 1-2) Converted [3] config entries and added [14] replacements 12:28:09,267 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = postgresql 12:28:09,277 INFO [org.jboss.as.naming] (MSC service thread 1-1) WFLYNAM0003: Starting Naming Service 12:28:09,311 INFO [org.jboss.as.mail.extension] (MSC service thread 1-1) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default] 12:28:09,348 WARN [org.jboss.as.remoting] (MSC service thread 1-1) ****** All authentication is ANONYMOUS for org.jboss.as.remoting.RemotingHttpUpgradeService 12:28:09,542 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0012: Started server default-server. 12:28:09,558 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow AJP listener ajp listening on 0.0.0.0:8009 12:28:09,561 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0018: Host default-host starting 12:28:09,574 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow HTTP listener default listening on 0.0.0.0:8080 12:28:09,623 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 80) MODCLUSTER000001: Initializing mod_cluster version 1.4.1.Final-redhat-00001 12:28:09,644 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 80) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364 12:28:09,669 INFO [org.jboss.as.ejb3] (MSC service thread 1-2) WFLYEJB0493: EJB subsystem suspension complete 12:28:09,814 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS] 12:28:09,852 INFO [org.jboss.as.patching] (MSC service thread 1-2) WFLYPAT0050: JBoss EAP cumulative patch ID is: base, one-off patches include: none 12:28:09,953 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0027: Starting deployment of "ROOT.war" (runtime-name: "ROOT.war") 12:28:09,960 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0027: Starting deployment of "activemq-rar.rar" (runtime-name: "activemq-rar.rar") 12:28:09,964 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-1) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/eap/standalone/deployments 12:28:10,016 INFO [org.jboss.ws.common.management] (MSC service thread 1-2) JBWS022052: Starting JBossWS 5.3.0.Final-redhat-00001 (Apache CXF 3.3.2.redhat-00001) 12:28:10,636 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fa, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6e34b48 > 12:28:10,642 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:204, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@16819ed > 12:28:10,643 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fa, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,644 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:204, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,659 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:201, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@77811e29 > 12:28:10,659 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:207, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@36e7c645 > 12:28:10,659 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:201, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,660 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:207, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,666 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:215, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5c2aaeb2 > 12:28:10,667 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@12c8d868 > 12:28:10,668 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:215, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,668 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,678 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20e, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6dc7fc4e > 12:28:10,680 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:218, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@659aca5d > 12:28:10,681 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20e, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,681 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:218, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,689 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:222, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6c4cb762 > 12:28:10,690 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:225, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3813291b > 12:28:10,690 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:222, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,690 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:225, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,696 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22c, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6a6487db > 12:28:10,696 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:236, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@599f60e8 > 12:28:10,696 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22c, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,697 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:236, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,703 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:233, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@50613b1e > 12:28:10,703 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:239, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@719b9bf8 > 12:28:10,704 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:233, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,704 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:239, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,709 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24a, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3f8f804f > 12:28:10,710 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:251, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@44555e15 > 12:28:10,711 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24a, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,711 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:251, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,717 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:240, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1e08750b > 12:28:10,718 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:245, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@70432529 > 12:28:10,718 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:240, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,718 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:245, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,724 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:26b, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7bbde7b4 > 12:28:10,725 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:284, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4079463e > 12:28:10,725 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:26b, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,725 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:284, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,731 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:257, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1d84c317 > 12:28:10,732 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27d, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7407c7e3 > 12:28:10,733 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:257, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,734 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27d, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,741 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:281, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@60131e90 > 12:28:10,742 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28a, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6ad1ec1a > 12:28:10,742 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:281, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,742 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28a, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,750 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:271, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5176162c > 12:28:10,751 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:287, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1b84eb3e > 12:28:10,751 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:271, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,751 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:287, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,757 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:265, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@639b9140 > 12:28:10,758 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@64f0015 > 12:28:10,758 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:265, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,759 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,764 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:277, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@67e9a186 > 12:28:10,765 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:293, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5b087b5e > 12:28:10,765 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:277, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,766 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:293, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,772 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:260, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6426e0ee > 12:28:10,772 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28f, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1a971bd7 > 12:28:10,772 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:260, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:10,773 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28f, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:10,829 INFO [org.jboss.weld.deployer] (MSC service thread 1-1) WFLYWELD0003: Processing weld deployment ROOT.war 12:28:10,956 INFO [org.hibernate.validator.internal.util.Version] (MSC service thread 1-1) HV000001: Hibernate Validator 6.0.16.Final-redhat-00001 12:28:11,721 INFO [org.jboss.weld.Version] (MSC service thread 1-1) WELD-000900: 3.1.1 (redhat) 12:28:12,226 INFO [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-2) IJ020001: Required license terms for file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,250 INFO [org.jgroups.protocols.kubernetes.KUBE_PING] (ServerService Thread Pool -- 80) namespace default set; clustering enabled 12:28:12,296 WARN [org.jboss.as.connector.deployers.RADeployer] (MSC service thread 1-2) IJ020017: Invalid archive: file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,301 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (MSC service thread 1-2) IJ020001: Required license terms for file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,334 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-2) WFLYJCA0007: Registered connection factory java:/jbamqa-amq/ConnectionFactory 12:28:12,338 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-2) WFLYJCA0006: Registered admin object at java:/queue/xaQueue1 12:28:12,350 WARN [org.jboss.as.connector.deployers.RaXmlDeployer] (MSC service thread 1-2) IJ020017: Invalid archive: file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,363 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (MSC service thread 1-1) IJ020001: Required license terms for file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,358 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (MSC service thread 1-2) IJ020002: Deployed: file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,366 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0007: Registered connection factory java:/jbamqb-amq/ConnectionFactory 12:28:12,371 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0006: Registered admin object at java:/queue/xaQueue2 12:28:12,374 WARN [org.jboss.as.connector.deployers.RaXmlDeployer] (MSC service thread 1-1) IJ020017: Invalid archive: file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,375 INFO [org.jboss.as.connector.deployers.RaXmlDeployer] (MSC service thread 1-1) IJ020002: Deployed: file:/opt/eap/standalone/tmp/vfs/temp/temp29699ae2153c002e/content-b68e8c8695645402/contents/ 12:28:12,377 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0002: Bound JCA ConnectionFactory [java:/jbamqa-amq/ConnectionFactory] 12:28:12,376 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-2) WFLYJCA0002: Bound JCA AdminObject [java:/queue/xaQueue1] 12:28:12,379 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-2) WFLYJCA0002: Bound JCA AdminObject [java:/queue/xaQueue2] 12:28:12,379 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0002: Bound JCA ConnectionFactory [java:/jbamqb-amq/ConnectionFactory] 12:28:12,556 INFO [io.smallrye.metrics] (MSC service thread 1-2) MicroProfile: Metrics activated 12:28:13,380 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fa, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@b25552b > 12:28:13,591 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:204, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1c1964ae > 12:28:13,694 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fa, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > Checking readiness probe status for server start. 12:28:13,882 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1f6, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:204, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:14,018 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:201, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5ea87c56 > { "probe.eap.dmr.EapProbe": "Error sending probe request: No JSON object could be decoded", "probe.eap.dmr.HealthCheckProbe": "Error sending probe request: No JSON object could be decoded" } Sleeping 5 seconds before retrying readiness probe. 12:28:14,124 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:207, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3223ef96 > 12:28:14,433 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:201, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:14,632 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:1fd, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:207, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:14,818 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:215, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@25859f6c > 12:28:14,906 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@41b1a64d > 12:28:14,979 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:215, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:15,178 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:211, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:15,300 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20e, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@43a4b2fb > 12:28:15,426 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:218, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@159d821e > 12:28:15,597 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20e, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:15,711 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:20a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:218, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:15,758 WARN [org.jgroups.protocols.kubernetes.KUBE_PING] (ServerService Thread Pool -- 80) failed getting JSON response from Kubernetes Client[masterUrl=https://172.30.0.1:443/api/v1, headers={Authorization=#MASKED:887#}, connectTimeout=5000, readTimeout=30000, operationAttempts=3, operationSleep=1000, streamProvider=org.jgroups.protocols.kubernetes.stream.InsecureStreamProvider@64995a9f] for cluster [ee], namespace [default], labels [null]; encountered [java.lang.Exception: 3 attempt(s) with a 1000ms sleep to execute [OpenStream] failed. Last failure was [java.io.IOException: Server returned HTTP response code: 403 for URL: https://172.30.0.1:443/api/v1/namespaces/default/pods]] 12:28:15,799 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:222, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1dedd15b > 12:28:15,898 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:225, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@69c3341b > 12:28:16,036 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:222, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:16,109 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:21e, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:225, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:16,243 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22c, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@78576f7 > 12:28:16,381 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:236, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4f13d11e > 12:28:16,473 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22c, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:16,625 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:228, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:236, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:16,709 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:233, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2f9f48d5 > 12:28:16,858 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:239, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqb-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1a35ab5e > 12:28:16,926 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:233, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:17,004 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:22f, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:239, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > 12:28:17,175 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24a, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3e93e11c > 12:28:17,269 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24a, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:17,270 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:243,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:251] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:17,274 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:243, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:251, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@65419c8d[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@7707d399,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-119:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-118:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:17,453 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:240, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6e9f8cc7 > 12:28:17,545 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:240, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:17,547 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:23c,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:245] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:17,547 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:23c, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:245, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@29963cab[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@4b948e94,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-127:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-126:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:17,778 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:26b, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5eed6938 > 12:28:17,911 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:26b, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:17,911 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:254,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:284] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:17,911 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:254, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:284, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@53d80050[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@1bee0ea3,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-135:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-134:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:18,006 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:257, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@798034d > 12:28:18,160 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:257, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:18,163 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:24d,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:27d] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:18,163 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:24d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27d, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@4b220a54[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@1c3a98c3,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-143:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-142:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:18,242 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:281, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@e50b829 > 12:28:18,329 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:281, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:18,330 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:27a,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:28a] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:18,331 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:27a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28a, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@4a7d954c[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@2116e036,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-151:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-150:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:18,512 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:271, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@139e33ed > 12:28:18,592 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:271, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:18,592 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:268,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:287] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:18,595 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:268, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:287, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@88d5f96[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@411a6d1c,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-159:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-158:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:18,709 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:265, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4816f94e > 12:28:18,762 INFO [org.jgroups.protocols.pbcast.GMS] (ServerService Thread Pool -- 80) xa-load-3: no members discovered after 6331 ms: creating cluster as first member 12:28:18,817 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:265, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:18,818 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:25d,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:28b] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:18,820 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25d, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28b, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@3afbf309[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@15054b5e,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-167:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-166:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:19,012 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:277, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@796f46b4 > Checking readiness probe status for server start. 12:28:19,211 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:277, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:19,211 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:269,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:293] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:19,214 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:269, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:293, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@7b34e5fb[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@6a90acab,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-175:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-174:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:19,264 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Red Hat Data Grid 'Infinity Minus ONE +2' 9.4.15.Final-redhat-00001 12:28:20,199 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016037: Could not find new XAResource to use for recovering non-serializable XAResource XAResourceRecord < resource:null, txid:< formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:260, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory >, heuristic: TwoPhaseOutcome.FINISH_OK, product: ActiveMQ/5.11.0.redhat-630371, jndiName: java:/jbamqa-amq/ConnectionFactory com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4594a7ca > { "probe.eap.dmr.EapProbe": "Error sending probe request: No JSON object could be decoded", "probe.eap.dmr.HealthCheckProbe": "Error sending probe request: No JSON object could be decoded" } Sleeping 5 seconds before retrying readiness probe. 12:28:20,488 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016038: No XAResource to recover < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:260, subordinatenodename=null, eis_name=java:/jbamqa-amq/ConnectionFactory > 12:28:20,488 WARN [org.apache.activemq.TransactionContext] (Periodic Recovery) commit of: XID:[131077,globalId=0:ffff0a80017f:7ae4880a:5e26ee07:25a,branchId=0:ffff0a80017f:7ae4880a:5e26ee07:28f] failed with: org.apache.activemq.ConnectionClosedException: The connection is already closed: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) 12:28:20,490 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=37, bqual_length=36, tx_uid=0:ffff0a80017f:7ae4880a:5e26ee07:25a, node_name=xa-load-3, branch_uid=0:ffff0a80017f:7ae4880a:5e26ee07:28f, subordinatenodename=null, eis_name=java:/jbamqb-amq/ConnectionFactory > (XAResourceWrapperImpl@1af8f9b9[xaResource=[org.apache.activemq.ra.ActiveMQManagedConnection$1@d691bec,TransactionContext{transactionId=null,connection=ActiveMQConnection {id=ID:xa-load-migration-1-nt88c-35941-1579609693023-183:1,clientId=ID:xa-load-migration-1-nt88c-35941-1579609693023-182:1,started=false}}] pad=false overrideRmValue=false productName=ActiveMQ productVersion=5.11.0.redhat-630371 jndiName=java:/jbamqb-amq/ConnectionFactory]) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: The connection is already closed at org.apache.activemq.TransactionContext.toXAException(TransactionContext.java:813) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:632) at org.apache.activemq.ra.LocalAndXATransaction.commit(LocalAndXATransaction.java:92) at org.jboss.jca.core.tx.jbossts.XAResourceWrapperImpl.commit(XAResourceWrapperImpl.java:107) at com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: org.apache.activemq.ConnectionClosedException: The connection is already closed at org.apache.activemq.ActiveMQConnection.checkClosed(ActiveMQConnection.java:1454) at org.apache.activemq.ActiveMQConnection.checkClosedOrFailed(ActiveMQConnection.java:1441) at org.apache.activemq.TransactionContext.commit(TransactionContext.java:589) ... 12 more 12:28:20,654 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000078: Starting JGroups channel ee 12:28:20,671 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups channel ee 12:28:20,704 INFO [org.infinispan.CLUSTER] (MSC service thread 1-1) ISPN000094: Received new cluster view for channel ee: [xa-load-3|0] (1) [xa-load-3] 12:28:20,704 INFO [org.infinispan.CLUSTER] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel ee: [xa-load-3|0] (1) [xa-load-3] 12:28:20,720 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000079: Channel ee local address is xa-load-3, physical addresses are [10.128.1.124:7600] 12:28:20,721 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Channel ee local address is xa-load-3, physical addresses are [10.128.1.124:7600] 12:28:20,793 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000078: Starting JGroups channel ee 12:28:20,794 INFO [org.infinispan.CLUSTER] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel ee: [xa-load-3|0] (1) [xa-load-3] 12:28:20,794 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups channel ee 12:28:20,794 INFO [org.infinispan.CLUSTER] (MSC service thread 1-1) ISPN000094: Received new cluster view for channel ee: [xa-load-3|0] (1) [xa-load-3] 12:28:20,798 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000079: Channel ee local address is xa-load-3, physical addresses are [10.128.1.124:7600] 12:28:20,798 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Channel ee local address is xa-load-3, physical addresses are [10.128.1.124:7600] 12:28:21,216 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 80) WFLYCLINF0002: Started client-mappings cache from ejb container 12:28:22,010 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 80) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.jboss.as.quickstarts.xa.RestApplication$Proxy$_$$_WeldClientProxy 12:28:22,091 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 80) WFLYUT0021: Registered web context: '/' for server 'default-server' 12:28:22,213 INFO [org.jboss.as.server] (ServerService Thread Pool -- 45) WFLYSRV0010: Deployed "ROOT.war" (runtime-name : "ROOT.war") 12:28:22,216 INFO [org.jboss.as.server] (ServerService Thread Pool -- 45) WFLYSRV0010: Deployed "activemq-rar.rar" (runtime-name : "activemq-rar.rar") 12:28:22,303 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server 12:28:22,312 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://127.0.0.1:9990/management 12:28:22,312 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0054: Admin console is not enabled 12:28:22,312 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.3.0.Beta (WildFly Core 9.0.2.Final-redhat-00001) started in 17970ms - Started 612 of 908 services (526 services are lazy, passive or on-demand) Checking readiness probe status for server start. Tue Jan 21 12:28:25 UTC 2020: Server started, checking for transactions Tue Jan 21 12:28:30 UTC 2020: Executing synchronous recovery scan for a first time DONE Tue Jan 21 12:28:32 UTC 2020: Executing synchronous recovery scan for a second time DONE Tue Jan 21 12:28:33 UTC 2020: Synchronous recovery scans finished for the first and the second time Tue Jan 21 12:28:36 UTC 2020: No transactions to recover 12:28:37,793 INFO [org.jboss.as.server] (management-handler-thread - 1) WFLYSRV0272: Suspending server 12:28:37,805 INFO [org.jboss.as.ejb3] (management-handler-thread - 1) WFLYEJB0493: EJB subsystem suspension complete 12:28:37,811 INFO [org.jboss.as.server] (Management Triggered Shutdown) WFLYSRV0241: Shutting down in response to management operation 'shutdown' 12:28:37,832 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = mysql 12:28:37,878 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0019: Stopped Driver service with driver-name = postgresql 12:28:37,888 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000080: Disconnecting JGroups channel ee 12:28:37,934 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000080: Disconnecting JGroups channel ee 12:28:37,937 INFO [org.jboss.as.mail.extension] (MSC service thread 1-1) WFLYMAIL0002: Unbound mail session [java:jboss/mail/Default] 12:28:37,941 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000080: Disconnecting JGroups channel ee 12:28:37,953 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/jbamqb-amq/ConnectionFactory] 12:28:37,979 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 80) WFLYUT0022: Unregistered web context: '/' from server 'default-server' 12:28:38,008 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0019: Host default-host stopping 12:28:38,009 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 19) MODCLUSTER000002: Initiating mod_cluster shutdown 12:28:38,041 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-1) WFLYJCA0010: Unbound data source [java:jboss/datasources/ExampleDS] 12:28:38,042 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0011: Unbound JCA AdminObject [java:/queue/xaQueue2] 12:28:38,041 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-2) WFLYJCA0011: Unbound JCA AdminObject [java:/queue/xaQueue1] 12:28:38,044 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0011: Unbound JCA ConnectionFactory [java:/jbamqa-amq/ConnectionFactory] 12:28:38,056 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0019: Stopped Driver service with driver-name = h2 12:28:38,131 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 19) WFLYCLINF0003: Stopped client-mappings cache from ejb container 12:28:38,137 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment ROOT.war (runtime-name: ROOT.war) in 316ms 12:28:38,142 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow HTTP listener default suspending 12:28:38,147 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow HTTP listener default stopped, was bound to 0.0.0.0:8080 12:28:38,149 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0008: Undertow AJP listener ajp suspending 12:28:38,150 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0007: Undertow AJP listener ajp stopped, was bound to 0.0.0.0:8009 12:28:38,160 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000080: Disconnecting JGroups channel ee 12:28:38,165 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0004: Undertow 2.0.21.Final-redhat-00001 stopping 12:28:38,177 INFO [org.jboss.as.server.deployment] (MSC service thread 1-2) WFLYSRV0028: Stopped deployment activemq-rar.rar (runtime-name: activemq-rar.rar) in 361ms 12:28:38,203 INFO [org.jboss.as] (MSC service thread 1-1) WFLYSRV0050: JBoss EAP 7.3.0.Beta (WildFly Core 9.0.2.Final-redhat-00001) stopped in 387ms Tue Jan 21 12:28:38 UTC 2020: Finished, recovery terminated successfully INFO [Tue Jan 21 12:28:38 UTC 2020] Migration of pod xa-load-3 terminated with status 0 (T) INFO [Tue Jan 21 12:28:38 UTC 2020] Migration finished for pod name xa-load-3, the record removed by recovery pod xa-load-migration-1-nt88c INFO [Tue Jan 21 12:28:38 UTC 2020] Finished migration check cycle, pausing for 30 seconds before resuming