WARN Configuration of an embedded messaging broker within the appserver is enabled but is not recommended. Support for such a configuration will be removed in a future release. INFO If you are not configuring messaging destinations, to disable configuring an embedded messaging broker set the DISABLE_EMBEDDED_JMS_BROKER environment variable to true. INFO Configuring JGroups discovery protocol to dns.DNS_PING INFO Configuring JGroups cluster traffic encryption protocol to SYM_ENCRYPT. WARN Detected missing JGroups encryption configuration, the communication within the cluster WILL NOT be encrypted. Using PicketBox SSL configuration. WARN Missing SSO_URL. Unable to properly configure SSO-enabled applications INFO Access log is disabled, ignoring configuration. INFO Running jboss-eap-7-tech-preview/eap-cd-openshift-rhel8 image, version 17.0 -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:MaxMetaspaceSize=100m -XX:+ExitOnOutOfMemoryError ========================================================================= JBoss Bootstrap Environment JBOSS_HOME: /opt/eap JAVA: /usr/lib/jvm/java-11/bin/java JAVA_OPTS: -server -Xms128m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,jdk.nashorn.api,com.sun.crypto.provider -Djava.awt.headless=true -javaagent:/opt/jboss/container/jolokia/jolokia.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -Xbootclasspath/a:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.13.Final.jar:/opt/eap/modules/system/layers/base/org/glassfish/javax/json/main/javax.json-1.1.2.jar:/opt/eap/modules/system/layers/base/javax/json/api/main/javax.json-api-1.1.2.jar:/opt/eap/modules/system/layers/base/org/wildfly/common/main/wildfly-common-1.5.1.Final.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dsun.util.logging.disableCallerCheck=true -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -Dwildfly.config.url=${JBOSS_HOME}/standalone/configuration/custom-config.xml -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED --add-exports=jdk.unsupported/sun.reflect=ALL-UNNAMED ========================================================================= Listening for transport dt_socket at address: 8787 08:45:26,065 INFO [org.jboss.modules] (main) JBoss Modules version 1.9.1.Final Aug 06, 2019 8:45:26 AM java.lang.System$LoggerFinder lambda$accessProvider$0 WARNING: Failed to instantiate LoggerFinder provider; Using default. WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.jolokia.util.ClassUtil (file:/opt/jboss/container/jolokia/jolokia-jvm-1.6.2.redhat-00002-agent.jar) to constructor sun.security.x509.X500Name(java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String,java.lang.String) WARNING: Please consider reporting this to the maintainers of org.jolokia.util.ClassUtil WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release 08:45:27,235 INFO [org.jboss.msc] (main) JBoss MSC version 1.4.8.Final 08:45:27,243 INFO [org.jboss.threads] (main) JBoss Threads version 2.3.3.Final 08:45:27,300 INFO [stdout] (JolokiaStart) I> No access restrictor found, access to any MBean is allowed 08:45:27,313 INFO [stdout] (JolokiaStart) Jolokia: Agent started with URL https://10.128.1.113:8778/jolokia/ 08:45:27,478 INFO [org.jboss.as] (MSC service thread 1-1) WFLYSRV0049: WildFly Full 18.0.0.Beta1-SNAPSHOT (WildFly Core 10.0.0.Beta2) starting 08:45:27,479 DEBUG [org.jboss.as.config] (MSC service thread 1-1) Configured system properties: [Standalone] = awt.toolkit = sun.awt.X11.XToolkit file.encoding = ANSI_X3.4-1968 file.separator = / java.awt.graphicsenv = sun.awt.X11GraphicsEnvironment java.awt.headless = true java.awt.printerjob = sun.print.PSPrinterJob java.class.path = /opt/eap/jboss-modules.jar java.class.version = 55.0 java.home = /usr/lib/jvm/java-11-openjdk-11.0.4.11-0.el8_0.x86_64 java.io.tmpdir = /tmp java.library.path = /usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib java.net.preferIPv4Stack = true java.runtime.name = OpenJDK Runtime Environment java.runtime.version = 11.0.4+11-LTS java.security.egd = file:/dev/./urandom java.specification.name = Java Platform API Specification java.specification.vendor = Oracle Corporation java.specification.version = 11 java.util.logging.manager = org.jboss.logmanager.LogManager java.vendor = Oracle Corporation java.vendor.url = http://java.oracle.com/ java.vendor.url.bug = http://bugreport.java.com/bugreport/ java.vendor.version = 18.9 java.version = 11.0.4 java.version.date = 2019-07-16 java.vm.compressedOopsMode = 32-bit java.vm.info = mixed mode, sharing java.vm.name = OpenJDK 64-Bit Server VM java.vm.specification.name = Java Virtual Machine Specification java.vm.specification.vendor = Oracle Corporation java.vm.specification.version = 11 java.vm.vendor = Oracle Corporation java.vm.version = 11.0.4+11-LTS javax.management.builder.initial = org.jboss.as.jmx.PluggableMBeanServerBuilder jboss.bind.address = 10.128.1.113 jboss.bind.address.management = 0.0.0.0 jboss.bind.address.private = 10.128.1.113 jboss.home.dir = /opt/eap jboss.host.name = tx-client-0 jboss.messaging.cluster.password = jboss.messaging.host = 10.128.1.113 jboss.modules.dir = /opt/eap/modules jboss.modules.system.pkgs = org.jboss.logmanager,jdk.nashorn.api,com.sun.crypto.provider jboss.node.name = tx-client-0 jboss.qualified.host.name = tx-client-0 jboss.server.base.dir = /opt/eap/standalone jboss.server.config.dir = /opt/eap/standalone/configuration jboss.server.data.dir = /opt/eap/standalone/data jboss.server.deploy.dir = /opt/eap/standalone/data/content jboss.server.log.dir = /opt/eap/standalone/log jboss.server.name = tx-client-0 jboss.server.persist.config = true jboss.server.temp.dir = /opt/eap/standalone/tmp jdk.debug = release jolokia.agent = https://10.128.1.113:8778/jolokia/ line.separator = logging.configuration = file:/opt/eap/standalone/configuration/logging.properties module.path = /opt/eap/modules org.jboss.boot.log.file = /opt/eap/standalone/log/server.log org.jboss.resolver.warning = true os.arch = amd64 os.name = Linux os.version = 3.10.0-957.el7.x86_64 path.separator = : sun.arch.data.model = 64 sun.boot.library.path = /usr/lib/jvm/java-11-openjdk-11.0.4.11-0.el8_0.x86_64/lib sun.cpu.endian = little sun.cpu.isalist = sun.io.unicode.encoding = UnicodeLittle sun.java.command = /opt/eap/jboss-modules.jar -mp /opt/eap/modules org.jboss.as.standalone -Djboss.home.dir=/opt/eap -Djboss.server.base.dir=/opt/eap/standalone -c standalone-openshift.xml -bmanagement 0.0.0.0 -Dwildfly.statistics-enabled=true -b 10.128.1.113 -bprivate 10.128.1.113 -Djboss.node.name=tx-client-0 -Djboss.messaging.host=10.128.1.113 -Djboss.messaging.cluster.password=aaaaaa sun.java.launcher = SUN_STANDARD sun.jnu.encoding = ANSI_X3.4-1968 sun.management.compiler = HotSpot 64-Bit Tiered Compilers sun.os.patch.level = unknown sun.util.logging.disableCallerCheck = true user.country = US user.dir = /home/jboss user.home = /home/jboss user.language = en user.name = jboss user.timezone = Etc/UTC wildfly.config.url = /opt/eap/standalone/configuration/custom-config.xml wildfly.statistics-enabled = true 08:45:27,480 DEBUG [org.jboss.as.config] (MSC service thread 1-1) VM Arguments: -D[Standalone] -Xms128m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.logmanager,jdk.nashorn.api,com.sun.crypto.provider -Djava.awt.headless=true -javaagent:/opt/jboss/container/jolokia/jolokia.jar=config=/opt/jboss/container/jolokia/etc/jolokia.properties -Xbootclasspath/a:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.13.Final.jar:/opt/eap/modules/system/layers/base/org/glassfish/javax/json/main/javax.json-1.1.2.jar:/opt/eap/modules/system/layers/base/javax/json/api/main/javax.json-api-1.1.2.jar:/opt/eap/modules/system/layers/base/org/wildfly/common/main/wildfly-common-1.5.1.Final.jar -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Dsun.util.logging.disableCallerCheck=true -XX:+UseParallelOldGC -XX:MinHeapFreeRatio=10 -XX:MaxHeapFreeRatio=20 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -XX:+ExitOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -Dwildfly.config.url=/opt/eap/standalone/configuration/custom-config.xml -agentlib:jdwp=transport=dt_socket,address=8787,server=y,suspend=n --add-exports=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=jdk.unsupported/sun.misc=ALL-UNNAMED --add-exports=jdk.unsupported/sun.reflect=ALL-UNNAMED -Dorg.jboss.boot.log.file=/opt/eap/standalone/log/server.log -Dlogging.configuration=file:/opt/eap/standalone/configuration/logging.properties 08:45:28,963 INFO [org.wildfly.security] (ServerService Thread Pool -- 28) ELY00001: WildFly Elytron version 1.10.0.CR3 08:45:30,442 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 21) WFLYCTL0028: Attribute 'username' in the resource at address '/subsystem=remoting/remote-outbound-connection=remote-ejb-connection' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation. 08:45:30,451 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 21) WFLYCTL0028: Attribute 'security-realm' in the resource at address '/subsystem=remoting/remote-outbound-connection=remote-ejb-connection' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation. 08:45:30,452 INFO [org.jboss.as.controller.management-deprecated] (ServerService Thread Pool -- 21) WFLYCTL0028: Attribute 'protocol' in the resource at address '/subsystem=remoting/remote-outbound-connection=remote-ejb-connection' is deprecated, and may be removed in a future version. See the attribute description in the output of the read-resource-description operation to learn more about the deprecation. 08:45:30,578 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 3) WFLYDR0001: Content added at location /opt/eap/standalone/data/content/c2/62d289abcd72b0bcc456178b11d13d657ddcf1/content 08:45:30,665 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0039: Creating http management service using socket-binding (management-http) 08:45:30,678 WARN [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0035: No security realm or http server authentication defined for http management service; all access will be unrestricted. 08:45:30,709 INFO [org.xnio] (MSC service thread 1-2) XNIO version 3.7.2.Final 08:45:30,736 INFO [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.7.2.Final 08:45:30,771 INFO [org.jboss.remoting] (MSC service thread 1-1) JBoss Remoting version 5.0.12.Final 08:45:30,871 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 42) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.4) 08:45:30,944 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 50) WFLYIO001: Worker 'default' has auto-configured to 2 core threads with 16 task threads based on your 1 available processors 08:45:30,976 INFO [org.jboss.as.jaxrs] (ServerService Thread Pool -- 51) WFLYRS0016: RESTEasy version 3.8.1.Final 08:45:30,992 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 63) WFLYNAM0001: Activating Naming Subsystem 08:45:31,021 INFO [org.wildfly.extension.microprofile.config.smallrye._private] (ServerService Thread Pool -- 60) WFLYCONF0001: Activating WildFly MicroProfile Config Subsystem 08:45:31,049 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 49) WFLYCLINF0001: Activating Infinispan subsystem. 08:45:31,071 INFO [org.jboss.as.webservices] (ServerService Thread Pool -- 74) WFLYWS0002: Activating WebServices Extension 08:45:31,081 INFO [org.jboss.as.security] (ServerService Thread Pool -- 69) WFLYSEC0002: Activating Security Subsystem 08:45:31,015 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 57) WFLYJSF0007: Activated the following JSF Implementations: [main] 08:45:31,132 INFO [org.jboss.as.connector] (MSC service thread 1-1) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.4.16.Final) 08:45:31,106 INFO [org.jboss.as.clustering.jgroups] (ServerService Thread Pool -- 54) WFLYCLJG0001: Activating JGroups subsystem. JGroups version 4.1.1 08:45:31,212 INFO [org.wildfly.extension.microprofile.health.smallrye] (ServerService Thread Pool -- 61) WFLYHEALTH0001: Activating Eclipse MicroProfile Health Subsystem 08:45:31,248 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0003: Undertow 2.0.22.Final starting 08:45:31,257 INFO [org.jboss.as.security] (MSC service thread 1-2) WFLYSEC0001: Current PicketBox version=5.0.3.Final 08:45:31,275 WARN [org.wildfly.clustering.web.undertow] (ServerService Thread Pool -- 73) WFLYCLWEBUT0007: No routing provider found for default-server; using legacy provider based on static configuration 08:45:31,472 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-2) WFLYJCA0018: Started Driver service with driver-name = h2 08:45:31,493 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 73) WFLYUT0014: Creating file handler for path '/opt/eap/welcome-content' with options [directory-listing: 'false', follow-symlink: 'false', case-sensitive: 'true', safe-symlink-paths: '[]'] 08:45:31,509 INFO [org.jboss.as.naming] (MSC service thread 1-2) WFLYNAM0003: Starting Naming Service 08:45:31,942 INFO [org.jboss.as.ejb3] (MSC service thread 1-2) WFLYEJB0482: Strict pool mdb-strict-max-pool is using a max instance size of 4 (per class), which is derived from the number of CPUs on this host. 08:45:31,949 INFO [org.jboss.as.mail.extension] (MSC service thread 1-2) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default] 08:45:31,952 WARN [org.jboss.as.remoting] (MSC service thread 1-2) ****** All authentication is ANONYMOUS for org.jboss.as.remoting.RemotingHttpUpgradeService 08:45:31,965 INFO [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0481: Strict pool slsb-strict-max-pool is using a max instance size of 16 (per class), which is derived from thread worker pool sizing. 08:45:32,352 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) FileSystemStore.createHierarchy(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/) 08:45:32,369 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) InputObjectState::InputObjectState() 08:45:32,369 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) FileSystemStore.allObjUids(/Recovery/TransactionStatusManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,369 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) OutputObjectState::OutputObjectState() 08:45:32,371 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule 08:45:32,371 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) StateManager::StateManager( 2, 0 ) 08:45:32,372 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) BasicAction::BasicAction() 08:45:32,372 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) AtomicActionRecoveryModule created 08:45:32,372 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) InputObjectState::InputObjectState() 08:45:32,372 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) FileSystemStore.allObjUids(/Recovery/TransactionStatusManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,372 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) OutputObjectState::OutputObjectState() 08:45:32,373 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.txoj.recovery.TORecoveryModule 08:45:32,375 DEBUG [com.arjuna.ats.txoj] (MSC service thread 1-2) TORecoveryModule created 08:45:32,376 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.arjuna.objectstore.ShadowNoFileLockStore 08:45:32,376 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) FileSystemStore.createHierarchy(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/) 08:45:32,376 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule 08:45:32,376 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule 08:45:32,378 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) InputObjectState::InputObjectState() 08:45:32,378 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.JTATransactionLogXAResourceOrphanFilter 08:45:32,378 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.JTANodeNameXAResourceOrphanFilter 08:45:32,379 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateJTAXAResourceOrphanFilter 08:45:32,382 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0012: Started server default-server. 08:45:32,385 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinationManagerXAResourceOrphanFilter 08:45:32,385 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.jta.recovery.arjunacore.JTAActionStatusServiceXAResourceOrphanFilter 08:45:32,388 INFO [com.arjuna.ats.arjuna] (MSC service thread 1-2) ARJUNA012310: Recovery manager listening on endpoint 10.128.1.113:4712 08:45:32,388 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) PeriodicRecovery: starting listener worker thread 08:45:32,390 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) Expiry scan interval set to 43200 seconds 08:45:32,390 TRACE [com.arjuna.ats.common] (MSC service thread 1-2) Loading class com.arjuna.ats.internal.arjuna.recovery.ExpiredTransactionStatusManagerScanner 08:45:32,394 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) Expiry scan interval set to 43200 seconds 08:45:32,395 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) ExpiredTransactionStatusManagerScanner created, with expiry time of 43200 seconds 08:45:32,395 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) ExpiredEntryMonitor - constructed 08:45:32,395 INFO [com.arjuna.ats.arjuna] (MSC service thread 1-2) ARJUNA012344: RecoveryManagerImple is ready on port 4712 08:45:32,395 INFO [com.arjuna.ats.jbossatx] (MSC service thread 1-2) ARJUNA032013: Starting transaction recovery manager 08:45:32,395 DEBUG [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ExpiredEntryMonitor running at Tue, 6 Aug 2019 08:45:32 08:45:32,417 DEBUG [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) ExpiredTransactionStatusManagerScanner - scanning to remove items from before Mon, 5 Aug 2019 20:45:32 08:45:32,417 TRACE [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) InputObjectState::InputObjectState() 08:45:32,417 TRACE [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) FileSystemStore.allObjUids(/Recovery/TransactionStatusManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,417 TRACE [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) OutputObjectState::OutputObjectState() 08:45:32,417 DEBUG [com.arjuna.ats.arjuna] (Transaction Expired Entry Monitor) 08:45:32,396 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:45:32,423 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:45:32,424 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:45:32 08:45:32,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:45:32,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:32,425 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,425 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:32,425 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:45:32,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:32,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:32,427 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:45:32,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:32,427 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:45:32,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:32,429 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:45:32,429 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:45:32,429 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:45:32,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:32,431 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:45:32,431 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:32,431 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:45:32,431 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:32,433 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,436 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:32,437 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:45:32,437 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:45:32,437 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:45:32,438 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:32,428 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0018: Host default-host starting 08:45:32,436 INFO [org.jboss.as.patching] (MSC service thread 1-1) WFLYPAT0050: WildFly Full cumulative patch ID is: base, one-off patches include: none 08:45:32,441 INFO [org.wildfly.extension.undertow] (MSC service thread 1-2) WFLYUT0006: Undertow AJP listener ajp listening on 0.0.0.0:8009 08:45:32,444 INFO [org.wildfly.extension.undertow] (MSC service thread 1-1) WFLYUT0006: Undertow HTTP listener default listening on 0.0.0.0:8080 08:45:32,450 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) ObjectStoreDir: /opt/eap/standalone/data/tx-object-store 08:45:32,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:32,452 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:45:32,452 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:45:32,452 INFO [com.arjuna.ats.jbossatx] (MSC service thread 1-2) ARJUNA032017: JBossTS Transaction Service (JTA version - tag: 54b60) - JBoss Inc. 08:45:32,453 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) TransactionReaper::instantiate() 08:45:32,454 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-2) TransactionReaper::TransactionReaper ( 9223372036854775807 ) 08:45:32,455 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) ReaperThread.run () 08:45:32,455 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) Thread Thread[Transaction Reaper,5,main] sleeping for 9223370471773643352 08:45:32,457 DEBUG [com.arjuna.ats.arjuna] (MSC service thread 1-2) registering bean jboss.jta:type=ObjectStore 08:45:32,470 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) ReaperWorkerThread.run () 08:45:32,470 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper Worker 0) Thread Thread[Transaction Reaper Worker 0,5,main] waiting for cancelled TXs 08:45:32,481 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-2) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/eap/standalone/deployments 08:45:32,498 INFO [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0027: Starting deployment of "tx-client.war" (runtime-name: "tx-client.war") 08:45:32,513 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 76) MODCLUSTER000001: Initializing mod_cluster version 1.4.1.Final 08:45:32,520 TRACE [com.arjuna.ats.common] (MSC service thread 1-1) Loading class com.arjuna.ats.jbossatx.jta.TransactionManagerDelegate 08:45:32,527 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-1) InputObjectState::InputObjectState() 08:45:32,527 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-1) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:32,527 TRACE [com.arjuna.ats.arjuna] (MSC service thread 1-1) OutputObjectState::OutputObjectState() 08:45:32,532 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 76) MODCLUSTER000032: Listening to proxy advertisements on /224.0.1.105:23364 08:45:32,572 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0075: AIO wasn't located on this platform, it will fall back to using pure Java NIO. Your platform is Linux, install LibAIO to enable the AIO journal and achieve optimal performance. 08:45:32,615 INFO [org.jboss.as.ejb3] (MSC service thread 1-1) WFLYEJB0493: EJB subsystem suspension complete 08:45:32,772 INFO [org.jboss.ws.common.management] (MSC service thread 1-1) JBWS022052: Starting JBossWS 5.3.0.Final (Apache CXF 3.3.2) 08:45:33,011 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-2) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS] 08:45:33,034 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=false,journalDirectory=/opt/eap/standalone/data/activemq/journal,bindingsDirectory=/opt/eap/standalone/data/activemq/bindings,largeMessagesDirectory=/opt/eap/standalone/data/activemq/largemessages,pagingDirectory=/opt/eap/standalone/data/activemq/paging) 08:45:33,111 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221013: Using NIO Journal 08:45:33,189 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE 08:45:33,190 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP 08:45:33,190 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ 08:45:33,191 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP 08:45:33,416 INFO [org.jboss.weld.deployer] (MSC service thread 1-1) WFLYWELD0003: Processing weld deployment tx-client.war 08:45:33,422 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221034: Waiting indefinitely to obtain live lock 08:45:33,422 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221035: Live Server Obtained live lock 08:45:33,602 INFO [org.hibernate.validator.internal.util.Version] (MSC service thread 1-1) HV000001: Hibernate Validator 6.0.16.Final 08:45:33,785 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'StatelessBeanManagedToPass' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/StatelessBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatelessBeanManagedToPass java:app/tx-client/StatelessBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatelessBeanManagedToPass java:module/StatelessBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatelessBeanManagedToPass ejb:/tx-client/StatelessBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatelessBeanManagedToPass java:global/tx-client/StatelessBeanManagedToPass java:app/tx-client/StatelessBeanManagedToPass java:module/StatelessBeanManagedToPass 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'StatelessServerCallerTwoPhase' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/StatelessServerCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerTwoPhase java:app/tx-client/StatelessServerCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerTwoPhase java:module/StatelessServerCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerTwoPhase ejb:/tx-client/StatelessServerCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerTwoPhase java:global/tx-client/StatelessServerCallerTwoPhase java:app/tx-client/StatelessServerCallerTwoPhase java:module/StatelessServerCallerTwoPhase 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'MockXAResourceInitializer' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/MockXAResourceInitializer!org.jboss.as.quickstarts.xa.resources.MockXAResourceInitializer java:app/tx-client/MockXAResourceInitializer!org.jboss.as.quickstarts.xa.resources.MockXAResourceInitializer java:module/MockXAResourceInitializer!org.jboss.as.quickstarts.xa.resources.MockXAResourceInitializer ejb:/tx-client/MockXAResourceInitializer!org.jboss.as.quickstarts.xa.resources.MockXAResourceInitializer java:global/tx-client/MockXAResourceInitializer java:app/tx-client/MockXAResourceInitializer java:module/MockXAResourceInitializer 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'RecoveryClientStateless' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/RecoveryClientStateless!org.jboss.as.quickstarts.xa.client.RecoveryClientStateless java:app/tx-client/RecoveryClientStateless!org.jboss.as.quickstarts.xa.client.RecoveryClientStateless java:module/RecoveryClientStateless!org.jboss.as.quickstarts.xa.client.RecoveryClientStateless ejb:/tx-client/RecoveryClientStateless!org.jboss.as.quickstarts.xa.client.RecoveryClientStateless java:global/tx-client/RecoveryClientStateless java:app/tx-client/RecoveryClientStateless java:module/RecoveryClientStateless 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'StatelessServerProgramaticCallerTwoPhase' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/StatelessServerProgramaticCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerProgramaticCallerTwoPhase java:app/tx-client/StatelessServerProgramaticCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerProgramaticCallerTwoPhase java:module/StatelessServerProgramaticCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerProgramaticCallerTwoPhase ejb:/tx-client/StatelessServerProgramaticCallerTwoPhase!org.jboss.as.quickstarts.xa.client.StatelessServerProgramaticCallerTwoPhase java:global/tx-client/StatelessServerProgramaticCallerTwoPhase java:app/tx-client/StatelessServerProgramaticCallerTwoPhase java:module/StatelessServerProgramaticCallerTwoPhase 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'BasicClientStateless' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/BasicClientStateless!org.jboss.as.quickstarts.xa.client.BasicClientStateless java:app/tx-client/BasicClientStateless!org.jboss.as.quickstarts.xa.client.BasicClientStateless java:module/BasicClientStateless!org.jboss.as.quickstarts.xa.client.BasicClientStateless ejb:/tx-client/BasicClientStateless!org.jboss.as.quickstarts.xa.client.BasicClientStateless java:global/tx-client/BasicClientStateless java:app/tx-client/BasicClientStateless java:module/BasicClientStateless 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'StatelessServerCallerOnePhase' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/StatelessServerCallerOnePhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerOnePhase java:app/tx-client/StatelessServerCallerOnePhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerOnePhase java:module/StatelessServerCallerOnePhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerOnePhase ejb:/tx-client/StatelessServerCallerOnePhase!org.jboss.as.quickstarts.xa.client.StatelessServerCallerOnePhase java:global/tx-client/StatelessServerCallerOnePhase java:app/tx-client/StatelessServerCallerOnePhase java:module/StatelessServerCallerOnePhase 08:45:33,786 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'StatefulBeanManagedToPass' in deployment unit 'deployment "tx-client.war"' are as follows: java:global/tx-client/StatefulBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatefulBeanManagedToPass java:app/tx-client/StatefulBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatefulBeanManagedToPass java:module/StatefulBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatefulBeanManagedToPass ejb:/tx-client/StatefulBeanManagedToPass!org.jboss.as.quickstarts.xa.client.StatefulBeanManagedToPass java:global/tx-client/StatefulBeanManagedToPass java:app/tx-client/StatefulBeanManagedToPass java:module/StatefulBeanManagedToPass 08:45:33,998 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor-throughput acceptor 08:45:34,014 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor-throughput acceptor 08:45:34,014 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor acceptor 08:45:34,014 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-1) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor acceptor 08:45:34,362 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221007: Server is now live 08:45:34,362 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221001: Apache ActiveMQ Artemis Message Broker version 2.8.1 [default, nodeID=8d45f16b-b826-11e9-99d9-0a580a800171] 08:45:34,373 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 76) WFLYMSGAMQ0002: Bound messaging object to jndi name java:/ConnectionFactory 08:45:34,442 INFO [org.jboss.weld.Version] (MSC service thread 1-2) WELD-000900: 3.1.1 (Final) 08:45:34,560 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0007: Registered connection factory java:/JmsXA 08:45:34,660 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 78) WFLYMSGAMQ0002: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory 08:45:34,714 INFO [org.apache.activemq.artemis.ra] (MSC service thread 1-1) AMQ151007: Resource adaptor started 08:45:34,714 INFO [org.jboss.as.connector.services.resourceadapters.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-1) IJ020002: Deployed: file://RaActivatoractivemq-ra 08:45:34,717 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0002: Bound JCA ConnectionFactory [java:/JmsXA] 08:45:34,717 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-1) WFLYJCA0118: Binding connection factory named java:/JmsXA to alias java:jboss/DefaultJMSConnectionFactory 08:45:42,453 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:45:42 08:45:42,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:45:42,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:45:42,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:45:42,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:42,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 40 InputObjectState Buffer: , -1) 08:45:42,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:45:42,455 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:45:42,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:45:42,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:45:42,459 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:45:42,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:45:42,460 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:45:42,460 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:45:42,460 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:45:42,460 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:45:42,461 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:45:42,461 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:45:42,461 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:45:42,461 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:45:42,461 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:45:44,795 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-1) ISPN000128: Infinispan version: Infinispan 'Infinity Minus ONE +2' 9.4.15.Final 08:45:44,963 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups channel ee 08:45:44,964 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000078: Starting JGroups channel ee 08:45:44,968 INFO [org.infinispan.CLUSTER] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel ee: [tx-client-0|0] (1) [tx-client-0] 08:45:44,979 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000079: Channel ee local address is tx-client-0, physical addresses are [10.128.1.113:7600] 08:45:44,979 INFO [org.infinispan.CLUSTER] (MSC service thread 1-1) ISPN000094: Received new cluster view for channel ee: [tx-client-0|0] (1) [tx-client-0] 08:45:44,991 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Channel ee local address is tx-client-0, physical addresses are [10.128.1.113:7600] 08:45:45,041 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000078: Starting JGroups channel ee 08:45:45,049 INFO [org.infinispan.CLUSTER] (MSC service thread 1-2) ISPN000094: Received new cluster view for channel ee: [tx-client-0|0] (1) [tx-client-0] 08:45:45,050 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-2) ISPN000079: Channel ee local address is tx-client-0, physical addresses are [10.128.1.113:7600] 08:45:45,120 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000078: Starting JGroups channel ee 08:45:45,122 INFO [org.infinispan.CLUSTER] (MSC service thread 1-1) ISPN000094: Received new cluster view for channel ee: [tx-client-0|0] (1) [tx-client-0] 08:45:45,133 INFO [org.infinispan.remoting.transport.jgroups.JGroupsTransport] (MSC service thread 1-1) ISPN000079: Channel ee local address is tx-client-0, physical addresses are [10.128.1.113:7600] 08:45:45,459 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 78) WFLYCLINF0002: Started client-mappings cache from ejb container 08:45:45,670 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImpleManager.suspend 08:45:45,673 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) BaseTransaction.begin 08:45:45,674 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) StateManager::StateManager( 2, 0 ) 08:45:45,674 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::BasicAction() 08:45:45,674 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::Begin() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:10 08:45:45,674 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::actionInitialise() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:10 08:45:45,676 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) ActionHierarchy::ActionHierarchy(1) 08:45:45,677 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) ActionHierarchy::add(0:ffff0a800171:35fe0b3b:5d493e2c:10, 1) 08:45:45,677 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 adding Thread[ServerService Thread Pool -- 76,5,ServerService ThreadGroup] 08:45:45,677 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 adding Thread[ServerService Thread Pool -- 76,5,ServerService ThreadGroup] result = true 08:45:45,678 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) TransactionReaper::insert ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:10 status: ActionStatus.RUNNING, 300 ) 08:45:45,678 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) ReaperElement::ReaperElement ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:10 status: ActionStatus.RUNNING, 300 ) 08:45:45,678 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) ReaperThread.run () 08:45:45,679 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) TransactionReaper::check () 08:45:45,679 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) TransactionReaper::check - comparing 1565081445678 08:45:45,679 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) Thread Thread[Transaction Reaper,5,main] sleeping for 299999 08:45:45,680 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:45:45,680 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImpleManager.suspend 08:45:45,680 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 removing 1 08:45:45,680 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 removing 1 result = true 08:45:45,683 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:45:45,697 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.registerSynchronization - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 1746691353 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@681c6519 08:45:45,698 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImpleManager.resume 08:45:45,698 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 adding Thread[ServerService Thread Pool -- 76,5,ServerService ThreadGroup] 08:45:45,698 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 adding Thread[ServerService Thread Pool -- 76,5,ServerService ThreadGroup] result = true 08:45:45,698 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.equals 08:45:45,698 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.equals 08:45:45,705 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (ServerService Thread Pool -- 76) Creating org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 with test action NONE 08:45:45,705 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (ServerService Thread Pool -- 76) Adding instance of the MockXAResourceRecoveryHelper 'org.jboss.as.quickstarts.xa.resources.MockXAResourceRecoveryHelper@346edd08' to XARecoveryModule 08:45:45,707 WARN [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (ServerService Thread Pool -- 76) file /opt/eap/standalone/data/MockXAResource does not exists - no data can be recovered 08:45:45,711 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:45:45,711 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.equals 08:45:45,711 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:45:45,712 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) BaseTransaction.commit 08:45:45,712 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.commitAndDisassociate 08:45:45,712 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 1746691353 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@681c6519 08:45:45,712 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.equals 08:45:45,713 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 1818015224 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@6c5cb5f8 08:45:45,713 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::End() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:10 08:45:45,713 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::End() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:10) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:45:45,713 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 1818015224 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@6c5cb5f8 08:45:45,714 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 1746691353 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@681c6519 08:45:45,714 TRACE [com.arjuna.ats.jta] (ServerService Thread Pool -- 76) TransactionImple.equals 08:45:45,715 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 removing 1 08:45:45,715 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:10 removing 1 result = true 08:45:45,715 TRACE [com.arjuna.ats.arjuna] (ServerService Thread Pool -- 76) TransactionReaper::remove ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:10 status: ActionStatus.COMMITTED ) 08:45:45,810 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 76) WFLYUT0021: Registered web context: '/tx-client' for server 'default-server' 08:45:45,851 INFO [org.jboss.as.server] (ServerService Thread Pool -- 43) WFLYSRV0010: Deployed "tx-client.war" (runtime-name : "tx-client.war") 08:45:46,076 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0212: Resuming server 08:45:46,079 INFO [com.arjuna.ats.jbossatx] (Controller Boot Thread) ARJUNA032035: Resuming transaction recovery manager 08:45:46,081 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0060: Http management interface listening on http://0.0.0.0:9990/management 08:45:46,081 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0054: Admin console is not enabled 08:45:46,082 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: WildFly Full 18.0.0.Beta1-SNAPSHOT (WildFly Core 10.0.0.Beta2) started in 20550ms - Started 670 of 925 services (489 services are lazy, passive or on-demand) 08:46:38,577 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:46:38,577 TRACE [com.arjuna.ats.jta] (default task-1) BaseTransaction.begin 08:46:38,577 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 2, 0 ) 08:46:38,577 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::BasicAction() 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::Begin() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::actionInitialise() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) ActionHierarchy::ActionHierarchy(1) 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) ActionHierarchy::add(0:ffff0a800171:35fe0b3b:5d493e2c:13, 1) 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] result = true 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) TransactionReaper::insert ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:13 status: ActionStatus.RUNNING, 300 ) 08:46:38,578 TRACE [com.arjuna.ats.arjuna] (default task-1) ReaperElement::ReaperElement ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:13 status: ActionStatus.RUNNING, 300 ) 08:46:38,578 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,579 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:46:38,579 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 08:46:38,579 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 result = true 08:46:38,579 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,579 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.registerSynchronization - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 149459306 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@8e8916a 08:46:38,579 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.resume 08:46:38,579 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] 08:46:38,579 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] result = true 08:46:38,580 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,580 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,580 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,580 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,580 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,580 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:46:38,580 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 08:46:38,581 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 result = true 08:46:38,581 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,581 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:46:38,581 TRACE [com.arjuna.ats.jta] (default task-1) BaseTransaction.begin 08:46:38,581 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 2, 0 ) 08:46:38,581 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::BasicAction() 08:46:38,581 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::Begin() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:16 08:46:38,581 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::actionInitialise() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:16 08:46:38,581 TRACE [com.arjuna.ats.arjuna] (default task-1) ActionHierarchy::ActionHierarchy(1) 08:46:38,582 TRACE [com.arjuna.ats.arjuna] (default task-1) ActionHierarchy::add(0:ffff0a800171:35fe0b3b:5d493e2c:16, 1) 08:46:38,582 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 adding Thread[default task-1,5,main] 08:46:38,582 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 adding Thread[default task-1,5,main] result = true 08:46:38,582 TRACE [com.arjuna.ats.arjuna] (default task-1) TransactionReaper::insert ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:16 status: ActionStatus.RUNNING, 300 ) 08:46:38,582 TRACE [com.arjuna.ats.arjuna] (default task-1) ReaperElement::ReaperElement ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:16 status: ActionStatus.RUNNING, 300 ) 08:46:38,582 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,582 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:46:38,582 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 removing 2 08:46:38,583 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 removing 2 result = true 08:46:38,583 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,583 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.registerSynchronization - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 983672427 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@3aa1a66b 08:46:38,583 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.resume 08:46:38,583 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 adding Thread[default task-1,5,main] 08:46:38,583 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 adding Thread[default task-1,5,main] result = true 08:46:38,584 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,584 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,584 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,584 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,585 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,585 TRACE [com.arjuna.ats.jta] (default task-1) BaseTransaction.commit 08:46:38,585 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.commitAndDisassociate 08:46:38,585 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 983672427 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@3aa1a66b 08:46:38,585 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,585 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 83207757 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@4f5a64d 08:46:38,586 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::End() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:16 08:46:38,586 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::End() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:16) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:46:38,586 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 83207757 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@4f5a64d 08:46:38,586 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 983672427 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@3aa1a66b 08:46:38,586 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,586 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 removing 2 08:46:38,587 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:16 removing 2 result = true 08:46:38,587 TRACE [com.arjuna.ats.arjuna] (default task-1) TransactionReaper::remove ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:16 status: ActionStatus.COMMITTED ) 08:46:38,587 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.resume 08:46:38,587 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] 08:46:38,587 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] result = true 08:46:38,587 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,596 INFO [org.wildfly.naming] (default task-1) WildFly Naming version 1.0.11.Final 08:46:38,613 INFO [org.jboss.ejb.client] (default task-1) JBoss EJB Client version 4.0.21.Final 08:46:38,619 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,620 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) Creating org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed with test action NONE 08:46:38,620 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.enlistResource ( org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed ) 08:46:38,620 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,620 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 1, 0 ) 08:46:38,620 TRACE [com.arjuna.ats.arjuna] (default task-1) AbstractRecord::AbstractRecord (0:ffff0a800171:35fe0b3b:5d493e2c:1a, 1) 08:46:38,621 TRACE [com.arjuna.ats.arjuna] (default task-1) InputObjectState::InputObjectState() 08:46:38,621 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.allObjUids(EISNAME, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:46:38,621 TRACE [com.arjuna.ats.arjuna] (default task-1) OutputObjectState::OutputObjectState() 08:46:38,622 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.XAResourceRecord ( < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >, org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed ), record id=0:ffff0a800171:35fe0b3b:5d493e2c:1a 08:46:38,622 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:1a 08:46:38,623 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,623 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) Creating org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b with test action NONE 08:46:38,623 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.enlistResource ( org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b ) 08:46:38,623 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,623 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 1, 0 ) 08:46:38,623 TRACE [com.arjuna.ats.arjuna] (default task-1) AbstractRecord::AbstractRecord (0:ffff0a800171:35fe0b3b:5d493e2c:1e, 1) 08:46:38,623 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.XAResourceRecord ( < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >, org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b ), record id=0:ffff0a800171:35fe0b3b:5d493e2c:1e 08:46:38,623 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:1a) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:1e 08:46:38,668 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,668 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:38,668 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:38,668 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:46:38,669 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 08:46:38,669 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 result = true 08:46:38,669 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,085 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.resume 08:46:39,085 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] 08:46:39,086 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 adding Thread[default task-1,5,main] result = true 08:46:39,086 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,110 INFO [org.jboss.ejb.client] (XNIO-1 task-1) Using existing AuthenticationContext for writeTransaction(...) 08:46:39,132 TRACE [com.arjuna.ats.jta] (XNIO-1 task-1) TransactionImple.enlistResource ( Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080 ) 08:46:39,132 TRACE [com.arjuna.ats.jta] (XNIO-1 task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:39,134 TRACE [com.arjuna.ats.arjuna] (XNIO-1 task-1) StateManager::StateManager( 1, 0 ) 08:46:39,134 TRACE [com.arjuna.ats.arjuna] (XNIO-1 task-1) AbstractRecord::AbstractRecord (0:ffff0a800171:35fe0b3b:5d493e2c:21, 1) 08:46:39,134 TRACE [com.arjuna.ats.jta] (XNIO-1 task-1) XAResourceRecord.XAResourceRecord ( < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080 ), record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:46:39,136 TRACE [com.arjuna.ats.arjuna] (XNIO-1 task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:1a 0:ffff0a800171:35fe0b3b:5d493e2c:1e) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:46:39,136 TRACE [com.arjuna.ats.jta] (XNIO-1 task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:39,136 TRACE [com.arjuna.ats.jta] (XNIO-1 task-1) TransactionImple.registerSynchronization - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 59300605 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@388dafd 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) BaseTransaction.commit 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.commitAndDisassociate 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 149459306 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@8e8916a 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 59300605 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@388dafd 08:46:39,314 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,361 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 1189415507 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@46e50a53 08:46:39,361 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::End() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:46:39,361 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::prepare () for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:46:39,361 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1589529c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:1a 08:46:39,362 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) prepare 'org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed' xid: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >] 08:46:39,363 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >][number: 1] records to /opt/eap/standalone/data/MockXAResource 08:46:39,374 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doPrepare() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:1a) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:46:39,375 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:1a 08:46:39,375 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@36226ad4 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:1e 08:46:39,375 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) prepare 'org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b' xid: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >] 08:46:39,375 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >, < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >][number: 2] records to /opt/eap/standalone/data/MockXAResource 08:46:39,383 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doPrepare() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:1e) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:46:39,383 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:1a) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:1e 08:46:39,384 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2f37a4bf >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:46:39,435 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doPrepare() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:46:39,436 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:1a 0:ffff0a800171:35fe0b3b:5d493e2c:1e) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:46:39,436 TRACE [com.arjuna.ats.arjuna] (default task-1) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:46:39,436 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state () 08:46:39,436 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081199436 08:46:39,436 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:46:39,437 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:46:39,438 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:46:39,438 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:46:39,438 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:46:39,438 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:46:39,444 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a NONE_RECORD 08:46:39,445 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing action status of ActionStatus.COMMITTING 08:46:39,445 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:46:39,445 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:46:39,445 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:46:39,445 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:46:39,446 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:46:39,446 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.createHierarchy(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13) 08:46:39,465 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@7aebb90b) 08:46:39,465 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:46:39,466 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit (XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1589529c >) 08:46:39,466 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1589529c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:1a 08:46:39,466 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) commit 'org.jboss.as.quickstarts.xa.resources.MockXAResource@425da2ed' xid:[< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:19, subordinatenodename=null, eis_name=unknown eis name >], without one phase 08:46:39,466 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >][number: 1] records to /opt/eap/standalone/data/MockXAResource 08:46:39,479 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:1a) is (TwoPhaseOutcome.FINISH_OK) node id: (tx-client-0) 08:46:39,479 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit (XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@36226ad4 >) 08:46:39,479 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@36226ad4 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:1e 08:46:39,479 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) commit 'org.jboss.as.quickstarts.xa.resources.MockXAResource@5f011a2b' xid:[< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:1d, subordinatenodename=null, eis_name=unknown eis name >], without one phase 08:46:39,480 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [][number: 0] records to /opt/eap/standalone/data/MockXAResource 08:46:39,487 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:1e) is (TwoPhaseOutcome.FINISH_OK) node id: (tx-client-0) 08:46:39,487 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2f37a4bf >) 08:46:39,487 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2f37a4bf >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:46:39,892 WARN [com.arjuna.ats.jta] (default task-1) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: WFTXN0048: Failed to receive an XA response from peer at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:424) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1529) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96) at org.jboss.jts//com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at org.jboss.jts//com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1288) at org.jboss.jts//com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at org.jboss.jts.integration//com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:75) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:88) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:261) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:362) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:144) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.core@3.1.1.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72) at org.jboss.as.weld.common@18.0.0.Beta1-SNAPSHOT//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.elytron-private@1.10.0.CR3//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:618) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at deployment.tx-client.war//org.jboss.as.quickstarts.xa.client.RecoveryClientStateless$$$view4.txStatelessServerSecondCommitJvmHalt(Unknown Source) at deployment.tx-client.war//org.jboss.as.quickstarts.xa.client.EJBTestCallerRestEndpoints.txStatelessServerSecondCommitJvmHalt(EJBTestCallerRestEndpoints.java:105) at deployment.tx-client.war//org.jboss.as.quickstarts.xa.client.EJBTestCallerRestEndpoints$Proxy$_$$_WeldClientProxy.txStatelessServerSecondCommitJvmHalt(Unknown Source) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:138) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:517) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:406) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:370) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:372) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:344) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:317) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:440) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.api@1.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:791) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.MetricsHandler.handleRequest(MetricsHandler.java:64) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.core.MetricsChainHandler.handleRequest(MetricsChainHandler.java:59) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.core@2.0.22.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) at io.undertow.core@2.0.22.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads@2.3.3.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.jboss.remoting3.ChannelClosedException: Channel was closed at org.jboss.remoting@5.0.12.Final//org.jboss.remoting3.util.BlockingInvocation$Response.getInputStream(BlockingInvocation.java:136) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:392) ... 115 more 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state () 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081199905 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:46:39,905 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a NONE_RECORD 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing action status of ActionStatus.COMMITTED 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:46:39,906 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:46:39,915 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@50d183fc) 08:46:39,916 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::End() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) is (TwoPhaseOutcome.FINISH_OK) node id: (tx-client-0) 08:46:39,916 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 1189415507 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@46e50a53 08:46:39,916 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 59300605 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@388dafd 08:46:39,916 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,916 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 149459306 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@8e8916a 08:46:39,916 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:46:39,916 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 08:46:39,916 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:13 removing 2 result = true 08:46:39,916 TRACE [com.arjuna.ats.arjuna] (default task-1) TransactionReaper::remove ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:13 status: ActionStatus.COMMITTED ) 08:47:04,881 DEBUG [com.arjuna.ats.arjuna] (Listener:4712) Connected to 10.128.1.113 on port 36758 on listener port 4712 for service com.arjuna.ats.internal.arjuna.recovery.WorkerService 08:47:04,886 DEBUG [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:36758) PeriodicRecovery: listener worker interrupts background thread 08:47:04,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:47:04,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:47:04,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:47:04 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:04,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,886 INFO [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:36758) ARJUNA012340: RecoveryManager scan scheduled to begin. 08:47:04,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:04,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:04,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:04,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:04,889 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:47:04,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:04,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2699f90b, null) 08:47:04,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:47:04,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:47:04,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081199905 08:47:04,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:47:04,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:04,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:04,893 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:28 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:47:04,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:28 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:47:04,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:28 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:47:04,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:28 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:04,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:04,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,898 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:04,898 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:04,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:04,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:04,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:04,900 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:47:04,901 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:47:04,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:04,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:04,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:04,901 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:47:04,949 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:47:04,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081224950 08:47:04,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:47:04,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:47:04,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081224950 08:47:04,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:47:04,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:14,951 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:47:14 08:47:14,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:47:14,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:14,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:14,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:14,952 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 72 InputObjectState Buffer: , -1) 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:14,952 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:47:14,952 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:14,952 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:14,953 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:14,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@69c76d01, null) 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081199905 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:47:14,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:47:14,955 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:47:14,955 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:14,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1b354d13 >) 08:47:14,955 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1b354d13 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:15,239 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$receiveMessage$2(RemoteConnectionChannel.java:361) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348) at java.lang.Thread.run(Thread.java:834) 08:47:15,242 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:47:15,242 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081235243 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:47:15,243 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:15,244 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:47:15,253 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@6eb40297) 08:47:15,253 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:47:15,253 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:47:15,253 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:47:15,253 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:47:15,253 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:47:15,254 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:47:15,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081224950 1565081235255 false 08:47:15,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081224950 1565081235255 false 08:47:15,255 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:47:15,255 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:47:15,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:47:15,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:47:15,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: scan thread signals listener worker 08:47:15,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:47:16,706 DEBUG [com.arjuna.ats.arjuna] (Listener:4712) Connected to 10.128.1.113 on port 36928 on listener port 4712 for service com.arjuna.ats.internal.arjuna.recovery.WorkerService 08:47:16,797 DEBUG [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:36928) PeriodicRecovery: listener worker interrupts background thread 08:47:16,797 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:47:16,797 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:47:16,797 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:47:16 08:47:16,797 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:47:16,797 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,798 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:16,798 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,798 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:47:16,798 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,798 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:16,798 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,798 INFO [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:36928) ARJUNA012340: RecoveryManager scan scheduled to begin. 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:16,799 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:16,800 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@7dde20fd, null) 08:47:16,801 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:47:16,801 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:47:16,801 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081235243 08:47:16,801 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:16,801 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,801 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:47:16,801 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:16,801 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,802 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:47:16,802 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:16,802 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:16,802 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:47:16,802 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,802 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:47:16,802 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,803 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:34 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:47:16,803 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:34 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:47:16,803 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:34 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:47:16,803 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:34 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:16,804 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,804 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:16,804 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,804 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,804 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:16,804 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,805 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,805 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:16,805 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,805 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,805 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:47:16,805 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:16,806 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:16,806 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,807 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:16,807 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:47:16,807 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:47:16,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:16,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:16,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:16,808 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:47:16,809 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:47:16,809 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:47:16,809 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:47:16,810 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:47:16,810 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:47:26,810 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:47:26 08:47:26,810 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:47:26,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:47:26,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:26,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:26,811 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:26,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 72 InputObjectState Buffer: , -1) 08:47:26,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:47:26,812 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:47:26,812 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:26,812 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:26,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@1c0891e8, null) 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081235243 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:47:26,813 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:47:26,813 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:47:26,814 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:26,814 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@10dd835c >) 08:47:26,814 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@10dd835c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:26,837 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:47:26,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:47:26,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:26,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081246838 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:26,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:47:26,848 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@3d6030e8) 08:47:26,848 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:47:26,848 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:47:26,848 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:47:26,848 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:47:26,848 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:47:26,848 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:47:26,849 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081224950 1565081246849 false 08:47:26,849 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081224950 1565081246849 false 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:47:26,849 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:47:26,849 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:47:26,849 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:47:26,849 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: scan thread signals listener worker 08:47:26,849 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:47:28,877 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:47:28,877 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:47:28,878 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:47:28,878 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:3d OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:47:28,878 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:3d OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:47:28,878 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:3d OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:47:28,878 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:3d OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:28,879 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:47:28,880 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:47:28,880 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:47:28,880 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:28,883 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@6f72c831, null) 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) BasicAction::restore_state () 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081246838 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Unpacked a 171 record 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::StateManager( 0:0:0:0:0 ) 08:47:28,884 TRACE [com.arjuna.ats.arjuna] (jolokia-0) AbstractRecord::AbstractRecord () - crash recovery constructor 08:47:28,885 TRACE [com.arjuna.ats.jta] (jolokia-0) XAResourceRecord.restore_state - XAResource de-serialized 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Unpacked a 463 record 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) HeuristicList - Unpacked heuristic list size of 0 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Restored action status of ActionStatus.COMMITTED 7 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Restored action type Top-level 0 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:47:28,885 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:28,886 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2db2ea16, null) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::activate( null) for object-id 0:0:0:0:0 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::setupStore ( null ) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.read_committed(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, StateType.OS_ORIGINAL) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, StateType.OS_SHADOW) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, 10) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, StateType.OS_ORIGINAL) 08:47:28,902 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, 11) 08:47:28,903 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.currentState(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord) - returning StateStatus.OS_UNKNOWN 08:47:28,903 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state could not find committed or uncommitted state for 0:0:0:0:0 instead found state StateStatus.OS_UNKNOWN 08:47:28,903 WARN [com.arjuna.ats.arjuna] (jolokia-0) ARJUNA012035: Activate of object with id = 0:0:0:0:0 and type /StateManager/AbstractRecord/XAResourceRecord unexpectedly failed 08:47:28,903 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Registering: jboss.jta:type=ObjectStore,itype=StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction,uid=0_ffff0a800171_35fe0b3b_5d493e2c_13 08:47:28,903 DEBUG [com.arjuna.ats.arjuna] (jolokia-0) registering bean jboss.jta:type=ObjectStore,itype=StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction,uid=0_ffff0a800171_35fe0b3b_5d493e2c_13 08:47:28,904 DEBUG [com.arjuna.ats.arjuna] (jolokia-0) registering bean jboss.jta:type=ObjectStore,itype=StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction,uid=0_ffff0a800171_35fe0b3b_5d493e2c_13,puid=0_ffff0a800171_35fe0b3b_5d493e2c_21 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) BaseTransaction.begin 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 2, 0 ) 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::BasicAction() 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::Begin() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::actionInitialise() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) ActionHierarchy::ActionHierarchy(1) 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) ActionHierarchy::add(0:ffff0a800171:35fe0b3b:5d493e2c:42, 1) 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 adding Thread[default task-1,5,main] 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 adding Thread[default task-1,5,main] result = true 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) TransactionReaper::insert ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:42 status: ActionStatus.RUNNING, 300 ) 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) ReaperElement::ReaperElement ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:42 status: ActionStatus.RUNNING, 300 ) 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 removing 2 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 removing 2 result = true 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.registerSynchronization - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 1215247779 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@486f35a3 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.resume 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 adding Thread[default task-1,5,main] 08:47:47,018 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 adding Thread[default task-1,5,main] result = true 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,018 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,038 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) Creating org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0 with test action NONE 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.enlistResource ( org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0 ) 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,038 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 1, 0 ) 08:47:47,038 TRACE [com.arjuna.ats.arjuna] (default task-1) AbstractRecord::AbstractRecord (0:ffff0a800171:35fe0b3b:5d493e2c:46, 1) 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.XAResourceRecord ( < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >, org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0 ), record id=0:ffff0a800171:35fe0b3b:5d493e2c:46 08:47:47,038 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:46 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,038 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) Creating org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced with test action NONE 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.enlistResource ( org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced ) 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,038 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager::StateManager( 1, 0 ) 08:47:47,038 TRACE [com.arjuna.ats.arjuna] (default task-1) AbstractRecord::AbstractRecord (0:ffff0a800171:35fe0b3b:5d493e2c:49, 1) 08:47:47,038 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.XAResourceRecord ( < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >, org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced ), record id=0:ffff0a800171:35fe0b3b:5d493e2c:49 08:47:47,038 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:46) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:49 08:47:47,039 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,039 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,039 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,039 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.suspend 08:47:47,039 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 removing 2 08:47:47,039 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 removing 2 result = true 08:47:47,039 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,041 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImpleManager.resume 08:47:47,041 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 adding Thread[default task-1,5,main] 08:47:47,042 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::addChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 adding Thread[default task-1,5,main] result = true 08:47:47,042 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,076 INFO [org.jboss.ejb.client] (XNIO-1 task-2) Using existing AuthenticationContext for writeTransaction(...) 08:47:47,088 TRACE [com.arjuna.ats.jta] (XNIO-1 task-2) TransactionImple.enlistResource ( Subordinate XAResource at remote+http://tx-server:8080 ) 08:47:47,088 TRACE [com.arjuna.ats.jta] (XNIO-1 task-2) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,089 TRACE [com.arjuna.ats.arjuna] (XNIO-1 task-2) StateManager::StateManager( 1, 0 ) 08:47:47,089 TRACE [com.arjuna.ats.arjuna] (XNIO-1 task-2) AbstractRecord::AbstractRecord (0:ffff0a800171:35fe0b3b:5d493e2c:4c, 1) 08:47:47,089 TRACE [com.arjuna.ats.jta] (XNIO-1 task-2) XAResourceRecord.XAResourceRecord ( < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, Subordinate XAResource at remote+http://tx-server:8080 ), record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:47:47,089 TRACE [com.arjuna.ats.arjuna] (XNIO-1 task-2) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:46 0:ffff0a800171:35fe0b3b:5d493e2c:49) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:47:47,089 TRACE [com.arjuna.ats.jta] (XNIO-1 task-2) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,089 TRACE [com.arjuna.ats.jta] (XNIO-1 task-2) TransactionImple.registerSynchronization - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 821774360 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@30fb4818 08:47:47,239 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.getStatus: javax.transaction.Status.STATUS_ACTIVE 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) BaseTransaction.commit 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.commitAndDisassociate 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 1215247779 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@486f35a3 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 821774360 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@30fb4818 08:47:47,240 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,245 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.beforeCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 1473887533 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@57d9bd2d 08:47:47,245 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::End() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:47:47,246 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::prepare () for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:47:47,246 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@e15c6a6 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:46 08:47:47,246 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) prepare 'org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0' xid: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >] 08:47:47,246 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >][number: 1] records to /opt/eap/standalone/data/MockXAResource 08:47:47,252 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doPrepare() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:46) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:47:47,252 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:46 08:47:47,252 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@56b715bf >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:49 08:47:47,253 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) prepare 'org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced' xid: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >] 08:47:47,253 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >, < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >][number: 2] records to /opt/eap/standalone/data/MockXAResource 08:47:47,261 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doPrepare() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:49) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:47:47,261 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:46) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:49 08:47:47,261 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelPrepare for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@72563b95 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doPrepare() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.PREPARE_OK) node id: (tx-client-0) 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: 0:ffff0a800171:35fe0b3b:5d493e2c:46 0:ffff0a800171:35fe0b3b:5d493e2c:49) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state () 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081267300 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:47:47,300 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a NONE_RECORD 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing action status of ActionStatus.COMMITTING 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:47,301 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:47,302 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:47:47,307 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@70f6ffb1) 08:47:47,307 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:47:47,308 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit (XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@e15c6a6 >) 08:47:47,308 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@e15c6a6 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:46 08:47:47,308 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) commit 'org.jboss.as.quickstarts.xa.resources.MockXAResource@585078b0' xid:[< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:45, subordinatenodename=null, eis_name=unknown eis name >], without one phase 08:47:47,308 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >][number: 1] records to /opt/eap/standalone/data/MockXAResource 08:47:47,323 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:46) is (TwoPhaseOutcome.FINISH_OK) node id: (tx-client-0) 08:47:47,323 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit (XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@56b715bf >) 08:47:47,323 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@56b715bf >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:49 08:47:47,323 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResource] (default task-1) commit 'org.jboss.as.quickstarts.xa.resources.MockXAResource@3bd33ced' xid:[< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:48, subordinatenodename=null, eis_name=unknown eis name >], without one phase 08:47:47,324 INFO [org.jboss.as.quickstarts.xa.resources.MockXAResourceStorage] (default task-1) writing to disk xids: [][number: 0] records to /opt/eap/standalone/data/MockXAResource 08:47:47,330 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:49) is (TwoPhaseOutcome.FINISH_OK) node id: (tx-client-0) 08:47:47,330 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@72563b95 >) 08:47:47,330 TRACE [com.arjuna.ats.jta] (default task-1) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@72563b95 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:47:47,716 WARN [com.arjuna.ats.jta] (default task-1) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XAER_RMFAIL: javax.transaction.xa.XAException: WFTXN0048: Failed to receive an XA response from peer at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:424) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.End(BasicAction.java:1529) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:96) at org.jboss.jts//com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at org.jboss.jts//com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1288) at org.jboss.jts//com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at org.jboss.jts.integration//com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.LocalTransaction.commitAndDissociate(LocalTransaction.java:75) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.ContextTransactionManager.commit(ContextTransactionManager.java:71) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:88) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:261) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:362) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:144) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.core@3.1.1.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72) at org.jboss.as.weld.common@18.0.0.Beta1-SNAPSHOT//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.elytron-private@1.10.0.CR3//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:618) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee@18.0.0.Beta1-SNAPSHOT//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at deployment.tx-client.war//org.jboss.as.quickstarts.xa.client.RecoveryClientStateless$$$view4.txStatelessServerSecondCommitJvmHalt(Unknown Source) at deployment.tx-client.war//org.jboss.as.quickstarts.xa.client.EJBTestCallerRestEndpoints.txStatelessServerSecondCommitJvmHalt(EJBTestCallerRestEndpoints.java:105) at deployment.tx-client.war//org.jboss.as.quickstarts.xa.client.EJBTestCallerRestEndpoints$Proxy$_$$_WeldClientProxy.txStatelessServerSecondCommitJvmHalt(Unknown Source) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:138) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:517) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:406) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:370) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:372) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:344) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:317) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:440) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.resteasy-jaxrs@3.8.1.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.api@1.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:791) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.core@2.0.22.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.GlobalRequestControllerHandler.handleRequest(GlobalRequestControllerHandler.java:68) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.0.22.Final//io.undertow.server.handlers.MetricsHandler.handleRequest(MetricsHandler.java:64) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.core.MetricsChainHandler.handleRequest(MetricsChainHandler.java:59) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow@18.0.0.Beta1-SNAPSHOT//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet@2.0.22.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.core@2.0.22.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:376) at io.undertow.core@2.0.22.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads@2.3.3.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.jboss.remoting3.ChannelClosedException: Channel was closed at org.jboss.remoting@5.0.12.Final//org.jboss.remoting3.util.BlockingInvocation$Response.getInputStream(BlockingInvocation.java:136) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:392) ... 115 more 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state () 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081267717 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a 171 record 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing a NONE_RECORD 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) Packing action status of ActionStatus.COMMITTED 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:47:47,717 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:47:47,728 TRACE [com.arjuna.ats.arjuna] (default task-1) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@389796e0) 08:47:47,728 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::End() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) is (TwoPhaseOutcome.FINISH_OK) node id: (tx-client-0) 08:47:47,729 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1 HashCode: 1473887533 toString: org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$1@57d9bd2d 08:47:47,729 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 821774360 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@30fb4818 08:47:47,729 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,729 TRACE [com.arjuna.ats.jta] (default task-1) SynchronizationImple.afterCompletion - Class: class org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization HashCode: 1215247779 toString: org.wildfly.transaction.client.AbstractTransaction$AssociatingSynchronization@486f35a3 08:47:47,729 TRACE [com.arjuna.ats.jta] (default task-1) TransactionImple.equals 08:47:47,729 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 removing 2 08:47:47,729 TRACE [com.arjuna.ats.arjuna] (default task-1) BasicAction::removeChildThread () action 0:ffff0a800171:35fe0b3b:5d493e2c:42 removing 2 result = true 08:47:47,729 TRACE [com.arjuna.ats.arjuna] (default task-1) TransactionReaper::remove ( BasicAction: 0:ffff0a800171:35fe0b3b:5d493e2c:42 status: ActionStatus.COMMITTED ) 08:48:23,693 DEBUG [com.arjuna.ats.arjuna] (Listener:4712) Connected to 10.128.1.113 on port 37734 on listener port 4712 for service com.arjuna.ats.internal.arjuna.recovery.WorkerService 08:48:23,695 DEBUG [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:37734) PeriodicRecovery: listener worker interrupts background thread 08:48:23,696 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:48:23,696 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:48:23,696 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:48:23 08:48:23,696 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:48:23,696 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,696 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:23,696 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,697 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:48:23,697 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,697 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:23,697 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,698 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@7b17687b, null) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081246838 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,699 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:48:23,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@4e02b1d8, null) 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081267717 08:48:23,701 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,701 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:23,701 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,701 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:48:23,701 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:23,701 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:23,702 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:23,702 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:48:23,702 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,702 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:48:23,702 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,702 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:52 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:48:23,702 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:52 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:48:23,702 INFO [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:37734) ARJUNA012340: RecoveryManager scan scheduled to begin. 08:48:23,702 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:52 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:52 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:23,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:23,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:23,705 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,705 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:23,705 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:48:23,705 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:23,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:23,706 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:48:23,707 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:48:23,707 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:48:23,707 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:48:23,707 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:48:23,707 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:33,707 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:48:33 08:48:33,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:48:33,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:33,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:33,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:33,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:48:33,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:33,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:33,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@411c1989, null) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081246838 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:48:33,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:48:33,709 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:48:33,710 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:33,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@108f561 >) 08:48:33,710 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@108f561 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:48:34,413 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$receiveMessage$2(RemoteConnectionChannel.java:361) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348) at java.lang.Thread.run(Thread.java:834) 08:48:34,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081314415 08:48:34,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:34,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:48:34,425 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@744bef6) 08:48:34,425 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:48:34,425 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:34,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:34,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:34,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:34,426 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:34,427 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:34,427 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7824507b, null) 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081267717 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:48:34,428 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:48:34,428 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:34,428 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1495dfe6 >) 08:48:34,428 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1495dfe6 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:34,487 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:48:34,488 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:48:34,488 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:34,488 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:48:34,488 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:34,489 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:34,489 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:48:34,489 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081314489 08:48:34,489 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:48:34,489 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:48:34,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:48:34,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:48:34,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:34,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:34,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:34,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:48:34,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@7a6242d9) 08:48:34,500 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:48:34,500 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:48:34,501 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:48:34,501 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:48:34,501 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:48:34,501 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:48:34,501 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:48:34,501 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:48:34,502 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:48:34,502 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:48:34,502 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:48:34,502 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:48:34,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081224950 1565081314502 true 08:48:34,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081224950 1565081314502 true 08:48:34,502 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:48:34,503 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:48:34,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:48:34,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:48:34,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: scan thread signals listener worker 08:48:34,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:48:36,397 DEBUG [com.arjuna.ats.arjuna] (Listener:4712) Connected to 10.128.1.113 on port 37904 on listener port 4712 for service com.arjuna.ats.internal.arjuna.recovery.WorkerService 08:48:36,398 DEBUG [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:37904) PeriodicRecovery: listener worker interrupts background thread 08:48:36,398 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:48:36,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:48:36,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:48:36 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:36,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,400 INFO [com.arjuna.ats.arjuna] (Server.Connection:10.128.1.113:37904) ARJUNA012340: RecoveryManager scan scheduled to begin. 08:48:36,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:36,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:36,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:36,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2c6cb023, null) 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081314415 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:36,402 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:48:36,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:36,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@6615aef0, null) 08:48:36,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:48:36,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:48:36,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081314489 08:48:36,404 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:36,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,405 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:48:36,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:36,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,405 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:48:36,405 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:36,405 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:36,405 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:36,405 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:48:36,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,406 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:48:36,406 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,406 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:5e OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:48:36,406 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:5e OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:48:36,407 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:5e OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:48:36,407 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:5e OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:36,407 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,407 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:36,407 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,408 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,408 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:36,408 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,408 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,408 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:36,409 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,409 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,409 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:48:36,409 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:36,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:36,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:36,412 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:36,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:36,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,413 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:36,413 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:48:36,413 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:48:36,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:36,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:36,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:36,414 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:48:36,415 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:48:36,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081316415 08:48:36,415 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:48:36,415 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:48:36,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081316415 08:48:36,415 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:48:36,415 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:48:46,415 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:48:46 08:48:46,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:48:46,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:48:46,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:46,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:46,416 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:46,416 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:46,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:48:46,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:48:46,417 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:48:46,417 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:46,417 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:46,418 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:48:46,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:46,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@d03f635, null) 08:48:46,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:48:46,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081314415 08:48:46,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:48:46,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:48:46,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:48:46,420 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:48:46,421 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:46,421 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@659f19c6 >) 08:48:46,421 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@659f19c6 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:48:46,936 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:48:46,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:48:46,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081326937 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:48:46,937 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,938 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:48:46,956 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@b058ce2) 08:48:46,957 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:48:46,957 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:46,957 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:46,957 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,957 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,957 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:46,958 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:48:46,958 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:46,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:46,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:46,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:48:46,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:46,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@171c2fc2, null) 08:48:46,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:48:46,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081314489 08:48:46,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:48:46,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:48:46,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:48:46,962 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:48:46,962 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:48:46,962 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:46,963 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@29e7c6b9 >) 08:48:46,963 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@29e7c6b9 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:47,005 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081327005 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:48:47,005 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:47,006 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:48:47,019 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@4655e721) 08:48:47,019 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:48:47,020 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:48:47,020 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081316415 1565081327020 false 08:48:47,020 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081316415 1565081327020 false 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:48:47,020 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: scan thread signals listener worker 08:48:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:48:48,577 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:48:48,577 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:48:48,577 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:48:48,577 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:67 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:48:48,577 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:67 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:48:48,578 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:67 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:48:48,578 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:67 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:48,578 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:48:48,579 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState() 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) OutputObjectState::OutputObjectState() 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:48,580 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@5d68643e, null) 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) BasicAction::restore_state () 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081327005 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Unpacked a 171 record 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::StateManager( 0:0:0:0:0 ) 08:48:48,581 TRACE [com.arjuna.ats.arjuna] (jolokia-0) AbstractRecord::AbstractRecord () - crash recovery constructor 08:48:48,581 TRACE [com.arjuna.ats.jta] (jolokia-0) XAResourceRecord.restore_state - XAResource de-serialized 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Unpacked a 463 record 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) HeuristicList - Unpacked heuristic list size of 0 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Restored action status of ActionStatus.COMMITTED 7 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Restored action type Top-level 0 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@f1c2ce7, null) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::activate( null) for object-id 0:0:0:0:0 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) StateManager::setupStore ( null ) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.read_committed(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, StateType.OS_ORIGINAL) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, StateType.OS_SHADOW) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, 10) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, StateType.OS_ORIGINAL) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) FileSystemStore.genPathName(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord, 11) 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.currentState(0:0:0:0:0, /StateManager/AbstractRecord/XAResourceRecord) - returning StateStatus.OS_UNKNOWN 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) ShadowingStore.read_state could not find committed or uncommitted state for 0:0:0:0:0 instead found state StateStatus.OS_UNKNOWN 08:48:48,582 WARN [com.arjuna.ats.arjuna] (jolokia-0) ARJUNA012035: Activate of object with id = 0:0:0:0:0 and type /StateManager/AbstractRecord/XAResourceRecord unexpectedly failed 08:48:48,582 TRACE [com.arjuna.ats.arjuna] (jolokia-0) Registering: jboss.jta:type=ObjectStore,itype=StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction,uid=0_ffff0a800171_35fe0b3b_5d493e2c_42 08:48:48,583 DEBUG [com.arjuna.ats.arjuna] (jolokia-0) registering bean jboss.jta:type=ObjectStore,itype=StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction,uid=0_ffff0a800171_35fe0b3b_5d493e2c_42 08:48:48,583 DEBUG [com.arjuna.ats.arjuna] (jolokia-0) registering bean jboss.jta:type=ObjectStore,itype=StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction,uid=0_ffff0a800171_35fe0b3b_5d493e2c_42,puid=0_ffff0a800171_35fe0b3b_5d493e2c_4c 08:49:39,455 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) InputObjectState::InputObjectState() 08:49:39,455 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:49:39,456 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) OutputObjectState::OutputObjectState() 08:49:39,456 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:6c OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:49:39,456 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:6c OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:49:39,456 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:6c OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:6c OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) InputObjectState::InputObjectState() 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) OutputObjectState::OutputObjectState() 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) InputObjectState::InputObjectState() 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) OutputObjectState::OutputObjectState() 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) InputObjectState::InputObjectState() 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:49:39,457 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) OutputObjectState::OutputObjectState() 08:49:39,458 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) InputObjectState::InputObjectState() 08:49:39,458 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:49:39,458 TRACE [com.arjuna.ats.arjuna] (management-handler-thread - 2) OutputObjectState::OutputObjectState() 08:50:45,678 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) ReaperThread.run () 08:50:45,678 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) TransactionReaper::check () 08:50:45,678 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) TransactionReaper::check - comparing 1565081445678 08:50:45,678 TRACE [com.arjuna.ats.arjuna] (Transaction Reaper) Thread Thread[Transaction Reaper,5,main] sleeping for 9223370471773330129 08:50:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:50:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:50:47,020 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:50:47 08:50:47,020 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:50:47,020 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:50:47,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,021 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:50:47,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:50:47,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,023 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@50880afe, null) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081326937 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,025 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@16c1396e, null) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081327005 08:50:47,027 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,027 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:50:47,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,027 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:50:47,028 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:50:47,028 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:50:47,028 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:50:47,028 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:50:47,028 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,028 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:50:47,029 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,029 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:74 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:50:47,029 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:74 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:50:47,029 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:74 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:50:47,029 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:74 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:50:47,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:47,031 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:47,032 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:50:47,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:50:47,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,032 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:50:47,032 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:50:47,032 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:50:47,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:47,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:50:47,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:47,033 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:50:47,034 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:50:47,034 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:50:47,034 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:50:47,034 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:50:47,034 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:50:57,034 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:50:57 08:50:57,034 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:50:57,034 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:50:57,034 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:50:57,034 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:57,035 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:50:57,035 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:50:57,035 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:50:57,035 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,035 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:57,036 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@3e64653d, null) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081326937 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:50:57,036 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:50:57,036 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:50:57,037 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:50:57,037 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:50:57,037 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:50:57,037 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@217131d6 >) 08:50:57,037 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@217131d6 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:50:57,059 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081457059 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:50:57,059 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:50:57,060 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:50:57,060 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,060 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,060 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,060 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,060 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:50:57,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@1e538f51) 08:50:57,071 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:57,071 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@373b38a0, null) 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:50:57,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081327005 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:50:57,072 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:50:57,072 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:50:57,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7af87668 >) 08:50:57,072 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7af87668 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:50:57,096 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081457096 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:50:57,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:50:57,097 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,097 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:50:57,097 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:50:57,097 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:50:57,116 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@405910a0) 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:50:57,117 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:50:57,117 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081316415 1565081457117 true 08:50:57,117 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081316415 1565081457117 true 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:50:57,117 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:50:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:52:57,117 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:52:57,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:52:57,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:52:57 08:52:57,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:52:57,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:52:57,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:52:57,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@5e0a2e29, null) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081457059 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@1dd4be61, null) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081457096 08:52:57,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:52:57,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:52:57,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:52:57,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:52:57,121 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:80 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:80 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:80 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:80 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:52:57,123 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:52:57,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,124 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:52:57,124 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:52:57,124 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:52:57,124 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:52:57,124 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:52:57,124 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:52:57,124 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:52:57,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:52:57,125 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081577125 08:52:57,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:52:57,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:52:57,125 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081577125 08:52:57,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:52:57,125 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:53:07,126 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:53:07 08:53:07,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:53:07,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:53:07,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:53:07,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:53:07,127 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:53:07,127 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:53:07,127 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:53:07,127 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:53:07,128 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@11e68956, null) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081457059 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:53:07,128 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:53:07,128 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:53:07,129 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:53:07,129 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1e128625 >) 08:53:07,129 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1e128625 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:53:07,154 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:53:07,154 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081587154 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,155 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@25a61610) 08:53:07,173 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:53:07,173 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,173 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@35e19bc8, null) 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081457096 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:53:07,174 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:53:07,174 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:53:07,174 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7e588940 >) 08:53:07,174 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7e588940 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:53:07,201 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:53:07,201 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:53:07,201 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:53:07,201 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:53:07,201 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081587202 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:53:07,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:53:07,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@29d487d) 08:53:07,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:53:07,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:53:07,214 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:53:07,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:53:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:53:07,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081577125 1565081587215 false 08:53:07,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081577125 1565081587215 false 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:53:07,215 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:53:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:53:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:53:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:55:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:55:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:55:07,215 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:55:07 08:55:07,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:55:07,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:55:07,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,216 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:55:07,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:55:07,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:07,217 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@38140af5, null) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081587154 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:07,218 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7a5d00aa, null) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:55:07,219 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081587202 08:55:07,219 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,220 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,220 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:55:07,220 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:55:07,220 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:55:07,220 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:55:07,220 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,220 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:8c OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:55:07,222 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:8c OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:8c OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:8c OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:55:07,223 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:07,224 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:55:07,224 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,225 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:55:07,225 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:55:07,225 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:55:07,225 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:07,225 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:55:07,225 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:07,226 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:55:07,226 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:55:07,227 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:55:07,227 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:55:07,227 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:55:07,227 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:55:17,227 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:55:17 08:55:17,227 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:55:17,227 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:55:17,227 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:55:17,227 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:17,228 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:55:17,228 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:55:17,228 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:55:17,228 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:17,228 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,228 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2332284, null) 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081587154 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:55:17,229 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:55:17,229 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:55:17,229 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@10f93a38 >) 08:55:17,229 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@10f93a38 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:55:17,248 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:55:17,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:55:17,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:55:17,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:55:17,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081717249 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,249 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:55:17,256 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@607d1f77) 08:55:17,257 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:17,257 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:55:17,257 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@25b7c2, null) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081587202 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:55:17,258 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:55:17,258 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:55:17,258 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:55:17,259 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:55:17,259 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7c8c43cb >) 08:55:17,259 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@7c8c43cb >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:55:17,276 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081717277 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:55:17,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:55:17,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:55:17,288 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@f694dea) 08:55:17,289 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:55:17,289 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:55:17,289 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:55:17,289 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:55:17,289 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:55:17,289 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081577125 1565081717289 true 08:55:17,289 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081577125 1565081717289 true 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:55:17,289 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:55:17,289 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:55:17,290 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:55:17,290 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:57:17,290 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:57:17,290 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:57:17,290 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:57:17 08:57:17,290 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:57:17,290 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,290 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:57:17,290 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,291 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:57:17,291 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,291 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:57:17,291 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@3a06ff87, null) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:57:17,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081717249 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:17,294 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@74fabd83, null) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:57:17,295 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081717277 08:57:17,296 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:57:17,296 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,296 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:57:17,296 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:57:17,296 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,296 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:57:17,296 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:57:17,296 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:57:17,297 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:57:17,297 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:57:17,297 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,297 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:57:17,297 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,297 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:98 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:57:17,297 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:98 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:57:17,298 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:98 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:57:17,298 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:98 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:17,298 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,298 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:57:17,298 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,298 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:17,299 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:17,299 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:57:17,300 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,300 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:57:17,300 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,300 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:57:17,300 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:57:17,300 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:57:17,300 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:17,300 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:57:17,300 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:17,300 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:57:17,346 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:57:17,346 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081837346 08:57:17,346 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:57:17,347 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:57:17,347 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081837347 08:57:17,347 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:57:17,347 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:57:27,347 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:57:27 08:57:27,347 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:57:27,347 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:57:27,347 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:57:27,347 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:27,347 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:57:27,347 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:57:27,347 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:57:27,348 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:57:27,348 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:27,348 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2c479d38, null) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081717249 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:57:27,348 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:57:27,349 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:57:27,349 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:57:27,349 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@d702b96 >) 08:57:27,349 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@d702b96 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:57:27,370 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081847370 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@41887763) 08:57:27,378 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:27,378 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:57:27,378 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@57faa612, null) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081717277 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:57:27,379 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:57:27,380 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:57:27,380 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:57:27,380 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1898eb61 >) 08:57:27,380 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1898eb61 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:57:27,395 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081847395 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:57:27,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:57:27,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@473c1a8e) 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:57:27,403 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:57:27,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081837346 1565081847403 false 08:57:27,403 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081837347 1565081847403 false 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:57:27,403 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:57:27,403 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 08:59:27,404 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 08:59:27,404 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 08:59:27,404 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 08:59:27 08:59:27,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 08:59:27,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:59:27,404 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,405 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:59:27,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:59:27,405 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,411 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:27,412 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@23c4d9b3, null) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081847370 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,413 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7aee835c, null) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081847395 08:59:27,414 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,414 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:59:27,414 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,414 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 08:59:27,414 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:59:27,414 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:59:27,414 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:59:27,415 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 08:59:27,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 08:59:27,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,415 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:a4 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 08:59:27,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:a4 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 08:59:27,416 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:a4 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:a4 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:59:27,422 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:27,423 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:27,424 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,424 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:59:27,424 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 08:59:27,424 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:59:27,424 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:27,425 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:59:27,432 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:59:27,432 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:59:27,432 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 08:59:27,433 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 08:59:27,433 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 08:59:37,433 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 08:59:37 08:59:37,433 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 08:59:37,433 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 08:59:37,433 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 08:59:37,433 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:37,433 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:59:37,433 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 08:59:37,434 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 08:59:37,434 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:37,434 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@51b79c81, null) 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081847370 08:59:37,434 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:59:37,435 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:59:37,435 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:59:37,435 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5f4d0188 >) 08:59:37,435 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5f4d0188 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 08:59:37,454 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081977455 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:59:37,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@113e44f4) 08:59:37,466 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:37,466 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 08:59:37,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 08:59:37,471 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,471 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@668c3dfe, null) 08:59:37,471 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081847395 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 08:59:37,472 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 08:59:37,472 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:59:37,472 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@547b405c >) 08:59:37,472 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@547b405c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:59:37,490 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 08:59:37,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 08:59:37,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 08:59:37,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 08:59:37,490 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081977491 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 08:59:37,491 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 08:59:37,498 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@473c1189) 08:59:37,498 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 08:59:37,498 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 08:59:37,498 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 08:59:37,498 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 08:59:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 08:59:37,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565081837346 1565081977499 true 08:59:37,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565081837347 1565081977499 true 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 08:59:37,499 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 08:59:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 08:59:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 08:59:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:01:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:01:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:01:37,499 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:01:37 09:01:37,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:01:37,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:01:37,499 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,500 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:01:37,500 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,500 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:01:37,500 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4c42734d, null) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081977455 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:37,501 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,502 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7e3eec8a, null) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081977491 09:01:37,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:01:37,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:01:37,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:01:37,503 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:01:37,503 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:b0 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:01:37,503 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:b0 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:b0 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:b0 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:01:37,504 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:37,505 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,505 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:01:37,505 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:01:37,505 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:01:37,505 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:37,506 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:01:37,507 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:01:37,507 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082097507 09:01:37,507 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:01:37,507 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:01:37,507 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082097507 09:01:37,507 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:01:37,507 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:01:47,507 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:01:47 09:01:47,507 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:01:47,507 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:01:47,507 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:01:47,507 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:47,508 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:01:47,508 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:01:47,508 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:01:47,508 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:47,508 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@182600a3, null) 09:01:47,508 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565081977455 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:01:47,509 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:01:47,509 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:01:47,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1175f591 >) 09:01:47,509 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1175f591 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:01:47,527 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:01:47,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082107527 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@f984cb1) 09:01:47,536 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:47,536 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,536 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@54b0870d, null) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565081977491 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:01:47,537 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:01:47,537 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:01:47,537 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1fec4e71 >) 09:01:47,538 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1fec4e71 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:01:47,553 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082107553 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:01:47,553 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:01:47,554 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:01:47,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@5754a0a9) 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:01:47,565 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:01:47,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082097507 1565082107565 false 09:01:47,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082097507 1565082107565 false 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:01:47,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:01:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:03:47,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:03:47,566 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:03:47,566 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:03:47 09:03:47,566 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:03:47,566 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,566 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:03:47,566 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:03:47,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:03:47,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:47,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:47,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@1c34d7ae, null) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082107527 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@4e67039c, null) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082107553 09:03:47,570 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,570 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:03:47,570 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,570 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:03:47,570 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:03:47,571 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:03:47,571 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:03:47,571 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:03:47,571 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,571 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:03:47,571 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,571 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:bc OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:03:47,571 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:bc OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:03:47,571 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:bc OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:bc OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:03:47,572 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,573 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,579 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:03:47,580 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:47,587 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:03:47,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,587 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:03:47,587 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:03:47,588 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:03:47,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:47,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:03:47,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:47,588 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:03:47,589 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:03:47,589 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:03:47,589 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:03:47,589 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:03:47,589 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:03:57,590 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:03:57 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:57,590 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:03:57,590 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:03:57,590 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:03:57,590 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:57,590 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:57,591 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@427daaa8, null) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082107527 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:03:57,591 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:03:57,592 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:03:57,592 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:03:57,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@34703a98 >) 09:03:57,592 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@34703a98 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:03:57,630 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:03:57,630 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:03:57,630 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:03:57,630 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:03:57,630 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082237631 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,631 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@1d62394c) 09:03:57,646 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:57,646 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:57,647 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@8be07ea, null) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082107553 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:03:57,647 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:03:57,647 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:03:57,647 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:03:57,648 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6e6c4b21 >) 09:03:57,648 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6e6c4b21 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:03:57,668 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:03:57,668 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:03:57,668 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:03:57,668 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:03:57,668 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:03:57,668 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,668 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082237669 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:03:57,669 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:03:57,677 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@5a3392c9) 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:03:57,678 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:03:57,678 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082097507 1565082237678 true 09:03:57,678 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082097507 1565082237678 true 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:03:57,678 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:03:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:05:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:05:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:05:57,678 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:05:57 09:05:57,678 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:05:57,678 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,678 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:05:57,678 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,679 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:05:57,679 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,679 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:05:57,679 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,679 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:05:57,679 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@7048fcc5, null) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082237631 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,680 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@157c354d, null) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082237669 09:05:57,681 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:05:57,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,681 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:05:57,869 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:05:57,869 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,870 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:05:57,870 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:05:57,870 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:05:57,870 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:05:57,870 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:05:57,870 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,870 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:05:57,873 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,874 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:c8 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:05:57,874 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:c8 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:05:57,875 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:c8 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:05:57,875 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:c8 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:05:57,875 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,876 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:05:57,876 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,876 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,876 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:05:57,876 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,877 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,877 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:05:57,877 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,878 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,878 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:05:57,878 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:05:57,881 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,881 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:05:57,881 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:05:57,881 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:05:57,881 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:05:57,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:05:57,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:05:57,883 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:05:57,928 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:05:57,929 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082357929 09:05:57,929 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:05:57,929 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:05:57,929 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082357929 09:05:57,929 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:05:57,929 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:06:07,930 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:06:07 09:06:07,930 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:06:07,930 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:06:07,930 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:06:07,930 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:06:07,930 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:06:07,930 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:06:07,930 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:06:07,930 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:06:07,930 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:06:07,930 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:06:07,931 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@7d37e5fd, null) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082237631 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:06:07,931 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:06:07,932 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:06:07,932 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:06:07,932 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@915714b >) 09:06:07,932 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@915714b >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:06:07,951 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082367951 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:06:07,959 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@182ac95c) 09:06:07,960 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:06:07,960 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@6b9fbb25, null) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082237669 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:06:07,960 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:06:07,961 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:06:07,961 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:06:07,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@28109006 >) 09:06:07,961 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@28109006 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:06:07,983 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082367983 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:06:07,983 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:06:07,991 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@d2eaaaf) 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:06:07,991 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:06:07,991 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082357929 1565082367991 false 09:06:07,991 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082357929 1565082367991 false 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:06:07,991 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:06:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:08:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:08:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:08:07,991 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:08:07 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:07,992 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:08:07,992 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@498a90ba, null) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:08:07,993 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082367951 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@59c32bfb, null) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082367983 09:08:07,994 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,994 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:07,994 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:08:07,994 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:08:07,994 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:08:07,994 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:08:07,994 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:08:07,994 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,995 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:08:07,995 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:07,995 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:d4 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:08:07,995 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:d4 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:08:07,995 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:d4 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:08:07,995 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:d4 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:07,997 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:07,998 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:08:07,998 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:08,000 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:08,000 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:08:08,000 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:08:08,000 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:08:08,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:08,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:08:08,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:08,008 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:08:08,009 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:08:08,009 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:08:08,009 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:08:08,009 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:08:08,009 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:08:18,009 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:08:18 09:08:18,009 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:08:18,009 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:08:18,009 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:08:18,009 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:18,010 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:08:18,010 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:08:18,010 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:08:18,010 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:18,010 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,010 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@6b1cde34, null) 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082367951 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:08:18,011 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:08:18,012 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:08:18,012 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:08:18,012 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2b40bc5f >) 09:08:18,012 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2b40bc5f >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:08:18,031 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082498032 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,032 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:08:18,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@11659f0c) 09:08:18,040 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:18,040 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@62c79a6, null) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082367983 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:08:18,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:08:18,041 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:08:18,041 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:08:18,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5e76b0f0 >) 09:08:18,041 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5e76b0f0 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:08:18,056 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082498057 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:08:18,057 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:08:18,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@8c7e351) 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:08:18,065 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:08:18,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082357929 1565082498065 true 09:08:18,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082357929 1565082498065 true 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:08:18,065 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:08:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:10:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:10:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:10:18,065 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:10:18 09:10:18,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:10:18,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:10:18,066 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,066 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:10:18,066 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,066 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:10:18,066 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:18,067 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@70698c19, null) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082498032 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@740752da, null) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082498057 09:10:18,068 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,068 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:10:18,068 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,069 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:10:18,069 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:10:18,069 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:10:18,069 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:10:18,069 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:e0 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:e0 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:e0 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:10:18,069 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:e0 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:10:18,070 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:18,071 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,071 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:10:18,071 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:10:18,071 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:10:18,071 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:18,071 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:10:18,072 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:10:18,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082618073 09:10:18,073 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:10:18,073 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:10:18,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082618073 09:10:18,073 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:10:18,073 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:10:28,073 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:10:28 09:10:28,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:10:28,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:10:28,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:10:28,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:28,073 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:10:28,074 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:10:28,074 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:10:28,074 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:28,074 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@749aa2ab, null) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082498032 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:10:28,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:10:28,075 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:10:28,075 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:10:28,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3ac5af60 >) 09:10:28,076 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3ac5af60 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:10:28,174 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082628175 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,175 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@35b56227) 09:10:28,183 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:28,184 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7d3d042c, null) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082498057 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:10:28,184 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:10:28,184 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:10:28,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@40ba61eb >) 09:10:28,185 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@40ba61eb >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:10:28,202 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082628202 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:10:28,202 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:10:28,210 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@5f65ec53) 09:10:28,210 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:10:28,210 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:10:28,210 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:10:28,210 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:10:28,210 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:10:28,210 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:10:28,211 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082618073 1565082628211 false 09:10:28,211 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082618073 1565082628211 false 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:10:28,211 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:10:28,211 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:10:28,211 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:10:28,211 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:12:28,211 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:12:28,211 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:12:28,211 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:12:28 09:12:28,211 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:12:28,211 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,211 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:12:28,211 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,212 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:12:28,212 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,212 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:12:28,212 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,212 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:28,212 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@740a1f40, null) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082628175 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,213 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@17943ea3, null) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082628202 09:12:28,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:12:28,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:12:28,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:12:28,214 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:12:28,214 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,214 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:ec OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:ec OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:ec OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:ec OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:12:28,215 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:28,216 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,216 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:12:28,216 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:12:28,216 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:12:28,216 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:28,217 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:12:28,263 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:12:28,263 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:12:28,263 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:12:28,263 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:12:28,263 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:12:38,263 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:12:38 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:38,264 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:12:38,264 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:12:38,264 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:12:38,264 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:38,264 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:38,265 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@3077bfe2, null) 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082628175 09:12:38,265 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:12:38,266 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:12:38,266 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:12:38,266 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6a0fe69 >) 09:12:38,266 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6a0fe69 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:12:38,283 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082758284 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,284 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:12:38,291 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@48c903bf) 09:12:38,292 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:38,292 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@3ccdb2b0, null) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082628202 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:12:38,292 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:12:38,293 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:12:38,293 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:12:38,293 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2a3da469 >) 09:12:38,293 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2a3da469 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:12:38,313 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082758319 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:12:38,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:12:38,335 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@4f75fc7) 09:12:38,335 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:12:38,335 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:12:38,335 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:12:38,335 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:12:38,335 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:12:38,335 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:12:38,335 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:12:38,335 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:12:38,335 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:12:38,335 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:12:38,336 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:12:38,336 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:12:38,336 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082618073 1565082758336 true 09:12:38,336 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082618073 1565082758336 true 09:12:38,336 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:12:38,336 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:12:38,336 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:12:38,336 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:12:38,336 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:14:38,336 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:14:38,336 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:14:38,336 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:14:38 09:14:38,336 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:14:38,336 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,336 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:14:38,336 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,337 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,337 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@6e8f8a, null) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082758284 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,338 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@61915bf8, null) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082758319 09:14:38,339 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,339 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,339 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:14:38,339 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:14:38,339 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:14:38,339 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:14:38,339 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,339 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:f8 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:f8 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:f8 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:f8 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:14:38,340 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,341 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:38,342 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,342 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:14:38,342 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:14:38,342 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:14:38,342 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:38,342 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:14:38,387 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:14:38,387 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082878387 09:14:38,387 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:14:38,387 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:14:38,387 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082878387 09:14:38,387 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:14:38,387 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:14:48,387 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:14:48 09:14:48,387 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:14:48,387 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:14:48,388 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:14:48,388 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:48,389 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:14:48,389 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:14:48,389 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:14:48,389 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:14:48,389 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:14:48,389 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:14:48,389 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:48,389 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:48,390 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@9b66d00, null) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082758284 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:14:48,390 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:14:48,390 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:14:48,390 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:14:48,391 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6213b955 >) 09:14:48,391 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6213b955 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:14:48,409 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082888410 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,410 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@20dc6c1d) 09:14:48,418 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:48,418 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:14:48,418 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@286f1e0b, null) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082758319 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:14:48,419 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:14:48,419 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:14:48,419 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4a16d05b >) 09:14:48,419 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4a16d05b >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:14:48,439 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082888439 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:14:48,439 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:14:48,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@7feb2543) 09:14:48,450 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:14:48,450 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:14:48,450 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:14:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:14:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:14:48,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082878387 1565082888451 false 09:14:48,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082878387 1565082888451 false 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:14:48,451 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:14:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:14:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:14:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:16:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:16:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:16:48,451 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:16:48 09:16:48,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:16:48,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:16:48,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,452 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@3f8fc718, null) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082888410 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@62051f30, null) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:16:48,453 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082888439 09:16:48,454 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,454 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,454 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:16:48,454 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:16:48,454 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:16:48,454 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:16:48,454 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:104 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:104 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:104 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:16:48,454 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:104 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:48,455 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:48,456 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,456 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:16:48,456 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:16:48,456 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:16:48,456 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:48,457 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:16:48,458 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:16:48,458 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:16:48,458 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:16:48,458 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:16:48,458 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:16:58,458 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:16:58 09:16:58,458 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:16:58,458 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:16:58,458 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:16:58,458 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:58,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:16:58,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:16:58,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:16:58,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:58,459 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@6070fdd7, null) 09:16:58,459 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565082888410 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:16:58,460 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:16:58,460 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:16:58,460 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2ac69276 >) 09:16:58,460 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2ac69276 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:16:58,478 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083018478 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:16:58,478 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,479 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@76873f54) 09:16:58,486 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,486 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:58,487 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@4eb62cf9, null) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565082888439 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:16:58,487 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:16:58,487 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:16:58,494 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:16:58,494 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@50a5a614 >) 09:16:58,494 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@50a5a614 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:16:58,509 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083018509 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:16:58,509 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:16:58,522 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@626147be) 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:16:58,522 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:16:58,522 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565082878387 1565083018522 true 09:16:58,522 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565082878387 1565083018522 true 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:16:58,522 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:16:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:18:58,522 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:18:58,523 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:18:58,523 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:18:58 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,523 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:18:58,523 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@44e5b17c, null) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083018478 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:18:58,524 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@5cdbda3c, null) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083018509 09:18:58,525 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,525 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,525 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:18:58,525 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:18:58,525 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:18:58,525 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:18:58,525 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,525 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:110 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:110 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:110 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:110 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,526 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:18:58,527 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:18:58,527 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,527 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:18:58,528 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:18:58,528 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:18:58,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:18:58,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:18:58,528 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:18:58,528 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:18:58,529 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:18:58,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083138529 09:18:58,529 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:18:58,529 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:18:58,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083138529 09:18:58,529 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:18:58,530 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:19:08,530 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:19:08 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:19:08,530 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:19:08,530 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:19:08,530 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:19:08,530 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:19:08,530 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:19:08,531 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,531 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4955e7a, null) 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083018478 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:19:08,539 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:19:08,539 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:19:08,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@61d7eee >) 09:19:08,539 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@61d7eee >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:19:08,564 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:19:08,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:19:08,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083148565 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@55f41b6c) 09:19:08,577 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:19:08,577 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,577 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7b7777b1, null) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083018509 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:19:08,578 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:19:08,578 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:19:08,578 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1efeaa00 >) 09:19:08,578 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@1efeaa00 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:19:08,602 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083148602 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:19:08,602 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:19:08,610 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@4f5d910) 09:19:08,610 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:19:08,610 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:19:08,610 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:19:08,610 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:19:08,610 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:19:08,610 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083138529 1565083148610 false 09:19:08,610 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083138529 1565083148610 false 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:19:08,610 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:19:08,611 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:19:08,611 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:19:08,611 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:21:08,611 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:21:08,611 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:21:08,611 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:21:08 09:21:08,611 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:21:08,611 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,611 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:21:08,611 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,612 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,612 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@18c7d37e, null) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083148565 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@149f1413, null) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083148602 09:21:08,614 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,614 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:21:08,614 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,615 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:21:08,615 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:21:08,615 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:21:08,615 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:21:08,615 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:11c OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:11c OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:11c OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:21:08,615 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:11c OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:08,616 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:08,617 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,617 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:21:08,617 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:21:08,617 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:21:08,617 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:08,617 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:21:08,663 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:21:08,663 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:21:08,663 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:21:08,663 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:21:08,663 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:21:18,663 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:21:18 09:21:18,663 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:21:18,663 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:21:18,663 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:21:18,663 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:18,664 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:21:18,664 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:21:18,664 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:21:18,664 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:18,664 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@5ac9d8ac, null) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083148565 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:21:18,664 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:21:18,665 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:21:18,665 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:21:18,665 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@369a606a >) 09:21:18,665 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@369a606a >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:21:18,774 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083278774 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:21:18,774 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,775 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:21:18,793 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@eacd994) 09:21:18,794 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:18,794 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@38b3f565, null) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083148602 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:21:18,794 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:21:18,795 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:21:18,795 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:21:18,795 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@76884d46 >) 09:21:18,795 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@76884d46 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:21:18,811 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083278811 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:21:18,811 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:21:18,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:21:18,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:21:18,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:21:18,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:21:18,812 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:21:18,834 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@4fefde63) 09:21:18,834 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:21:18,834 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:21:18,834 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:21:18,834 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:21:18,834 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:21:18,834 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083138529 1565083278834 true 09:21:18,834 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083138529 1565083278834 true 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:21:18,834 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:21:18,835 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:21:18,835 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:21:18,835 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:23:18,835 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:23:18,835 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:23:18,835 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:23:18 09:23:18,835 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:23:18,835 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,835 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:23:18,835 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,836 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,836 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2a13c6ab, null) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083278774 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,837 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@15f11bf9, null) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083278811 09:23:18,838 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,838 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,838 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:23:18,838 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:23:18,838 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:23:18,838 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:23:18,838 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,838 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:128 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:23:18,839 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:128 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:23:18,839 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:128 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:23:18,839 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:128 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:18,839 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,839 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:23:18,839 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:18,840 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:23:18,840 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,841 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:23:18,841 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:23:18,841 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:23:18,841 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:18,841 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:23:18,841 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:18,841 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:23:18,885 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:23:18,885 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083398885 09:23:18,885 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:23:18,885 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:23:18,885 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083398885 09:23:18,885 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:23:18,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:23:28,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:23:28 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:28,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:23:28,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:23:28,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:23:28,886 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:28,886 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:28,887 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@6ceb1e34, null) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083278774 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:23:28,887 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:23:28,888 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:23:28,888 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:23:28,888 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@71d8ee83 >) 09:23:28,888 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@71d8ee83 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:23:28,913 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083408913 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:23:28,913 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,914 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@46683cb4) 09:23:28,921 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:28,921 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:28,922 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7b7b3bc8, null) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083278811 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:23:28,922 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:23:28,922 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:23:28,922 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@134a3fe0 >) 09:23:28,922 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@134a3fe0 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:23:28,936 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083408936 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:23:28,936 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:23:28,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@1aced5c4) 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:23:28,943 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:23:28,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083398885 1565083408943 false 09:23:28,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083398885 1565083408943 false 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:23:28,943 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:23:28,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:25:28,944 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:25:28,944 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:25:28,944 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:25:28 09:25:28,944 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:25:28,944 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,944 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:25:28,944 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,945 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,945 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@23f85aa3, null) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083408913 09:25:28,946 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7088b0ad, null) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083408936 09:25:28,947 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,947 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:25:28,947 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,948 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:25:28,948 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:25:28,948 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:25:28,948 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:25:28,948 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:134 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:134 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:134 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:25:28,948 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:134 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:25:28,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:28,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:28,951 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,951 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:25:28,951 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:25:28,951 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:25:28,951 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:28,951 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:25:28,952 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:25:28,952 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:25:28,952 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:25:28,952 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:25:28,953 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:25:38,953 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:25:38 09:25:38,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:25:38,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:25:38,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:25:38,953 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:38,954 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:25:38,954 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:25:38,954 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:25:38,954 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:38,954 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:38,954 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@5b837c72, null) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083408913 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:25:38,955 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:25:38,955 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:25:38,956 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:25:38,956 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:25:38,956 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:25:38,956 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:25:38,956 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:25:38,956 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6e3ba277 >) 09:25:38,956 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6e3ba277 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:25:38,987 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083538988 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:38,988 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@283bdc40) 09:25:39,000 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:39,000 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,000 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@60c7e4aa, null) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083408936 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:25:39,001 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:25:39,001 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:25:39,001 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:25:39,003 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@668f8cf5 >) 09:25:39,003 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@668f8cf5 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:25:39,020 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083539021 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:25:39,021 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:25:39,022 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:25:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@71fc966a) 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:25:39,030 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:25:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083398885 1565083539030 true 09:25:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083398885 1565083539030 true 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:25:39,030 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:25:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:27:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:27:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:27:39,030 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:27:39 09:27:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:27:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:27:39,030 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,037 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:27:39,037 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,037 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:27:39,037 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4242f3c2, null) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083538988 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:39,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@e838f7f, null) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083539021 09:27:39,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:27:39,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:27:39,040 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:27:39,040 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:27:39,040 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:27:39,040 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:140 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:140 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:140 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:140 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:27:39,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:39,041 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:27:39,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,041 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:27:39,042 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:27:39,042 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:27:39,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:39,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:27:39,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:39,042 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:27:39,043 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:27:39,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083659043 09:27:39,043 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:27:39,043 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:27:39,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083659043 09:27:39,043 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:27:39,043 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:27:49,043 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:27:49 09:27:49,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:27:49,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:27:49,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:27:49,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:49,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:27:49,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:27:49,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:27:49,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:49,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:27:49,051 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,051 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@776f8de, null) 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083538988 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:27:49,052 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:27:49,052 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:27:49,052 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@151c6738 >) 09:27:49,052 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@151c6738 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:27:49,074 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:27:49,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:27:49,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:27:49,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:27:49,074 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083669075 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,075 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@4274618a) 09:27:49,083 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:49,083 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,083 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@22f02236, null) 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083539021 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:27:49,086 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:27:49,086 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:27:49,086 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4abe2e6c >) 09:27:49,086 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4abe2e6c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:27:49,107 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083669107 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:27:49,107 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:27:49,108 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:27:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@4ee5c2b6) 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:27:49,118 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:27:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083659043 1565083669118 false 09:27:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083659043 1565083669118 false 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:27:49,118 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:27:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:29:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:29:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:29:49,118 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:29:49 09:29:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:29:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:29:49,118 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,119 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:29:49,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:29:49,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,119 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@1000e4b7, null) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083669075 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:49,120 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@5af08f6c, null) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083669107 09:29:49,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:29:49,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:29:49,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:29:49,121 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:29:49,121 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,121 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:14c OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:14c OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:14c OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:14c OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:29:49,122 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:49,123 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,123 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:29:49,123 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:29:49,123 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:29:49,123 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:49,124 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:29:49,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:29:49,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:29:49,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:29:49,125 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:29:49,125 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:29:59,125 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:29:59 09:29:59,125 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:29:59,125 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:29:59,125 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:29:59,125 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:59,126 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:29:59,126 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:29:59,126 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:29:59,126 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:59,126 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4f78d798, null) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083669075 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:29:59,126 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:29:59,127 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:29:59,127 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:29:59,127 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@112daaf4 >) 09:29:59,127 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@112daaf4 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:29:59,143 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083799143 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:29:59,143 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,144 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@4038517b) 09:29:59,152 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:59,152 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:29:59,152 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@2253d652, null) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083669107 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:29:59,153 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:29:59,153 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:29:59,153 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5c18ae8d >) 09:29:59,153 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5c18ae8d >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:29:59,172 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083799172 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:29:59,172 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:29:59,180 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@5bc65cdf) 09:29:59,180 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:29:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:29:59,181 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:29:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:29:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:29:59,181 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083659043 1565083799181 true 09:29:59,181 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083659043 1565083799181 true 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:29:59,181 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:29:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:29:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:29:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:31:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:31:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:31:59,181 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:31:59 09:31:59,181 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:31:59,181 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,181 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:31:59,181 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,182 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,182 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@258cdc1c, null) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083799143 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@408d319d, null) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083799172 09:31:59,183 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,183 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:31:59,183 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,184 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:31:59,184 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:31:59,184 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:31:59,184 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:31:59,184 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:158 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:158 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:158 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:158 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:31:59,184 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:31:59,185 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:31:59,185 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,185 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:31:59,186 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:31:59,186 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:31:59,186 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:31:59,186 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:31:59,186 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:31:59,186 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:31:59,231 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:31:59,231 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083919231 09:31:59,231 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:31:59,231 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:31:59,231 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083919231 09:31:59,231 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:31:59,231 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:32:09,231 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:32:09 09:32:09,231 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:32:09,231 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:32:09,231 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:32:09,231 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:32:09,232 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:32:09,232 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:32:09,232 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:32:09,232 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:32:09,232 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@1eda96ec, null) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083799143 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:32:09,232 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:32:09,233 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:32:09,233 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:32:09,233 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@37725d73 >) 09:32:09,233 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@37725d73 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:32:09,247 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083929248 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,248 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@4b888836) 09:32:09,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:32:09,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@23d8a41d, null) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083799172 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:32:09,255 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:32:09,255 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:32:09,255 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:32:09,256 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@39798bc >) 09:32:09,256 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@39798bc >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:32:09,269 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083929269 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:32:09,269 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:32:09,275 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@716c3960) 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:32:09,276 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:32:09,276 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083919231 1565083929276 false 09:32:09,276 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083919231 1565083929276 false 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:32:09,276 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:32:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:34:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:34:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:34:09,276 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:34:09 09:34:09,276 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:34:09,276 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,276 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:34:09,276 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,277 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,277 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2ca1bfbb, null) 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083929248 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:09,278 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@a495ec2, null) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083929269 09:34:09,448 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,448 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:34:09,448 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,449 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:34:09,449 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:34:09,449 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:34:09,449 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:34:09,449 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:34:09,449 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,449 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:34:09,449 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,449 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:164 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:34:09,449 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:164 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:34:09,449 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:164 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:164 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:34:09,450 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:09,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:34:09,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:34:09,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:34:09,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:09,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:34:09,510 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:34:09,510 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:34:09,510 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:34:09,510 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:34:09,510 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:34:19,510 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:34:19 09:34:19,510 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:34:19,510 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:34:19,510 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:34:19,510 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:19,512 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:34:19,512 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:34:19,512 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:34:19,512 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:19,512 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4f6ad0a3, null) 09:34:19,512 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565083929248 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:34:19,513 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:34:19,513 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:34:19,513 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@f1528c2 >) 09:34:19,513 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@f1528c2 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:34:19,528 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084059529 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,529 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@4cf19bde) 09:34:19,538 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:19,538 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@5d304077, null) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565083929269 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:34:19,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:34:19,539 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:34:19,539 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:34:19,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@19add9c >) 09:34:19,539 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@19add9c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:34:19,551 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084059551 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:34:19,551 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:34:19,552 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:34:19,552 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,552 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:34:19,552 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:34:19,552 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:34:19,560 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@4ceed3bd) 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:34:19,560 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:34:19,560 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565083919231 1565084059560 true 09:34:19,560 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565083919231 1565084059560 true 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:34:19,560 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:34:19,560 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:36:19,561 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:36:19,561 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:36:19,561 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:36:19 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,561 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:36:19,561 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@161bea63, null) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084059529 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,562 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@164e3cd4, null) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084059551 09:36:19,563 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,563 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,563 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:36:19,563 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:36:19,563 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:36:19,563 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:36:19,563 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:36:19,563 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:170 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:170 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:170 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:170 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:36:19,564 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:19,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,565 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:36:19,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:36:19,565 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:36:19,565 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:19,566 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:36:19,566 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:36:19,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084179567 09:36:19,567 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:36:19,567 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:36:19,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084179567 09:36:19,567 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:36:19,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:36:29,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:36:29 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:29,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:36:29,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:36:29,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:36:29,567 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:29,567 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:29,568 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4e0b52b9, null) 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084059529 09:36:29,568 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:36:29,569 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:36:29,569 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:36:29,569 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@29447f30 >) 09:36:29,569 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@29447f30 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:36:29,587 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084189587 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:36:29,587 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,588 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@4e159c16) 09:36:29,596 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:29,596 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:29,597 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@40a0261b, null) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084059551 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:36:29,597 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:36:29,597 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:36:29,597 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@552085e8 >) 09:36:29,597 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@552085e8 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:36:29,627 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084189627 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:36:29,627 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:36:29,635 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@45824111) 09:36:29,635 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:36:29,635 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:36:29,635 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:36:29,635 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:36:29,635 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:36:29,636 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084179567 1565084189636 false 09:36:29,636 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084179567 1565084189636 false 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:36:29,636 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:36:29,636 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:36:29,636 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:36:29,636 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:38:29,636 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:38:29,636 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:38:29,636 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:38:29 09:38:29,636 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:38:29,636 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,636 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:38:29,636 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,637 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:38:29,637 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,637 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:38:29,637 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4f6e4f67, null) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084189587 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,638 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@1351b0b1, null) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084189627 09:38:29,639 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,639 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,639 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:38:29,639 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:38:29,639 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:38:29,639 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:38:29,639 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,639 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:17c OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:17c OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:17c OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:17c OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:38:29,640 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:29,641 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,641 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:38:29,641 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:38:29,641 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:38:29,641 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:29,641 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:38:29,643 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:38:29,643 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:38:29,643 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:38:29,643 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:38:29,643 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:38:39,643 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:38:39 09:38:39,643 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:38:39,643 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:38:39,643 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:38:39,643 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:39,644 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:38:39,644 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:38:39,644 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:38:39,644 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:39,644 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@cc44269, null) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084189587 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:38:39,644 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:38:39,645 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:38:39,645 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:38:39,645 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@120eaf78 >) 09:38:39,645 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@120eaf78 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:38:39,669 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084319670 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,670 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@3a7a5172) 09:38:39,681 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:39,681 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@37dc9797, null) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084189627 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:38:39,682 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:38:39,682 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:38:39,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@546332e9 >) 09:38:39,682 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@546332e9 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:38:39,700 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084319700 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:38:39,700 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:38:39,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@2c4c66fa) 09:38:39,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:38:39,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:38:39,709 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:38:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:38:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:38:39,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084179567 1565084319709 true 09:38:39,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084179567 1565084319709 true 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:38:39,709 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:38:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:38:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:38:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:40:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:40:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:40:39,709 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:40:39 09:40:39,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:40:39,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:40:39,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,710 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@65b63c9, null) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084319670 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7fb5781e, null) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084319700 09:40:39,711 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,711 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:40:39,711 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,712 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:40:39,712 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:40:39,712 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:40:39,712 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:40:39,712 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:188 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:188 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:188 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:188 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:40:39,712 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:39,713 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:40:39,713 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,713 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:40:39,713 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:40:39,714 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:40:39,714 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:39,714 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:40:39,714 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:39,714 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:40:39,715 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:40:39,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084439716 09:40:39,716 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:40:39,716 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:40:39,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084439716 09:40:39,716 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:40:39,716 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:40:49,716 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:40:49 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:49,716 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:40:49,716 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:40:49,716 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:40:49,716 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:49,716 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:49,717 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@d9a1d69, null) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084319670 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:40:49,717 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:40:49,717 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:40:49,717 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:40:49,718 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:40:49,718 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@59bbe14c >) 09:40:49,718 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@59bbe14c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:40:49,737 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084449737 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,737 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:40:49,744 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@5241fcde) 09:40:49,745 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:49,745 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7840b99a, null) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084319700 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:40:49,745 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:40:49,745 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:40:49,745 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:40:49,746 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@16dab46a >) 09:40:49,746 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@16dab46a >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:40:49,762 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084449762 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:40:49,762 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:40:49,769 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@fee31da) 09:40:49,769 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:40:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:40:49,770 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:40:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:40:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:40:49,770 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084439716 1565084449770 false 09:40:49,770 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084439716 1565084449770 false 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:40:49,770 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:40:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:40:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:40:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:42:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:42:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:42:49,770 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:42:49 09:42:49,770 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:42:49,770 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,770 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:42:49,770 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,771 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,771 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@310f8471, null) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084449737 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,772 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@5829567a, null) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084449762 09:42:49,773 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,773 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,773 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:42:49,773 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:42:49,773 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:42:49,773 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:42:49,773 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,773 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:194 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:194 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:194 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:194 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:42:49,777 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:49,778 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,778 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:42:49,778 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:42:49,778 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:42:49,778 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:49,779 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:42:49,779 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:49,779 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:42:49,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:42:49,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:42:49,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:42:49,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:42:49,824 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:42:59,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:42:59 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:59,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:42:59,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:42:59,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:42:59,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:59,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@54dd6db1, null) 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084449737 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:42:59,826 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:42:59,826 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:42:59,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@46d57bd6 >) 09:42:59,826 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@46d57bd6 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:42:59,849 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084579850 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,850 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:42:59,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@15e1fc6b) 09:42:59,861 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:59,861 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@381d25b0, null) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084449762 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:42:59,861 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:42:59,861 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:42:59,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2d67c81 >) 09:42:59,861 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2d67c81 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:42:59,882 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084579882 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:42:59,882 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:42:59,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:42:59,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@7ff5376e) 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:42:59,892 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:42:59,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084439716 1565084579892 true 09:42:59,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084439716 1565084579892 true 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:42:59,892 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:42:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:44:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:44:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:44:59,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:44:59 09:44:59,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:44:59,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:44:59,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,893 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@76646b7d, null) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084579850 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@484bde4c, null) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084579882 09:44:59,894 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,894 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:44:59,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:44:59,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:44:59,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:44:59,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:44:59,895 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1a0 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1a0 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1a0 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1a0 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:44:59,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:44:59,896 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:44:59,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,896 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:44:59,897 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:44:59,897 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:44:59,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:44:59,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:44:59,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:44:59,897 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:44:59,941 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:44:59,941 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084699941 09:44:59,941 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:44:59,941 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:44:59,941 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084699941 09:44:59,941 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:44:59,941 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:45:09,941 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:45:09 09:45:09,941 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:45:09,941 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:45:09,941 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:45:09,941 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:45:09,942 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:45:09,942 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:45:09,942 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:45:09,942 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:45:09,942 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@506a8876, null) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084579850 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:45:09,942 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:45:09,942 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:45:09,943 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:45:09,943 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@46458589 >) 09:45:09,943 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@46458589 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:45:09,961 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084709961 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,961 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@22a8b613) 09:45:09,971 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:45:09,971 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:09,971 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@678c7824, null) 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084579882 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:45:09,972 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:45:09,972 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:45:09,972 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@32e7e3d7 >) 09:45:09,972 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@32e7e3d7 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:45:10,026 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:45:10,026 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084710027 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:45:10,027 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:45:10,038 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@26adf026) 09:45:10,038 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:45:10,038 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:45:10,038 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:45:10,038 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:45:10,038 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:45:10,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084699941 1565084710039 false 09:45:10,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084699941 1565084710039 false 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:45:10,039 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:45:10,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:45:10,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:45:10,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:47:10,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:47:10,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:47:10,039 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:47:10 09:47:10,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:47:10,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:47:10,039 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,040 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,040 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@959689f, null) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084709961 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@4d9cbe72, null) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084710027 09:47:10,041 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,041 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:47:10,041 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,042 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:47:10,042 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:47:10,042 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:47:10,042 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:47:10,042 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1ac OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1ac OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1ac OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:47:10,042 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1ac OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,043 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:10,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,044 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:47:10,044 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:47:10,044 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:47:10,044 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:10,044 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:47:10,045 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:47:10,045 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:47:10,045 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:47:10,045 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:47:10,045 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:47:20,045 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:47:20 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:20,046 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:47:20,046 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:47:20,046 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:47:20,046 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:20,046 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,046 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@7e4c53e6, null) 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084709961 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:47:20,047 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:47:20,047 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:47:20,047 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@568b0d73 >) 09:47:20,047 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@568b0d73 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:47:20,064 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084840064 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:47:20,064 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,065 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@518e455f) 09:47:20,072 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:20,072 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@2f78b176, null) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084710027 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:47:20,072 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:47:20,072 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:47:20,072 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:47:20,073 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3a5db0bd >) 09:47:20,073 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@3a5db0bd >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:47:20,088 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084840089 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:47:20,089 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:47:20,096 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@90868e1) 09:47:20,096 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:47:20,096 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:47:20,096 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:47:20,096 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:47:20,096 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:47:20,096 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:47:20,314 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:47:20,314 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:47:20,314 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:47:20,314 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:47:20,314 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:47:20,314 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:47:20,315 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084699941 1565084840315 true 09:47:20,315 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084699941 1565084840315 true 09:47:20,315 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:47:20,315 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:47:20,315 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:47:20,315 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:47:20,315 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:49:20,315 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:49:20,315 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:49:20,315 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:49:20 09:49:20,315 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:49:20,315 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,315 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:49:20,315 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,316 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,316 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@6e09d8, null) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084840064 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:49:20,317 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@7b9d5d5c, null) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084840089 09:49:20,318 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,318 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,318 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:49:20,318 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:49:20,318 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:49:20,318 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:49:20,318 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1b8 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1b8 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:49:20,318 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1b8 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1b8 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:49:20,319 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:20,320 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,320 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:49:20,320 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:49:20,320 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:49:20,320 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:20,321 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:49:20,322 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:49:20,322 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084960322 09:49:20,322 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:49:20,322 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:49:20,322 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084960322 09:49:20,322 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:49:20,322 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:49:30,322 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:49:30 09:49:30,322 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:49:30,322 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:49:30,322 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:49:30,322 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:30,322 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:49:30,323 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:49:30,323 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:49:30,323 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:30,323 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@3e6e3aef, null) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084840064 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:49:30,323 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:49:30,324 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:49:30,324 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:49:30,324 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2f08e7c3 >) 09:49:30,324 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2f08e7c3 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:49:30,342 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084970343 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,343 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@6cf31af0) 09:49:30,351 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:30,351 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:30,352 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@72fdc2e7, null) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084840089 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:49:30,352 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:49:30,352 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:49:30,352 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@124be36 >) 09:49:30,352 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@124be36 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:49:30,370 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:49:30,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:49:30,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:49:30,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:49:30,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:49:30,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,370 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084970370 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:49:30,371 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:49:30,392 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@2776ca6) 09:49:30,392 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:49:30,392 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:49:30,392 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:49:30,392 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:49:30,392 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:49:30,392 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:49:30,392 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084960322 1565084970392 false 09:49:30,393 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084960322 1565084970393 false 09:49:30,393 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:49:30,393 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:49:30,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:49:30,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:49:30,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:51:30,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:51:30,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:51:30,393 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:51:30 09:51:30,393 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:51:30,393 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,393 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:51:30,393 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,394 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,394 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@429311dc, null) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084970343 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@beeaa87, null) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084970370 09:51:30,395 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,395 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:51:30,395 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,396 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:51:30,396 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:51:30,396 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:51:30,396 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:51:30,396 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1c4 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1c4 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1c4 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1c4 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,396 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:30,397 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:51:30,397 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,398 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:51:30,398 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:51:30,398 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:51:30,398 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:30,398 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:51:30,398 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:30,398 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:51:30,399 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:51:30,399 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:51:30,399 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:51:30,399 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:51:30,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:51:40,399 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:51:40 09:51:40,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:51:40,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:51:40,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:51:40,399 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:40,400 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:51:40,400 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:51:40,400 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:51:40,400 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:40,400 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@664e1185, null) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565084970343 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:51:40,400 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:51:40,401 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:51:40,401 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:51:40,401 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@188a2559 >) 09:51:40,401 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@188a2559 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:51:40,420 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085100420 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,420 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@40d9d83f) 09:51:40,429 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:40,429 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@b3c52e1, null) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565084970370 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:51:40,429 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:51:40,430 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:51:40,430 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:51:40,430 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2e8c072f >) 09:51:40,430 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2e8c072f >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:51:40,451 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085100451 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:51:40,451 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:51:40,452 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:51:40,464 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@761ef903) 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:51:40,465 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:51:40,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565084960322 1565085100465 true 09:51:40,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565084960322 1565085100465 true 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:51:40,465 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:51:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:53:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:53:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:53:40,465 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:53:40 09:53:40,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:53:40,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:53:40,465 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,466 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:53:40,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:53:40,466 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@156cff0f, null) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:53:40,467 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085100420 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@ec6dcce, null) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085100451 09:53:40,468 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,468 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,468 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:53:40,468 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:53:40,468 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:53:40,468 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:53:40,468 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:53:40,468 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1d0 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1d0 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1d0 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1d0 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:53:40,469 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:40,470 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,470 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:53:40,470 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:53:40,470 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:53:40,470 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:40,471 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:53:40,516 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:53:40,516 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085220516 09:53:40,516 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:53:40,516 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:53:40,516 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085220516 09:53:40,517 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:53:40,517 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:53:50,517 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:53:50 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:50,517 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:53:50,517 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:53:50,517 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:53:50,517 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,517 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:50,518 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@729f0606, null) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085100420 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:53:50,518 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:53:50,518 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:53:50,518 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4a878437 >) 09:53:50,518 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@4a878437 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:53:50,538 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085230538 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:53:50,538 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,539 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:53:50,557 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@4faed7c6) 09:53:50,558 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:50,558 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@5e6cbb1c, null) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085100451 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:53:50,558 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:53:50,558 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:53:50,558 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@711baaa >) 09:53:50,559 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@711baaa >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:53:50,592 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085230592 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:53:50,592 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:53:50,603 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@1e32f615) 09:53:50,603 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:53:50,603 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:53:50,603 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:53:50,603 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:53:50,603 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:53:50,603 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:53:50,604 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085220516 1565085230604 false 09:53:50,604 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085220516 1565085230604 false 09:53:50,604 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:53:50,604 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:53:50,604 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:53:50,604 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:53:50,604 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:55:50,604 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:55:50,604 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:55:50,604 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:55:50 09:55:50,604 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:55:50,604 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,604 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:55:50,604 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,605 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,605 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@19313b4e, null) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085230538 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@76ebaf38, null) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:55:50,606 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085230592 09:55:50,607 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,607 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,607 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:55:50,607 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:55:50,607 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:55:50,607 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:55:50,607 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1dc OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1dc OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:55:50,607 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1dc OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1dc OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:55:50,608 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:55:50,609 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,609 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:55:50,609 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:55:50,609 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:55:50,609 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:55:50,609 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:55:50,655 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:55:50,655 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:55:50,655 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:55:50,655 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:55:50,655 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:56:00,655 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:56:00 09:56:00,655 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:56:00,655 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:56:00,655 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:56:00,655 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:56:00,656 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:56:00,656 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:56:00,656 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:56:00,656 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:56:00,656 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@4a4f061a, null) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085230538 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:56:00,656 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:56:00,657 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:56:00,657 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:56:00,657 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@61f6e204 >) 09:56:00,657 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@61f6e204 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:56:00,673 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085360673 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,673 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@12e1cec0) 09:56:00,681 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:56:00,681 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@73646898, null) 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085230592 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:56:00,681 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:56:00,682 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:56:00,682 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:56:00,682 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6372b910 >) 09:56:00,682 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@6372b910 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:56:00,695 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085360695 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:56:00,695 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:56:00,703 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@2f792de4) 09:56:00,703 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:56:00,703 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:56:00,703 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:56:00,703 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:56:00,703 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:56:00,703 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:56:00,703 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:56:00,703 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:56:00,703 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:56:00,703 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:56:00,704 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:56:00,704 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:56:00,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085220516 1565085360704 true 09:56:00,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085220516 1565085360704 true 09:56:00,704 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:56:00,704 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:56:00,704 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:56:00,704 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:56:00,704 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 09:58:00,704 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 09:58:00,704 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 09:58:00,704 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 09:58:00 09:58:00,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 09:58:00,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:58:00,704 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,705 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,705 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@2401a464, null) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085360673 09:58:00,706 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@c6f0f0c, null) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:58:00,707 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085360695 09:58:00,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 09:58:00,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:58:00,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:58:00,708 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:58:00,708 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,708 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1e8 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1e8 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1e8 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1e8 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:58:00,709 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:00,710 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,710 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:58:00,710 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 09:58:00,710 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:58:00,710 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:00,711 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:58:00,764 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:58:00,764 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085480764 09:58:00,764 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:58:00,764 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 09:58:00,764 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085480764 09:58:00,764 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 09:58:00,764 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 09:58:10,764 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 09:58:10 09:58:10,764 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 09:58:10,764 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 09:58:10,764 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 09:58:10,764 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:10,765 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:58:10,765 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 09:58:10,765 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 09:58:10,765 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:10,765 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@729cb158, null) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085360673 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:58:10,765 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:58:10,766 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:58:10,766 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:58:10,766 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@67fc768 >) 09:58:10,766 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@67fc768 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 09:58:10,781 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085490782 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,782 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 09:58:10,790 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@5ac0858d) 09:58:10,791 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:10,791 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@63f34b09, null) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085360695 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 09:58:10,791 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 09:58:10,791 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:58:10,791 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@699e127 >) 09:58:10,791 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@699e127 >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:58:10,806 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085490807 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 09:58:10,807 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 09:58:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@112b0bf4) 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 09:58:10,818 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 09:58:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085480764 1565085490818 false 09:58:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085480764 1565085490818 false 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 09:58:10,818 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 09:58:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 10:00:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 10:00:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 10:00:10,818 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 10:00:10 10:00:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 10:00:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:00:10,818 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,819 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 10:00:10,819 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,819 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:00:10,819 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,819 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:10,819 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@155ecdb1, null) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085490782 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,820 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@b98c4e, null) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085490807 10:00:10,821 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,821 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,821 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 10:00:10,821 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:00:10,821 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:00:10,821 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:00:10,821 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1f4 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1f4 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 10:00:10,821 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1f4 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:1f4 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,822 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:10,823 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,823 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:00:10,823 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 10:00:10,823 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:00:10,823 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:10,823 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 10:00:10,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 10:00:10,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 10:00:10,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 10:00:10,824 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 10:00:10,824 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:00:20,824 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 10:00:20 10:00:20,824 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 10:00:20,824 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:00:20,824 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:00:20,824 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:20,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:00:20,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:00:20,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 10:00:20,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:20,825 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@69c64c86, null) 10:00:20,825 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085490782 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 10:00:20,826 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 10:00:20,826 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:00:20,826 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2551760c >) 10:00:20,826 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@2551760c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 10:00:20,852 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085620852 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,852 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@6b7a8139) 10:00:20,860 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:20,860 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@3aca9986, null) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085490807 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 10:00:20,860 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 10:00:20,861 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 10:00:20,861 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:00:20,861 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5bc65a2f >) 10:00:20,861 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@5bc65a2f >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 10:00:20,883 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085620883 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:00:20,883 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 10:00:20,890 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@74424905) 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 10:00:20,891 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 10:00:20,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085480764 1565085620891 true 10:00:20,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085480764 1565085620891 true 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 10:00:20,891 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 10:00:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off 10:02:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== SCANNING 10:02:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread scanning 10:02:20,891 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery first pass at Tue, 6 Aug 2019 10:02:20 10:02:20,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule::periodicWorkFirstPass 10:02:20,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:02:20,891 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,892 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 10:02:20,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:02:20,892 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:20,893 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@62a80a72, null) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085620852 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@6790bf68, null) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085620883 10:02:20,894 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,894 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule first pass 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:02:20,894 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) processing /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction transactions 10:02:20,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:02:20,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:02:20,895 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:02:20,895 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - first pass 10:02:20,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: ) 10:02:20,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:200 OutputObjectState Type : null OutputObjectState Size : 28 OutputObjectState Buffer: , StateManager) 10:02:20,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:200 OutputObjectState Type : null OutputObjectState Size : 60 OutputObjectState Buffer: , StateManager/BasicAction) 10:02:20,895 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:200 OutputObjectState Type : null OutputObjectState Size : 112 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator) 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allTypes(OutputObjectState Uid : 0:ffff0a800171:35fe0b3b:5d493e2c:200 OutputObjectState Type : null OutputObjectState Size : 176 OutputObjectState Buffer: , StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:02:20,896 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , 2) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:20,897 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/SubordinateAtomicAction/JCA, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,897 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:02:20,897 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change IDLE->FIRST_PASS 10:02:20,897 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - first pass 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/AbstractRecord/XAResourceRecord, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:02:20,897 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:20,898 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 10:02:20,899 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 10:02:20,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085740899 10:02:20,899 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 10:02:20,899 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Found 0 xids in doubt 10:02:20,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids new recoveryXids org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085740899 10:02:20,899 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change FIRST_PASS->BETWEEN_PASSES 10:02:20,899 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) 10:02:30,899 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) Periodic recovery second pass at Tue, 6 Aug 2019 10:02:30 10:02:30,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) CommitMarkableResourceRecordRecoveryModule second pass 10:02:30,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState() 10:02:30,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 0 InputObjectState Buffer: , -1) 10:02:30,899 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:30,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:02:30,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) found transaction 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.allObjUids(/StateManager/BasicAction/TwoPhaseCoordinator/AtomicActionConnectable, InputObjectState Uid : 0:0:0:0:0 InputObjectState Type : null InputObjectState Size : 104 InputObjectState Buffer: , -1) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState() 10:02:30,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.CommitMarkableResourceRecordRecoveryModule@2cf61880' second pass processed 10:02:30,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) AtomicActionRecoveryModule second pass 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:30,900 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED in flight is false 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:13) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_RDLCK, false) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, java.io.FileInputStream@74cccb0, null) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085620852 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 10:02:30,900 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 10:02:30,901 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 10:02:30,901 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:13 ActionStatus is ActionStatus.COMMITTED 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:02:30,901 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@654558ce >) 10:02:30,901 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@654558ce >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:21 10:02:30,919 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:13, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:20, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at http-remoting://tx-server-0.tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:13 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:21 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:13) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:21) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:13 birth-date 1565085750919 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:13, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,919 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, FileLock.F_WRLCK, true) 10:02:30,926 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_13, null, java.io.FileOutputStream@316c5f31) 10:02:30,926 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:13 ) finished 10:02:30,926 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:30,926 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:30,926 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,926 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:30,927 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) transaction type is /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction uid is 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED in flight is false 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::BasicAction(0:ffff0a800171:35fe0b3b:5d493e2c:42) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::activate() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.read_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.read_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_SHADOW) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 10) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.currentState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) - returning StateStatus.OS_COMMITTED 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_RDLCK, false) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) InputObjectState::InputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, java.io.FileInputStream@79b34279, null) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::restore_state () 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.unpackHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085620883 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 171 record 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager::StateManager( 0:0:0:0:0 ) 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) AbstractRecord::AbstractRecord () - crash recovery constructor 10:02:30,927 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.restore_state - XAResource de-serialized 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Unpacked a 463 record 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) HeuristicList - Unpacked heuristic list size of 0 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action status of ActionStatus.COMMITTED 7 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored action type Top-level 0 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Restored heuristic decision of TwoPhaseOutcome.FINISH_OK 7 10:02:30,927 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2 recovering 0:ffff0a800171:35fe0b3b:5d493e2c:42 ActionStatus is ActionStatus.COMMITTED 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::phase2Commit() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:02:30,927 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit (XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@21e9660c >) 10:02:30,927 TRACE [com.arjuna.ats.jta] (Periodic Recovery) XAResourceRecord.topLevelCommit for XAResourceRecord < resource:Subordinate XAResource at remote+http://tx-server:8080, txid:< formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name >, heuristic: TwoPhaseOutcome.FINISH_OK com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord@21e9660c >, record id=0:ffff0a800171:35fe0b3b:5d493e2c:4c 10:02:30,939 WARN [com.arjuna.ats.jta] (Periodic Recovery) ARJUNA016036: commit on < formatId=131077, gtrid_length=39, bqual_length=36, tx_uid=0:ffff0a800171:35fe0b3b:5d493e2c:42, node_name=tx-client-0, branch_uid=0:ffff0a800171:35fe0b3b:5d493e2c:4b, subordinatenodename=null, eis_name=unknown eis name > (Subordinate XAResource at remote+http://tx-server:8080) failed with exception $XAException.XA_RETRY: javax.transaction.xa.XAException: WFTXN0029: The peer threw an XA exception at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.TransactionClientChannel.commit(TransactionClientChannel.java:400) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.provider.remoting.RemotingRemoteTransactionPeer$1.commit(RemotingRemoteTransactionPeer.java:193) at org.wildfly.transaction.client@1.1.6.Final//org.wildfly.transaction.client.SubordinateXAResource.commit(SubordinateXAResource.java:167) at org.jboss.jts//com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord.topLevelCommit(XAResourceRecord.java:473) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2892) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.doCommit(BasicAction.java:2808) at org.jboss.jts//com.arjuna.ats.arjuna.coordinator.BasicAction.phase2Commit(BasicAction.java:1873) at org.jboss.jts//com.arjuna.ats.arjuna.recovery.RecoverAtomicAction.replayPhase2(RecoverAtomicAction.java:71) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.doRecoverTransaction(AtomicActionRecoveryModule.java:152) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.processTransactionsStatus(AtomicActionRecoveryModule.java:253) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule.periodicWorkSecondPass(AtomicActionRecoveryModule.java:109) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.doWorkInternal(PeriodicRecovery.java:816) at org.jboss.jts//com.arjuna.ats.internal.arjuna.recovery.PeriodicRecovery.run(PeriodicRecovery.java:382) Caused by: Remote exception javax.transaction.xa.XAException Public fields: errorCode=4 at org.wildfly.transaction.client.provider.jboss.JBossLocalTransactionProvider$Entry.commit(JBossLocalTransactionProvider.java:505) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.lambda$handleXaTxnCommit$7(TransactionServerChannel.java:625) at org.wildfly.security.auth.server.SecurityIdentity.runAsConsumer(SecurityIdentity.java:361) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel.handleXaTxnCommit(TransactionServerChannel.java:616) at org.wildfly.transaction.client.provider.remoting.TransactionServerChannel$ReceiverImpl.handleMessage(TransactionServerChannel.java:125) at org.jboss.remoting3.remote.RemoteConnectionChannel.lambda$handleMessageData$3(RemoteConnectionChannel.java:430) at org.jboss.remoting3.EndpointImpl$TrackingExecutor.lambda$execute$0(EndpointImpl.java:975) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:834) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction.doCommit for 0:ffff0a800171:35fe0b3b:5d493e2c:42 received TwoPhaseOutcome.FINISH_ERROR from class com.arjuna.ats.internal.jta.resources.arjunacore.XAResourceRecord 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecordList::insert(RecordList: empty) : appending /StateManager/AbstractRecord/XAResourceRecord for 0:ffff0a800171:35fe0b3b:5d493e2c:4c 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::doCommit() result for action-id (0:ffff0a800171:35fe0b3b:5d493e2c:42) on record id: (0:ffff0a800171:35fe0b3b:5d493e2c:4c) is (TwoPhaseOutcome.FINISH_ERROR) node id: (tx-client-0) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::updateState() for action-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) OutputObjectState::OutputObjectState(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state () 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) StateManager.packHeader for object-id 0:ffff0a800171:35fe0b3b:5d493e2c:42 birth-date 1565085750940 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) BasicAction::save_state - next record to pack is a 171 record /StateManager/AbstractRecord/XAResourceRecord should save it? = true 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a 171 record 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing a NONE_RECORD 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) Packing action status of ActionStatus.COMMITTED 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.write_committed(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.write_state(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) ShadowingStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, StateType.OS_ORIGINAL) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.genPathName(0:ffff0a800171:35fe0b3b:5d493e2c:42, /StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction, 11) 10:02:30,940 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.openAndLock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, FileLock.F_WRLCK, true) 10:02:30,949 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) FileSystemStore.closeAndUnlock(/opt/eap/standalone/data/tx-object-store/ShadowNoFileLockStore/defaultStore/StateManager/BasicAction/TwoPhaseCoordinator/AtomicAction/0_ffff0a800171_35fe0b3b_5d493e2c_42, null, java.io.FileOutputStream@61e473d) 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) RecoverAtomicAction.replayPhase2( 0:ffff0a800171:35fe0b3b:5d493e2c:42 ) finished 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.arjuna.recovery.AtomicActionRecoveryModule@6029e3aa' second pass processed 10:02:30,950 DEBUG [com.arjuna.ats.txoj] (Periodic Recovery) TORecoveryModule - second pass 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.txoj.recovery.TORecoveryModule@5a73373d' second pass processed 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.SubordinateAtomicActionRecoveryModule@485ddade' second pass processed 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change BETWEEN_PASSES->SECOND_PASS 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule - second pass 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.transactionInitiatedRecovery completed 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) xarecovery second pass of org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Have 0 Xids to recover on this pass. 10:02:30,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.activemq.artemis.wildfly.integration.WildFlyActiveMQXAResourceWrapper@354c855e 1565085740899 1565085750950 false 10:02:30,950 TRACE [com.arjuna.ats.arjuna] (Periodic Recovery) RecoveryXids isStale Check org.jboss.as.quickstarts.xa.resources.MockXAResource@418cce47 1565085740899 1565085750950 false 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) Local XARecoveryModule.resourceInitiatedRecovery completed 10:02:30,950 DEBUG [com.arjuna.ats.jta] (Periodic Recovery) XARecoveryModule state change SECOND_PASS->IDLE 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: recovery module 'com.arjuna.ats.internal.jta.recovery.arjunacore.XARecoveryModule@5473f94d' second pass processed 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread Status <== INACTIVE 10:02:30,950 DEBUG [com.arjuna.ats.arjuna] (Periodic Recovery) PeriodicRecovery: background thread backing off