Starting Hawkular Services ------------------------------------ Generating the self-signed certificate Certificate fingerprint (SHA1): 1C:FD:74:00:3A:75:DC:0E:D4:6F:EB:7E:EB:0E:15:15:4D:48:E5:01 ------------------------------------ Certificate stored in file Certificate was added to keystore ========================================================================= JBoss Bootstrap Environment JBOSS_HOME: /opt/jboss/wildfly JAVA: /usr/lib/jvm/java/bin/java JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman,org.jboss.logmanager -Djava.awt.headless=true -Djava.util.logging.manager=org.jboss.logmanager.LogManager -javaagent:/opt/jboss/wildfly/bin/hawkular-javaagent.jar=config=/opt/jboss/wildfly/standalone/configuration/hawkular-javaagent-config.yaml,delay=10 ========================================================================= 18:36:17,968 INFO [org.jboss.modules] (main) JBoss Modules version 1.5.2.Final 18:36:18,487 INFO [org.jboss.msc] (main) JBoss MSC version 1.2.6.Final 18:36:18,694 INFO [org.jboss.as] (MSC service thread 1-6) WFLYSRV0049: Hawkular 0.40.0.Final-SNAPSHOT (WildFly Core 2.2.0.Final) starting 18:36:20,661 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 26) HAWKBUS130015: Initializing Nest subsystem 18:36:21,092 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130008: [7] deployments found 18:36:21,099 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 1. Deploying [hawkular-command-gateway-war.war] 18:36:21,100 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 2. Deploying [hawkular-inventory-service.war] 18:36:21,101 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 3. Deploying [hawkular-inventory-ui.war] 18:36:21,102 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 4. Deploying [hawkular-metrics.ear] 18:36:21,103 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 5. Deploying [hawkular-rest-api.war] 18:36:21,104 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 6. Deploying [hawkular-status.war] 18:36:21,105 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 31) HAWKBUS130009: 7. Deploying [hawkular-wildfly-agent-download.war] 18:36:21,152 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/d1/fdfd2b26daa3e194b9f46375ec0c6cfc56eb12/content 18:36:21,629 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/4f/996528011f0af0b066ffcb9e6dc66a3b242409/content 18:36:21,655 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/ef/bb056a636b672f6cfd7e4e989ef67feafbe597/content 18:36:22,274 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/c3/19b275b837412f4a0d0b662033d371bd9f25b2/content 18:36:22,314 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/86/fd323a0882dc2cdf6cd7c473e73d09011ae7b3/content 18:36:22,318 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/23/8134b2664b1cbc51af59863af77a664973c1ca/content 18:36:22,321 INFO [org.jboss.as.repository] (ServerService Thread Pool -- 31) WFLYDR0001: Content added at location /var/opt/hawkular/data/content/d4/ca9bcff704d29720d3e3983205c960df723578/content 18:36:22,371 INFO [org.jboss.as.server] (Controller Boot Thread) WFLYSRV0040: Creating http management service using secure-socket-binding (management-https) 18:36:22,423 INFO [org.xnio] (MSC service thread 1-7) XNIO version 3.4.0.Final 18:36:22,441 INFO [org.xnio.nio] (MSC service thread 1-7) XNIO NIO Implementation Version 3.4.0.Final 18:36:22,516 INFO [org.hawkular.nest.extension.log] (ServerService Thread Pool -- 61) HAWKBUS130014: Nest is enabled and will be deployed 18:36:22,518 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 56) WFLYTX0013: Node identifier property is set to the default value. Please make sure it is unique. 18:36:22,523 INFO [org.jboss.as.naming] (ServerService Thread Pool -- 48) WFLYNAM0001: Activating Naming Subsystem 18:36:22,535 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 40) WFLYCLINF0001: Activating Infinispan subsystem. 18:36:22,561 INFO [org.jboss.as.jsf] (ServerService Thread Pool -- 46) WFLYJSF0007: Activated the following JSF Implementations: [main] 18:36:22,562 INFO [org.jboss.as.security] (ServerService Thread Pool -- 55) WFLYSEC0002: Activating Security Subsystem 18:36:22,567 INFO [org.jboss.as.webservices] (ServerService Thread Pool -- 58) WFLYWS0002: Activating WebServices Extension 18:36:22,582 INFO [org.wildfly.extension.io] (ServerService Thread Pool -- 39) WFLYIO001: Worker 'default' has auto-configured to 48 core threads with 384 task threads based on your 24 available processors 18:36:22,599 INFO [org.hawkular.nest.extension.log] (MSC service thread 1-6) HAWKBUS130000: Nest service starting 18:36:22,601 INFO [org.hawkular.nest.extension.log] (MSC service thread 1-6) HAWKBUS130004: Nest name=[hawkular-services-2-7z0xc] 18:36:22,603 INFO [org.hawkular.nest.extension.log] (MSC service thread 1-6) HAWKBUS130001: Nest service started 18:36:22,624 INFO [org.jboss.as.security] (MSC service thread 1-1) WFLYSEC0001: Current PicketBox version=4.9.6.Final 18:36:22,699 INFO [org.jboss.as.connector] (MSC service thread 1-8) WFLYJCA0009: Starting JCA Subsystem (WildFly/IronJacamar 1.3.4.Final) 18:36:22,782 INFO [org.jboss.remoting] (MSC service thread 1-7) JBoss Remoting version 4.0.21.Final 18:36:22,815 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0003: Undertow 1.4.0.Final starting 18:36:22,840 INFO [org.jboss.as.naming] (MSC service thread 1-8) WFLYNAM0003: Starting Naming Service 18:36:22,846 INFO [org.jboss.as.connector.subsystems.datasources] (ServerService Thread Pool -- 35) WFLYJCA0004: Deploying JDBC-compliant driver class org.h2.Driver (version 1.3) 18:36:22,852 INFO [org.jboss.as.connector.deployers.jdbc] (MSC service thread 1-1) WFLYJCA0018: Started Driver service with driver-name = h2 18:36:22,856 INFO [org.jboss.as.mail.extension] (MSC service thread 1-3) WFLYMAIL0001: Bound mail session [java:jboss/mail/Default] 18:36:23,197 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 57) WFLYUT0014: Creating file handler for path '/opt/jboss/wildfly/hawkular-welcome-content' with options [directory-listing: 'false', follow-symlink: 'false', case-sensitive: 'true', safe-symlink-paths: '[]'] 18:36:23,238 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0012: Started server default-server. 18:36:23,248 INFO [org.wildfly.extension.undertow] (MSC service thread 1-3) WFLYUT0018: Host default-host starting 18:36:23,427 INFO [org.jboss.as.ejb3] (MSC service thread 1-3) WFLYEJB0482: Strict pool mdb-strict-max-pool is using a max instance size of 96 (per class), which is derived from the number of CPUs on this host. 18:36:23,430 INFO [org.jboss.as.ejb3] (MSC service thread 1-7) WFLYEJB0481: Strict pool slsb-strict-max-pool is using a max instance size of 384 (per class), which is derived from thread worker pool sizing. 18:36:23,565 INFO [org.wildfly.extension.undertow] (MSC service thread 1-8) WFLYUT0006: Undertow HTTP listener default listening on 0.0.0.0:8080 18:36:23,744 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-6) WFLYJCA0001: Bound data source [java:jboss/datasources/ExampleDS] 18:36:23,978 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-2) WFLYMSGAMQ0075: AIO wasn't located on this platform, it will fall back to using pure Java NIO. Your platform is Linux, install LibAIO to enable the AIO journal and achieve optimal performance. 18:36:24,075 WARN [org.jboss.as.domain.management.security] (MSC service thread 1-2) WFLYDM0111: Keystore /opt/jboss/wildfly/standalone/configuration/application.keystore not found, it will be auto generated on first use with a self signed certificate for host localhost 18:36:24,221 INFO [org.jboss.as.server.deployment.scanner] (MSC service thread 1-4) WFLYDS0013: Started FileSystemDeploymentService for directory /opt/jboss/wildfly/standalone/deployments 18:36:24,240 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0027: Starting deployment of "hawkular-command-gateway-war.war" (runtime-name: "hawkular-command-gateway-war.war") 18:36:24,240 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0027: Starting deployment of "hawkular-wildfly-agent-download.war" (runtime-name: "hawkular-wildfly-agent-download.war") 18:36:24,246 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0027: Starting deployment of "hawkular-inventory-service.war" (runtime-name: "hawkular-inventory-service.war") 18:36:24,246 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0027: Starting deployment of "hawkular-rest-api.war" (runtime-name: "hawkular-rest-api.war") 18:36:24,247 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0027: Starting deployment of "hawkular-status.war" (runtime-name: "hawkular-status.war") 18:36:24,247 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0027: Starting deployment of "hawkular-metrics.ear" (runtime-name: "hawkular-metrics.ear") 18:36:24,248 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0027: Starting deployment of "hawkular-inventory-ui.war" (runtime-name: "hawkular-inventory-ui.war") 18:36:24,335 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221000: live Message Broker is starting with configuration Broker Configuration (clustered=false,journalDirectory=/var/opt/hawkular/data/activemq/journal,bindingsDirectory=/var/opt/hawkular/data/activemq/bindings,largeMessagesDirectory=/var/opt/hawkular/data/activemq/largemessages,pagingDirectory=/var/opt/hawkular/data/activemq/paging) 18:36:24,351 INFO [org.infinispan.factories.GlobalComponentRegistry] (MSC service thread 1-8) ISPN000128: Infinispan version: Infinispan 'Chakra' 8.2.4.Final 18:36:24,447 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221013: Using NIO Journal 18:36:24,627 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 77) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. 18:36:24,628 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 71) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. 18:36:24,631 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 77) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. 18:36:24,634 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 71) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. 18:36:24,903 INFO [org.wildfly.extension.undertow] (MSC service thread 1-7) WFLYUT0006: Undertow HTTPS listener https listening on 0.0.0.0:8443 18:36:25,108 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221043: Protocol module found: [artemis-server]. Adding protocol support for: CORE 18:36:25,120 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221043: Protocol module found: [artemis-amqp-protocol]. Adding protocol support for: AMQP 18:36:25,128 INFO [org.jboss.ws.common.management] (MSC service thread 1-7) JBWS022052: Starting JBossWS 5.1.5.Final (Apache CXF 3.1.6) 18:36:25,153 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221043: Protocol module found: [artemis-hornetq-protocol]. Adding protocol support for: HORNETQ 18:36:25,158 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221043: Protocol module found: [artemis-stomp-protocol]. Adding protocol support for: STOMP 18:36:25,523 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-8) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor acceptor 18:36:25,524 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-4) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor acceptor 18:36:25,531 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-4) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor-throughput acceptor 18:36:25,533 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-8) WFLYMSGAMQ0016: Registered HTTP upgrade for activemq-remoting protocol handled by http-acceptor-throughput acceptor 18:36:25,564 WARN [org.jboss.as.dependency.private] (MSC service thread 1-7) WFLYSRV0018: Deployment "deployment.hawkular-command-gateway-war.war" is using a private module ("org.wildfly.extension.messaging-activemq:main") which may be changed or removed in future versions without notice. 18:36:25,660 INFO [org.jboss.weld.deployer] (MSC service thread 1-8) WFLYWELD0003: Processing weld deployment hawkular-command-gateway-war.war 18:36:25,682 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221007: Server is now live 18:36:25,683 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221001: Apache ActiveMQ Artemis Message Broker version 1.1.0.wildfly-017 [nodeID=91cf1c8a-ba7c-11e7-8f26-c9b2ac1e0bc7] 18:36:25,689 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 62) AMQ221003: trying to deploy queue jms.queue.DLQ 18:36:25,868 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 64) AMQ221003: trying to deploy queue jms.queue.ExpiryQueue 18:36:25,885 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 73) AMQ221003: trying to deploy queue jms.topic.HawkularTopic 18:36:25,932 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-7) WFLYJCA0007: Registered connection factory java:/JmsXA 18:36:25,932 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 69) AMQ221003: trying to deploy queue jms.topic.HawkularAlertData 18:36:25,937 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 68) AMQ221003: trying to deploy queue jms.queue.hawkular/metrics/gauges/new 18:36:25,942 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 78) AMQ221003: trying to deploy queue jms.queue.HawkularAlertsPluginsQueue 18:36:25,947 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 65) AMQ221003: trying to deploy queue jms.queue.hawkular/metrics/availability/new 18:36:25,953 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 74) AMQ221003: trying to deploy queue jms.queue.HawkularAlertsActionsResponseQueue 18:36:26,006 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 70) WFLYMSGAMQ0002: Bound messaging object to jndi name java:/ConnectionFactory 18:36:26,007 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 70) WFLYMSGAMQ0002: Bound messaging object to jndi name java:/HawkularBusConnectionFactory 18:36:26,008 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 76) AMQ221003: trying to deploy queue jms.topic.HawkularQueue 18:36:26,012 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 79) AMQ221003: trying to deploy queue jms.queue.hawkular/metrics/counters/new 18:36:26,015 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 72) AMQ221003: trying to deploy queue jms.topic.HawkularAlertsActionsTopic 18:36:26,020 INFO [org.apache.activemq.artemis.core.server] (ServerService Thread Pool -- 77) AMQ221003: trying to deploy queue jms.topic.HawkularCommandEvent 18:36:26,029 INFO [org.apache.activemq.artemis.jms.server] (ServerService Thread Pool -- 71) AMQ121005: Invalid "host" value "0.0.0.0" detected for "http-connector" connector. Switching to "hawkular-services-2-7z0xc". If this new address is incorrect please manually configure the connector to use the proper one. 18:36:26,030 INFO [org.wildfly.extension.messaging-activemq] (ServerService Thread Pool -- 71) WFLYMSGAMQ0002: Bound messaging object to jndi name java:jboss/exported/jms/RemoteConnectionFactory 18:36:26,120 INFO [org.apache.activemq.artemis.ra] (MSC service thread 1-7) Resource adaptor started 18:36:26,121 INFO [org.jboss.as.connector.services.resourceadapters.ResourceAdapterActivatorService$ResourceAdapterActivator] (MSC service thread 1-7) IJ020002: Deployed: file://RaActivatoractivemq-ra 18:36:26,123 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts.war") 18:36:26,123 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts-action-webhook.war") 18:36:26,123 INFO [org.jboss.as.server.deployment] (MSC service thread 1-5) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-elasticsearch-alerter.war") 18:36:26,124 INFO [org.jboss.as.server.deployment] (MSC service thread 1-3) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-metrics-alerter.war") 18:36:26,124 INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts-action-elasticsearch.war") 18:36:26,125 INFO [org.jboss.as.server.deployment] (MSC service thread 1-5) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-alerts-action-email.war") 18:36:26,124 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0207: Starting subdeployment (runtime-name: "hawkular-metrics.war") 18:36:26,174 INFO [org.hibernate.validator.internal.util.Version] (MSC service thread 1-8) HV000001: Hibernate Validator 5.2.4.Final 18:36:26,567 INFO [org.jboss.as.connector.deployment] (MSC service thread 1-7) WFLYJCA0002: Bound JCA ConnectionFactory [java:/JmsXA] 18:36:26,575 INFO [org.wildfly.extension.messaging-activemq] (MSC service thread 1-7) WFLYMSGAMQ0002: Bound messaging object to jndi name java:jboss/DefaultJMSConnectionFactory 18:36:26,638 INFO [org.jboss.weld.deployer] (MSC service thread 1-7) WFLYWELD0003: Processing weld deployment hawkular-wildfly-agent-download.war 18:36:26,976 INFO [org.jboss.weld.deployer] (MSC service thread 1-5) WFLYWELD0003: Processing weld deployment hawkular-status.war 18:36:27,157 INFO [org.jboss.weld.Version] (MSC service thread 1-7) WELD-000900: 2.3.5 (Final) 18:36:29,995 INFO [org.hawkular.agent.javaagent.JavaAgentEngine] (Hawkular Java Agent Start Thread) HAWKJAVAAGENT010000: Loaded configuration file [/opt/jboss/wildfly/standalone/configuration/hawkular-javaagent-config.yaml] 18:36:30,060 INFO [org.hawkular.agent.monitor.service.AgentCoreEngine] (Hawkular Java Agent Start Thread) HAWKAGENT010084: Hawkular Agent: Javaagent version 2.0.0.Alpha1-SRC-revision-5a4c645c9974e6978f5fa73d2f701364998bd780 18:36:30,061 INFO [org.hawkular.agent.monitor.service.AgentCoreEngine] (Hawkular Java Agent Start Thread) HAWKAGENT010000: Starting Hawkular Agent 18:36:30,521 WARN [org.jboss.as.server.deployment] (MSC service thread 1-6) WFLYSRV0001: /content/hawkular-metrics.ear/hawkular-metrics.war/WEB-INF/jboss-deployment-structure.xml in subdeployment ignored. jboss-deployment-structure.xml is only parsed for top level deployments. 18:36:30,710 INFO [io.undertow.websockets.jsr] (ServerService Thread Pool -- 71) UT026003: Adding annotated server endpoint class org.hawkular.cmdgw.command.ws.server.UIClientWebSocket for path /ui/ws 18:36:30,747 INFO [io.undertow.websockets.jsr] (ServerService Thread Pool -- 71) UT026003: Adding annotated server endpoint class org.hawkular.cmdgw.command.ws.server.FeedWebSocket for path /feed/{feedId} 18:36:30,933 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 71) WFLYUT0021: Registered web context: /hawkular/command-gateway 18:36:30,939 INFO [io.undertow.servlet] (ServerService Thread Pool -- 67) org.hawkular.component.wildflymonitor.WildFlyAgentServlet: Starting the WildFly Agent download servlet 18:36:30,940 INFO [io.undertow.servlet] (ServerService Thread Pool -- 67) org.hawkular.component.wildflymonitor.WildFlyAgentServlet: Latest Agent Module File: /opt/jboss/wildfly/standalone/configuration/hawkular-wildfly-agent-wf-extension.zip 18:36:30,940 INFO [io.undertow.servlet] (ServerService Thread Pool -- 67) org.hawkular.component.wildflymonitor.WildFlyAgentServlet: EAP6 Agent Module File: /opt/jboss/wildfly/standalone/configuration/hawkular-wildfly-agent-wf-extension-eap6.zip 18:36:30,941 INFO [io.undertow.servlet] (ServerService Thread Pool -- 67) org.hawkular.component.wildflymonitor.WildFlyAgentServlet: Agent Installer File: /opt/jboss/wildfly/standalone/configuration/hawkular-wildfly-agent-installer.jar 18:36:30,941 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 67) WFLYUT0021: Registered web context: /hawkular/wildfly-agent 18:36:31,054 INFO [org.jboss.weld.deployer] (MSC service thread 1-7) WFLYWELD0003: Processing weld deployment hawkular-inventory-service.war 18:36:31,068 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-7) WFLYEJB0473: JNDI bindings for session bean named 'InventoryConfig' in deployment unit 'deployment "hawkular-inventory-service.war"' are as follows: java:global/hawkular-inventory-service/InventoryConfig!org.hawkular.inventory.service.InventoryConfig java:app/hawkular-inventory-service/InventoryConfig!org.hawkular.inventory.service.InventoryConfig java:module/InventoryConfig!org.hawkular.inventory.service.InventoryConfig java:global/hawkular-inventory-service/InventoryConfig java:app/hawkular-inventory-service/InventoryConfig java:module/InventoryConfig 18:36:31,070 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-7) WFLYEJB0473: JNDI bindings for session bean named 'InventoryServiceIspn' in deployment unit 'deployment "hawkular-inventory-service.war"' are as follows: java:global/hawkular-inventory-service/InventoryServiceIspn!org.hawkular.inventory.api.InventoryService java:app/hawkular-inventory-service/InventoryServiceIspn!org.hawkular.inventory.api.InventoryService java:module/InventoryServiceIspn!org.hawkular.inventory.api.InventoryService java:global/hawkular-inventory-service/InventoryServiceIspn java:app/hawkular-inventory-service/InventoryServiceIspn java:module/InventoryServiceIspn 18:36:31,070 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-7) WFLYEJB0473: JNDI bindings for session bean named 'InventoryStats' in deployment unit 'deployment "hawkular-inventory-service.war"' are as follows: java:global/hawkular-inventory-service/InventoryStats!org.hawkular.inventory.service.InventoryStatsMBean java:app/hawkular-inventory-service/InventoryStats!org.hawkular.inventory.service.InventoryStatsMBean java:module/InventoryStats!org.hawkular.inventory.service.InventoryStatsMBean java:global/hawkular-inventory-service/InventoryStats java:app/hawkular-inventory-service/InventoryStats java:module/InventoryStats 18:36:31,321 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 67) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. 18:36:31,323 INFO [org.infinispan.configuration.cache.EvictionConfigurationBuilder] (ServerService Thread Pool -- 67) ISPN000152: Passivation configured without an eviction policy being selected. Only manually evicted entities will be passivated. 18:36:31,660 WARN [org.jboss.as.dependency.private] (MSC service thread 1-5) WFLYSRV0018: Deployment "deployment.hawkular-metrics.ear" is using a private module ("org.infinispan:main") which may be changed or removed in future versions without notice. 18:36:31,864 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 73) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.services.rest.HawkularServicesStatusApp$Proxy$_$$_WeldClientProxy 18:36:31,883 WARN [org.jboss.as.dependency.private] (MSC service thread 1-5) WFLYSRV0018: Deployment "deployment.hawkular-metrics.ear.hawkular-alerts.war" is using a private module ("com.google.guava:main") which may be changed or removed in future versions without notice. 18:36:31,918 INFO [org.jboss.weld.deployer] (MSC service thread 1-3) WFLYWELD0003: Processing weld deployment hawkular-metrics.ear 18:36:31,991 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 73) WFLYUT0021: Registered web context: /hawkular 18:36:32,010 INFO [org.jboss.weld.deployer] (MSC service thread 1-4) WFLYWELD0003: Processing weld deployment hawkular-metrics.war 18:36:32,010 INFO [org.jboss.weld.deployer] (MSC service thread 1-2) WFLYWELD0003: Processing weld deployment hawkular-alerts-action-email.war 18:36:32,010 INFO [org.jboss.weld.deployer] (MSC service thread 1-7) WFLYWELD0003: Processing weld deployment hawkular-alerts-action-webhook.war 18:36:32,010 INFO [org.jboss.weld.deployer] (MSC service thread 1-6) WFLYWELD0003: Processing weld deployment hawkular-elasticsearch-alerter.war 18:36:32,010 INFO [org.jboss.weld.deployer] (MSC service thread 1-5) WFLYWELD0003: Processing weld deployment hawkular-alerts-action-elasticsearch.war 18:36:32,014 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2) WFLYEJB0473: JNDI bindings for session bean named 'StandaloneActionPluginRegister' in deployment unit 'subdeployment "hawkular-alerts-action-email.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts-action-email/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:app/hawkular-alerts-action-email/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:module/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:global/hawkular-metrics/hawkular-alerts-action-email/StandaloneActionPluginRegister java:app/hawkular-alerts-action-email/StandaloneActionPluginRegister java:module/StandaloneActionPluginRegister 18:36:32,015 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-7) WFLYEJB0473: JNDI bindings for session bean named 'StandaloneActionPluginRegister' in deployment unit 'subdeployment "hawkular-alerts-action-webhook.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts-action-webhook/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:app/hawkular-alerts-action-webhook/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:module/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:global/hawkular-metrics/hawkular-alerts-action-webhook/StandaloneActionPluginRegister java:app/hawkular-alerts-action-webhook/StandaloneActionPluginRegister java:module/StandaloneActionPluginRegister 18:36:32,015 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-6) WFLYEJB0473: JNDI bindings for session bean named 'ElasticsearchAlerter' in deployment unit 'subdeployment "hawkular-elasticsearch-alerter.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-elasticsearch-alerter/ElasticsearchAlerter!org.hawkular.alerter.elasticsearch.ElasticsearchAlerter java:app/hawkular-elasticsearch-alerter/ElasticsearchAlerter!org.hawkular.alerter.elasticsearch.ElasticsearchAlerter java:module/ElasticsearchAlerter!org.hawkular.alerter.elasticsearch.ElasticsearchAlerter java:global/hawkular-metrics/hawkular-elasticsearch-alerter/ElasticsearchAlerter java:app/hawkular-elasticsearch-alerter/ElasticsearchAlerter java:module/ElasticsearchAlerter 18:36:32,015 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-5) WFLYEJB0473: JNDI bindings for session bean named 'StandaloneActionPluginRegister' in deployment unit 'subdeployment "hawkular-alerts-action-elasticsearch.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:app/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:module/StandaloneActionPluginRegister!org.hawkular.alerts.actions.standalone.StandaloneActionPluginRegister java:global/hawkular-metrics/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister java:app/hawkular-alerts-action-elasticsearch/StandaloneActionPluginRegister java:module/StandaloneActionPluginRegister 18:36:32,021 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-4) WFLYEJB0473: JNDI bindings for session bean named 'MetricsJNDIPublisher' in deployment unit 'subdeployment "hawkular-metrics.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-metrics/MetricsJNDIPublisher!org.hawkular.metrics.component.MetricsJNDIPublisher java:app/hawkular-metrics/MetricsJNDIPublisher!org.hawkular.metrics.component.MetricsJNDIPublisher java:module/MetricsJNDIPublisher!org.hawkular.metrics.component.MetricsJNDIPublisher java:global/hawkular-metrics/hawkular-metrics/MetricsJNDIPublisher java:app/hawkular-metrics/MetricsJNDIPublisher java:module/MetricsJNDIPublisher 18:36:32,033 INFO [org.jboss.weld.deployer] (MSC service thread 1-1) WFLYWELD0003: Processing weld deployment hawkular-metrics-alerter.war 18:36:32,036 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'ConditionManager' in deployment unit 'subdeployment "hawkular-metrics-alerter.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-metrics-alerter/ConditionManager!org.hawkular.metrics.alerter.ConditionManager java:app/hawkular-metrics-alerter/ConditionManager!org.hawkular.metrics.alerter.ConditionManager java:module/ConditionManager!org.hawkular.metrics.alerter.ConditionManager java:global/hawkular-metrics/hawkular-metrics-alerter/ConditionManager java:app/hawkular-metrics-alerter/ConditionManager java:module/ConditionManager 18:36:32,037 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-1) WFLYEJB0473: JNDI bindings for session bean named 'GroupTriggerManager' in deployment unit 'subdeployment "hawkular-metrics-alerter.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-metrics-alerter/GroupTriggerManager!org.hawkular.metrics.alerter.groups.GroupTriggerManager java:app/hawkular-metrics-alerter/GroupTriggerManager!org.hawkular.metrics.alerter.groups.GroupTriggerManager java:module/GroupTriggerManager!org.hawkular.metrics.alerter.groups.GroupTriggerManager java:global/hawkular-metrics/hawkular-metrics-alerter/GroupTriggerManager java:app/hawkular-metrics-alerter/GroupTriggerManager java:module/GroupTriggerManager 18:36:32,180 INFO [org.jboss.weld.deployer] (MSC service thread 1-8) WFLYWELD0003: Processing weld deployment hawkular-alerts.war 18:36:32,190 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'DroolsRulesEngineImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/DroolsRulesEngineImpl!org.hawkular.alerts.engine.service.RulesEngine java:app/hawkular-alerts/DroolsRulesEngineImpl!org.hawkular.alerts.engine.service.RulesEngine java:module/DroolsRulesEngineImpl!org.hawkular.alerts.engine.service.RulesEngine java:global/hawkular-metrics/hawkular-alerts/DroolsRulesEngineImpl java:app/hawkular-alerts/DroolsRulesEngineImpl java:module/DroolsRulesEngineImpl 18:36:32,191 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'CepEngineImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/CepEngineImpl!org.hawkular.alerts.extensions.CepEngine java:app/hawkular-alerts/CepEngineImpl!org.hawkular.alerts.extensions.CepEngine java:module/CepEngineImpl!org.hawkular.alerts.extensions.CepEngine java:global/hawkular-metrics/hawkular-alerts/CepEngineImpl java:app/hawkular-alerts/CepEngineImpl java:module/CepEngineImpl 18:36:32,191 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'PropertiesServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/PropertiesServiceImpl!org.hawkular.alerts.api.services.PropertiesService java:app/hawkular-alerts/PropertiesServiceImpl!org.hawkular.alerts.api.services.PropertiesService java:module/PropertiesServiceImpl!org.hawkular.alerts.api.services.PropertiesService java:global/hawkular-metrics/hawkular-alerts/PropertiesServiceImpl java:app/hawkular-alerts/PropertiesServiceImpl java:module/PropertiesServiceImpl 18:36:32,191 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'IncomingDataManagerImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/IncomingDataManagerImpl!org.hawkular.alerts.engine.service.IncomingDataManager java:app/hawkular-alerts/IncomingDataManagerImpl!org.hawkular.alerts.engine.service.IncomingDataManager java:module/IncomingDataManagerImpl!org.hawkular.alerts.engine.service.IncomingDataManager java:global/hawkular-metrics/hawkular-alerts/IncomingDataManagerImpl java:app/hawkular-alerts/IncomingDataManagerImpl java:module/IncomingDataManagerImpl 18:36:32,192 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'AlertsContext' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/AlertsContext!org.hawkular.alerts.engine.impl.AlertsContext java:app/hawkular-alerts/AlertsContext!org.hawkular.alerts.engine.impl.AlertsContext java:module/AlertsContext!org.hawkular.alerts.engine.impl.AlertsContext java:global/hawkular-metrics/hawkular-alerts/AlertsContext java:app/hawkular-alerts/AlertsContext java:module/AlertsContext 18:36:32,192 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'AlertsEngineImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/AlertsEngineImpl!org.hawkular.alerts.engine.service.AlertsEngine java:app/hawkular-alerts/AlertsEngineImpl!org.hawkular.alerts.engine.service.AlertsEngine java:module/AlertsEngineImpl!org.hawkular.alerts.engine.service.AlertsEngine java:global/hawkular-metrics/hawkular-alerts/AlertsEngineImpl java:app/hawkular-alerts/AlertsEngineImpl java:module/AlertsEngineImpl 18:36:32,192 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'CassActionsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/CassActionsServiceImpl!org.hawkular.alerts.api.services.ActionsService java:app/hawkular-alerts/CassActionsServiceImpl!org.hawkular.alerts.api.services.ActionsService java:module/CassActionsServiceImpl!org.hawkular.alerts.api.services.ActionsService java:global/hawkular-metrics/hawkular-alerts/CassActionsServiceImpl java:app/hawkular-alerts/CassActionsServiceImpl java:module/CassActionsServiceImpl 18:36:32,193 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'PartitionManagerImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/PartitionManagerImpl!org.hawkular.alerts.engine.service.PartitionManager java:app/hawkular-alerts/PartitionManagerImpl!org.hawkular.alerts.engine.service.PartitionManager java:module/PartitionManagerImpl!org.hawkular.alerts.engine.service.PartitionManager java:global/hawkular-metrics/hawkular-alerts/PartitionManagerImpl java:app/hawkular-alerts/PartitionManagerImpl java:module/PartitionManagerImpl 18:36:32,193 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'CassAlertsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/CassAlertsServiceImpl!org.hawkular.alerts.api.services.AlertsService java:app/hawkular-alerts/CassAlertsServiceImpl!org.hawkular.alerts.api.services.AlertsService java:module/CassAlertsServiceImpl!org.hawkular.alerts.api.services.AlertsService java:global/hawkular-metrics/hawkular-alerts/CassAlertsServiceImpl java:app/hawkular-alerts/CassAlertsServiceImpl java:module/CassAlertsServiceImpl 18:36:32,193 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'StatusServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/StatusServiceImpl!org.hawkular.alerts.api.services.StatusService java:app/hawkular-alerts/StatusServiceImpl!org.hawkular.alerts.api.services.StatusService java:module/StatusServiceImpl!org.hawkular.alerts.api.services.StatusService java:global/hawkular-metrics/hawkular-alerts/StatusServiceImpl java:app/hawkular-alerts/StatusServiceImpl java:module/StatusServiceImpl 18:36:32,194 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'StreamWatcher' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/StreamWatcher!org.hawkular.alerts.rest.StreamWatcher java:app/hawkular-alerts/StreamWatcher!org.hawkular.alerts.rest.StreamWatcher java:module/StreamWatcher!org.hawkular.alerts.rest.StreamWatcher java:global/hawkular-metrics/hawkular-alerts/StreamWatcher java:app/hawkular-alerts/StreamWatcher java:module/StreamWatcher 18:36:32,194 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'PublishCacheManager' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/PublishCacheManager!org.hawkular.alerts.engine.cache.PublishCacheManager java:app/hawkular-alerts/PublishCacheManager!org.hawkular.alerts.engine.cache.PublishCacheManager java:module/PublishCacheManager!org.hawkular.alerts.engine.cache.PublishCacheManager java:global/hawkular-metrics/hawkular-alerts/PublishCacheManager java:app/hawkular-alerts/PublishCacheManager java:module/PublishCacheManager 18:36:32,194 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'DataDrivenGroupCacheManager' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/DataDrivenGroupCacheManager!org.hawkular.alerts.engine.impl.DataDrivenGroupCacheManager java:app/hawkular-alerts/DataDrivenGroupCacheManager!org.hawkular.alerts.engine.impl.DataDrivenGroupCacheManager java:module/DataDrivenGroupCacheManager!org.hawkular.alerts.engine.impl.DataDrivenGroupCacheManager java:global/hawkular-metrics/hawkular-alerts/DataDrivenGroupCacheManager java:app/hawkular-alerts/DataDrivenGroupCacheManager java:module/DataDrivenGroupCacheManager 18:36:32,195 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'ExtensionsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/ExtensionsServiceImpl!org.hawkular.alerts.api.services.ExtensionsService java:app/hawkular-alerts/ExtensionsServiceImpl!org.hawkular.alerts.api.services.ExtensionsService java:module/ExtensionsServiceImpl!org.hawkular.alerts.api.services.ExtensionsService java:global/hawkular-metrics/hawkular-alerts/ExtensionsServiceImpl java:app/hawkular-alerts/ExtensionsServiceImpl java:module/ExtensionsServiceImpl 18:36:32,195 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'EventsAggregationExtension' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/EventsAggregationExtension!org.hawkular.alerts.api.services.EventExtension java:app/hawkular-alerts/EventsAggregationExtension!org.hawkular.alerts.api.services.EventExtension java:module/EventsAggregationExtension!org.hawkular.alerts.api.services.EventExtension java:global/hawkular-metrics/hawkular-alerts/EventsAggregationExtension java:app/hawkular-alerts/EventsAggregationExtension java:module/EventsAggregationExtension 18:36:32,196 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'CassCluster' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/CassCluster!org.hawkular.alerts.engine.impl.CassCluster java:app/hawkular-alerts/CassCluster!org.hawkular.alerts.engine.impl.CassCluster java:module/CassCluster!org.hawkular.alerts.engine.impl.CassCluster java:global/hawkular-metrics/hawkular-alerts/CassCluster java:app/hawkular-alerts/CassCluster java:module/CassCluster 18:36:32,196 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'CassDefinitionsServiceImpl' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/CassDefinitionsServiceImpl!org.hawkular.alerts.api.services.DefinitionsService java:app/hawkular-alerts/CassDefinitionsServiceImpl!org.hawkular.alerts.api.services.DefinitionsService java:module/CassDefinitionsServiceImpl!org.hawkular.alerts.api.services.DefinitionsService java:global/hawkular-metrics/hawkular-alerts/CassDefinitionsServiceImpl java:app/hawkular-alerts/CassDefinitionsServiceImpl java:module/CassDefinitionsServiceImpl 18:36:32,196 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-8) WFLYEJB0473: JNDI bindings for session bean named 'ActionsCacheManager' in deployment unit 'subdeployment "hawkular-alerts.war" of deployment "hawkular-metrics.ear"' are as follows: java:global/hawkular-metrics/hawkular-alerts/ActionsCacheManager!org.hawkular.alerts.engine.cache.ActionsCacheManager java:app/hawkular-alerts/ActionsCacheManager!org.hawkular.alerts.engine.cache.ActionsCacheManager java:module/ActionsCacheManager!org.hawkular.alerts.engine.cache.ActionsCacheManager java:global/hawkular-metrics/hawkular-alerts/ActionsCacheManager java:app/hawkular-alerts/ActionsCacheManager java:module/ActionsCacheManager 18:36:32,260 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started client-mappings cache from ejb container 18:36:32,685 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 80) WFLYCLINF0002: Started schema cache from hawkular-alerts container 18:36:32,685 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 71) WFLYCLINF0002: Started data cache from hawkular-alerts container 18:36:32,685 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 69) WFLYCLINF0002: Started publish cache from hawkular-alerts container 18:36:32,685 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 63) WFLYCLINF0002: Started dataIds cache from hawkular-alerts container 18:36:32,687 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 78) WFLYCLINF0002: Started globalActions cache from hawkular-alerts container 18:36:32,691 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 67) WFLYCLINF0002: Started partition cache from hawkular-alerts container 18:36:32,692 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 73) WFLYCLINF0002: Started triggers cache from hawkular-alerts container 18:36:33,990 INFO [org.infinispan.factories.GlobalComponentRegistry] (ServerService Thread Pool -- 67) ISPN000128: Infinispan version: Infinispan 'Bastille' 9.1.1.Final 18:36:34,043 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 76) HAWKALERT220023: Init Actions Cache 18:36:34,043 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 74) HAWKALERT220014: Hawkular Alerts deployed in single node mode 18:36:34,062 WARN [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 77) HAWKALERT220021: Clear Publish Cache 18:36:34,063 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 77) HAWKALERT220020: Init Publish Cache 18:36:34,068 INFO [org.hawkular.alerts.extensions.EventsAggregationExtension] (ServerService Thread Pool -- 82) Registering Distributed Trigger listener 18:36:34,172 INFO [com.datastax.driver.core.GuavaCompatibility] (ServerService Thread Pool -- 73) Detected Guava < 19 in the classpath, using legacy compatibility layer 18:36:34,190 INFO [com.datastax.driver.core.Native] (ServerService Thread Pool -- 73) Could not load JNR C Library, native system calls through this library will not be available (set this logger level to DEBUG to see the full stack trace). 18:36:34,191 INFO [com.datastax.driver.core.ClockFactory] (ServerService Thread Pool -- 73) Using java.lang.System clock to generate timestamps. 18:36:34,282 INFO [com.datastax.driver.core.NettyUtil] (ServerService Thread Pool -- 73) Found Netty's native epoll transport in the classpath, using it 18:36:34,744 INFO [org.infinispan.query.impl.LifecycleManager] (ServerService Thread Pool -- 67) ISPN014003: Registering Query interceptor for cache resource 18:36:34,778 INFO [org.hibernate.search.engine.Version] (ServerService Thread Pool -- 67) HSEARCH000034: Hibernate Search 5.8.0.Final 18:36:34,803 INFO [org.hibernate.annotations.common.Version] (ServerService Thread Pool -- 67) HCANN000001: Hibernate Commons Annotations {5.0.1.Final} 18:36:34,927 INFO [com.datastax.driver.core.policies.DCAwareRoundRobinPolicy] (ServerService Thread Pool -- 73) Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) 18:36:34,931 INFO [com.datastax.driver.core.Cluster] (ServerService Thread Pool -- 73) New Cassandra host hawkular-cassandra/172.30.23.124:9042 added 18:36:35,008 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Checking Schema existence for keyspace: hawkular_alerts 18:36:36,102 INFO [org.infinispan.query.impl.LifecycleManager] (ServerService Thread Pool -- 67) ISPN014003: Registering Query interceptor for cache resource_type 18:36:36,367 INFO [org.hawkular.inventory.service.InventoryConfig] (ServerService Thread Pool -- 67) HAWKINV100003: Inventory is reindexing caches 18:36:36,427 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Executing [script:vfs:/content/hawkular-metrics.ear/hawkular-alerts.war/WEB-INF/lib/hawkular-alerts-engine-1.6.2.Final.jar/org/hawkular/alerts/schema/cassalog.groovy, tags:[], vars:[keyspace:hawkular_alerts, reset:false, session:com.datastax.driver.core.SessionManager@bfc7f87, logger:org.jboss.logging.JBossLogManagerLogger@5ca45b21]] 18:36:36,659 INFO [org.infinispan.query.impl.massindex.IndexUpdater] (DefaultExecutorService--p25-t1) ISPN014016: Purging instances of 'org.hawkular.inventory.service.ispn.IspnResourceType' from the index 18:36:36,659 INFO [org.infinispan.query.impl.massindex.IndexUpdater] (DefaultExecutorService--p24-t1) ISPN014016: Purging instances of 'org.hawkular.inventory.service.ispn.IspnResource' from the index 18:36:36,813 INFO [org.infinispan.query.impl.massindex.IndexUpdater] (DefaultExecutorService--p24-t1) ISPN014017: Flushing index 'org.hawkular.inventory.service.ispn.IspnResource' 18:36:36,813 INFO [org.infinispan.query.impl.massindex.IndexUpdater] (DefaultExecutorService--p25-t1) ISPN014017: Flushing index 'org.hawkular.inventory.service.ispn.IspnResourceType' 18:36:36,816 INFO [org.hawkular.inventory.service.InventoryConfig] (ServerService Thread Pool -- 67) HAWKINV100004: Inventory finished reindexing in [443] ms 18:36:36,817 INFO [org.hawkular.inventory.service.InventoryConfig] (ServerService Thread Pool -- 67) HAWKINV100000: Inventory App Started 18:36:36,969 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 78) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.inventory.handlers.InventoryApp$Proxy$_$$_WeldClientProxy 18:36:37,062 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 78) WFLYUT0021: Registered web context: /hawkular/inventory 18:36:37,125 INFO [org.jboss.weld.deployer] (MSC service thread 1-4) WFLYWELD0003: Processing weld deployment hawkular-inventory-ui.war 18:36:37,613 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 67) WFLYUT0021: Registered web context: /hawkular/inventory/ui 18:36:37,855 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table triggers 18:36:38,637 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table triggers_actions 18:36:38,979 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table conditions 18:36:40,123 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table dampenings 18:36:41,206 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table action_plugins 18:36:41,732 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table actions_definitions 18:36:42,094 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table actions_history 18:36:42,514 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table actions_history_actions 18:36:42,991 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table actions_history_alerts 18:36:43,412 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table actions_history_ctimes 18:36:43,917 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table actions_history_results 18:36:44,399 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table tags 18:36:44,750 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table alerts 18:36:45,187 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table alerts_triggers 18:36:45,641 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table alerts_ctimes 18:36:46,067 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table alerts_statuses 18:36:46,551 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table alerts_severities 18:36:47,012 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table alerts_lifecycle 18:36:47,382 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table events 18:36:47,806 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table events_triggers 18:36:48,252 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table events_ctimes 18:36:48,633 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Creating table events_categories 18:36:49,510 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Creating change log table hawkular_alerts.cassalog 18:36:50,027 WARN [com.datastax.driver.core.Cluster] (cluster1-worker-10) Re-preparing already prepared query is generally an anti-pattern and will likely affect performance. Consider preparing the statement only once. Query='UPDATE hawkular_alerts.cassalog SET applied_at = ? WHERE bucket = ? AND revision = ?' 18:36:50,047 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: set-keyspace USE hawkular_alerts -- 18:36:50,122 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: set-keyspace USE hawkular_alerts -- 18:36:50,134 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 1.0 CREATE TABLE sys_config ( config_id text, name text, value text, PRIMARY KEY (config_id, name) ) -- 18:36:50,518 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: set-keyspace USE hawkular_alerts -- 18:36:50,527 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 2.0 ALTER TABLE conditions ADD interval bigint -- 18:36:51,004 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: set-keyspace USE hawkular_alerts -- 18:36:51,015 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 3.0 DROP TABLE alerts_statuses -- 18:36:52,178 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 3.1 DROP TABLE alerts_severities -- 18:36:52,883 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: set-keyspace USE hawkular_alerts -- 18:36:52,891 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 4.0 ALTER TABLE conditions ADD activeRules set -- 18:36:53,334 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 4.1 ALTER TABLE conditions ADD sampleSize int -- 18:36:53,819 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: set-keyspace USE hawkular_alerts -- 18:36:53,827 INFO [org.cassalog.core.CassalogImpl] (ServerService Thread Pool -- 73) Applying ChangeSet -- version: 5.0 CREATE TABLE alerts_stimes ( tenantId text, alertId text, stime bigint, PRIMARY KEY (tenantId, stime, alertId) ) -- 18:36:54,322 INFO [org.hawkular.alerts.engine.impl.CassCluster] (ServerService Thread Pool -- 73) Done creating Schema for keyspace: hawkular_alerts 18:36:54,332 INFO [org.hawkular.alerts.engine.impl.ExtensionsServiceImpl] (ServerService Thread Pool -- 82) Adding EventExtension org.hawkular.alerts.extensions.EventsAggregationExtension@215ed3a7 18:36:54,358 INFO [org.drools.compiler.kie.builder.impl.ClasspathKieProject] (EE-ManagedExecutorService-default-Thread-1) Found kmodule: vfs:/content/hawkular-metrics.ear/hawkular-alerts.war/WEB-INF/lib/hawkular-alerts-engine-1.6.2.Final.jar/META-INF/kmodule.xml 18:36:54,361 INFO [org.drools.compiler.kie.builder.impl.ClasspathKieProject] (EE-ManagedExecutorService-default-Thread-1) Virtual file physical path = /opt/jboss/wildfly/standalone/tmp/vfs/deployment/deployment58a7477e4b886dbd/hawkular-alerts-engine-1.6.2.Final.jar-d7c71ff68f117471/hawkular-alerts-engine-1.6.2.Final.jar 18:36:54,411 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 77) Initializing Mojarra 2.2.13.SP1 20160303-1204 for context '/hawkular/alerts' 18:36:54,739 INFO [org.drools.compiler.kie.builder.impl.KieRepositoryImpl] (EE-ManagedExecutorService-default-Thread-1) KieModule was added: ZipKieModule[releaseId=org.hawkular.alerts:hawkular-alerts-engine:1.6.2.Final,file=/opt/jboss/wildfly/standalone/tmp/vfs/deployment/deployment58a7477e4b886dbd/hawkular-alerts-engine-1.6.2.Final.jar-d7c71ff68f117471/hawkular-alerts-engine-1.6.2.Final.jar] 18:36:56,643 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 77) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.alerts.rest.HawkularAlertsApp$Proxy$_$$_WeldClientProxy 18:36:57,267 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0021: Registered web context: /hawkular/alerts 18:36:57,382 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 77) HAWKALERT220006: ActionListener [org.hawkular.alerts.actions.standalone.StandaloneActionPluginListener@46701850] registered 18:36:57,385 INFO [org.hawkular.alerts.actions.standalone] (ServerService Thread Pool -- 77) HAWKALERT270001: Action plugin [elasticsearch] registered 18:36:57,402 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0021: Registered web context: /hawkular/actions/elasticsearch 18:36:57,904 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 77) HAWKALERT220006: ActionListener [org.hawkular.alerts.actions.standalone.StandaloneActionPluginListener@52571596] registered 18:36:57,905 INFO [org.hawkular.alerts.actions.standalone] (ServerService Thread Pool -- 77) HAWKALERT270001: Action plugin [email] registered 18:36:57,927 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 77) WFLYUT0021: Registered web context: /hawkular/actions/email 18:36:58,019 INFO [org.hawkular.alerts.engine.log] (ServerService Thread Pool -- 73) HAWKALERT220006: ActionListener [org.hawkular.alerts.actions.standalone.StandaloneActionPluginListener@6d6e3451] registered 18:36:58,020 INFO [org.hawkular.alerts.actions.standalone] (ServerService Thread Pool -- 73) HAWKALERT270001: Action plugin [webhook] registered 18:36:58,059 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 73) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.alerts.actions.webhook.WebHookApp 18:36:58,066 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 73) WFLYUT0021: Registered web context: /hawkular/actions/webhook 18:36:58,125 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 73) Initializing Mojarra 2.2.13.SP1 20160303-1204 for context '/hawkular/__alerter-elasticsearch' 18:36:58,379 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 73) WFLYUT0021: Registered web context: /hawkular/__alerter-elasticsearch 18:36:58,406 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 73) WFLYCLINF0002: Started locks cache from hawkular-metrics container 18:36:58,471 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (ServerService Thread Pool -- 67) Hawkular Metrics version: 0.26.1.Final 18:36:58,473 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) HAWKMETRICS200002: Initializing metrics service 18:36:58,476 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 67) Initializing Mojarra 2.2.13.SP1 20160303-1204 for context '/hawkular/metrics' 18:36:58,479 INFO [com.datastax.driver.core.ClockFactory] (metricsservice-lifecycle-thread) Using java.lang.System clock to generate timestamps. 18:36:58,724 INFO [com.datastax.driver.core.policies.DCAwareRoundRobinPolicy] (metricsservice-lifecycle-thread) Using data-center name 'datacenter1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor) 18:36:58,725 INFO [com.datastax.driver.core.Cluster] (metricsservice-lifecycle-thread) New Cassandra host hawkular-cassandra/172.30.23.124:9042 added 18:36:58,836 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Executing [script:vfs:/content/hawkular-metrics.ear/hawkular-metrics.war/WEB-INF/lib/hawkular-metrics-schema-0.26.1.Final.jar/org/hawkular/schema/cassalog.groovy, tags:[0.15.x, 0.18.x, 0.19.x, 0.20.x, 0.21.x, 0.23.x, 0.26.x], vars:[keyspace:hawkular_metrics, reset:false, session:com.datastax.driver.core.SessionManager@6e309ee4, replicationFactor:1, logger:org.jboss.logging.JBossLogManagerLogger@38e2bfd1]] 18:36:59,107 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table tenants 18:36:59,301 INFO [org.hawkular.metrics.api.jaxrs.HawkularMetricsRestApp] (ServerService Thread Pool -- 67) HAWKMETRICS200001: Hawkular Metrics starting 18:36:59,302 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 67) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.metrics.api.jaxrs.HawkularMetricsRestApp 18:36:59,544 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating type aggregation_template 18:36:59,939 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 67) WFLYUT0021: Registered web context: /hawkular/metrics 18:36:59,944 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table tenants_by_time 18:37:00,040 INFO [org.hawkular.metrics.alerter.ConditionManager] (ServerService Thread Pool -- 73) Starting Hawkular Metrics External Alerter, distributed=false 18:37:00,042 INFO [org.hawkular.metrics.alerter.groups.GroupTriggerManager] (ServerService Thread Pool -- 82) Starting Hawkular Metrics Group Trigger Manager, distributed=false 18:37:00,091 INFO [org.hawkular.metrics.alerter.ConditionManager] (ServerService Thread Pool -- 73) Registering Trigger UPDATE/REMOVE listener 18:37:00,112 INFO [org.hawkular.metrics.alerter.groups.GroupTriggerManager] (ServerService Thread Pool -- 82) Registering Trigger UPDATE/REMOVE listener 18:37:00,138 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 82) Initializing Mojarra 2.2.13.SP1 20160303-1204 for context '/hawkular/__alerter-metrics' 18:37:00,403 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 82) WFLYUT0021: Registered web context: /hawkular/__alerter-metrics 18:37:00,479 WARN [org.jboss.as.dependency.private] (MSC service thread 1-8) WFLYSRV0018: Deployment "deployment.hawkular-rest-api.war" is using a private module ("com.google.guava:main") which may be changed or removed in future versions without notice. 18:37:00,480 WARN [org.jboss.as.dependency.private] (MSC service thread 1-8) WFLYSRV0018: Deployment "deployment.hawkular-rest-api.war" is using a private module ("org.infinispan:main") which may be changed or removed in future versions without notice. 18:37:00,491 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating type aggregate_data 18:37:00,503 INFO [org.jboss.weld.deployer] (MSC service thread 1-2) WFLYWELD0003: Processing weld deployment hawkular-rest-api.war 18:37:00,541 INFO [org.jboss.as.ejb3.deployment] (MSC service thread 1-2) WFLYEJB0473: JNDI bindings for session bean named 'BackfillCacheManager' in deployment unit 'deployment "hawkular-rest-api.war"' are as follows: java:global/hawkular-rest-api/BackfillCacheManager!org.hawkular.listener.cache.BackfillCache java:app/hawkular-rest-api/BackfillCacheManager!org.hawkular.listener.cache.BackfillCache java:module/BackfillCacheManager!org.hawkular.listener.cache.BackfillCache java:global/hawkular-rest-api/BackfillCacheManager java:app/hawkular-rest-api/BackfillCacheManager java:module/BackfillCacheManager 18:37:00,660 INFO [org.jboss.as.clustering.infinispan] (ServerService Thread Pool -- 82) WFLYCLINF0002: Started backfill cache from hawkular-services container 18:37:00,749 INFO [org.jboss.as.ejb3] (MSC service thread 1-4) WFLYEJB0042: Started message driven bean 'CommandEventListener' with 'activemq-ra.rar' resource adapter 18:37:00,750 INFO [org.jboss.as.ejb3] (MSC service thread 1-5) WFLYEJB0042: Started message driven bean 'HawkularTopicListener' with 'activemq-ra.rar' resource adapter 18:37:00,750 INFO [org.jboss.as.ejb3] (MSC service thread 1-2) WFLYEJB0042: Started message driven bean 'FeedAvailabilityDataListener' with 'activemq-ra.rar' resource adapter 18:37:00,751 INFO [org.jboss.as.ejb3] (MSC service thread 1-7) WFLYEJB0042: Started message driven bean 'HawkularQueueListener' with 'activemq-ra.rar' resource adapter 18:37:00,810 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table data 18:37:01,119 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table metrics_tags_idx 18:37:01,214 INFO [org.hawkular.listener.cache.BackfillCacheManager] (ServerService Thread Pool -- 82) Initializing Standalone Availability Cache 18:37:01,271 INFO [javax.enterprise.resource.webcontainer.jsf.config] (ServerService Thread Pool -- 67) Initializing Mojarra 2.2.13.SP1 20160303-1204 for context '/hawkular/api' 18:37:01,601 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table metrics_idx 18:37:01,606 INFO [org.apache.activemq.artemis.ra] (default-threads - 4) AMQ151000: awaiting topic/queue creation HawkularCommandEvent 18:37:01,612 INFO [org.apache.activemq.artemis.ra] (default-threads - 1) AMQ151000: awaiting topic/queue creation HawkularQueue 18:37:01,702 INFO [org.hawkular.rest] (ServerService Thread Pool -- 67) HAWKULAR005500: Hawkular REST Api is starting... 18:37:01,702 INFO [org.jboss.resteasy.resteasy_jaxrs.i18n] (ServerService Thread Pool -- 67) RESTEASY002225: Deploying javax.ws.rs.core.Application: class org.hawkular.rest.HawkularRestApi$Proxy$_$$_WeldClientProxy 18:37:01,702 INFO [org.hawkular.rest] (ServerService Thread Pool -- 67) HAWKULAR005500: Hawkular REST Api is starting... 18:37:01,710 INFO [org.wildfly.extension.undertow] (ServerService Thread Pool -- 67) WFLYUT0021: Registered web context: /hawkular/api 18:37:01,786 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-wildfly-agent-download.war" (runtime-name : "hawkular-wildfly-agent-download.war") 18:37:01,787 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-status.war" (runtime-name : "hawkular-status.war") 18:37:01,787 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-rest-api.war" (runtime-name : "hawkular-rest-api.war") 18:37:01,787 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-metrics.ear" (runtime-name : "hawkular-metrics.ear") 18:37:01,787 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-inventory-ui.war" (runtime-name : "hawkular-inventory-ui.war") 18:37:01,788 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-inventory-service.war" (runtime-name : "hawkular-inventory-service.war") 18:37:01,788 INFO [org.jboss.as.server] (ServerService Thread Pool -- 61) WFLYSRV0010: Deployed "hawkular-command-gateway-war.war" (runtime-name : "hawkular-command-gateway-war.war") 18:37:01,996 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0061: Http management interface listening on https://0.0.0.0:9993/management 18:37:01,997 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0052: Admin console listening on https://0.0.0.0:9993 18:37:01,998 INFO [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: Hawkular 0.40.0.Final-SNAPSHOT (WildFly Core 2.2.0.Final) started in 44812ms - Started 2269 of 2758 services (859 services are lazy, passive or on-demand) 18:37:02,076 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table retentions_idx 18:37:02,411 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating type trigger_def 18:37:02,672 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table tasks 18:37:03,118 INFO [org.hawkular.metrics.schema.SchemaService] (metricsservice-lifecycle-thread) Creating table leases 18:37:03,608 INFO [org.apache.activemq.artemis.ra] (default-threads - 4) AMQ151001: Attempting to reconnect org.apache.activemq.artemis.ra.inflow.ActiveMQActivationSpec(ra=org.apache.activemq.artemis.ra.ActiveMQResourceAdapter@78712571 destination=HawkularCommandEvent destinationType=javax.jms.Queue ack=Auto-acknowledge durable=false clientID=null user=null maxSession=15) 18:37:03,614 INFO [org.apache.activemq.artemis.ra] (default-threads - 1) AMQ151001: Attempting to reconnect org.apache.activemq.artemis.ra.inflow.ActiveMQActivationSpec(ra=org.apache.activemq.artemis.ra.ActiveMQResourceAdapter@78712571 destination=HawkularQueue destinationType=javax.jms.Queue ack=Auto-acknowledge durable=false clientID=null user=null maxSession=15) 18:37:04,117 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Creating change log table hawkular_metrics.cassalog 18:37:04,575 WARN [com.datastax.driver.core.Cluster] (cluster2-worker-6) Re-preparing already prepared query is generally an anti-pattern and will likely affect performance. Consider preparing the statement only once. Query='UPDATE hawkular_metrics.cassalog SET applied_at = ? WHERE bucket = ? AND revision = ?' 18:37:04,600 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: set-keyspace USE hawkular_metrics -- 18:37:04,608 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 1.0 CREATE TABLE sys_config ( config_id text, name text, value text, PRIMARY KEY (config_id, name) ) WITH compaction = { 'class': 'LeveledCompactionStrategy' } -- 18:37:04,890 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 1.1 ALTER TABLE data WITH gc_grace_seconds = 86400 -- 18:37:05,249 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 1.2 ALTER TABLE data ADD tags map -- 18:37:05,759 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 1.3 ALTER TABLE data ADD s_value text -- 18:37:06,058 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 1.4 INSERT INTO sys_config (config_id, name, value) VALUES ('org.hawkular.metrics', 'string-size', '2048') -- 18:37:06,066 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: set-keyspace USE hawkular_metrics -- 18:37:06,072 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 2.0 CREATE TABLE locks ( name text PRIMARY KEY, value text, ) WITH compaction = { 'class': 'LeveledCompactionStrategy' } -- 18:37:06,577 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 2.1 CREATE TABLE jobs ( id uuid PRIMARY KEY, type text, name text, params map, trigger frozen ) WITH compaction = { 'class': 'LeveledCompactionStrategy' } -- 18:37:07,161 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 2.2 CREATE TABLE scheduled_jobs_idx ( time_slice timestamp, job_id uuid, PRIMARY KEY (time_slice, job_id) ) WITH compaction = { 'class': 'LeveledCompactionStrategy' } -- 18:37:07,495 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 2.3 CREATE TABLE finished_jobs_idx ( time_slice timestamp, job_id uuid, PRIMARY KEY (time_slice, job_id) ) WITH compaction = { 'class': 'LeveledCompactionStrategy' } -- 18:37:07,862 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 2.4 CREATE TABLE active_time_slices ( time_slice timestamp PRIMARY KEY ) WITH compaction = { 'class' : 'LeveledCompactionStrategy' } -- 18:37:08,952 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.0 ALTER TABLE hawkular_metrics.data WITH COMPRESSION = {'sstable_compression': 'DeflateCompressor'}; -- 18:37:09,768 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.1 ALTER TABLE data WITH compaction = {'class': 'DateTieredCompactionStrategy'} AND default_time_to_live = 604800 -- 18:37:09,996 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.2 ALTER TABLE data DROP aggregates -- 18:37:10,680 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.3 DROP TYPE aggregate_data -- 18:37:10,970 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.4 ALTER TABLE data DROP data_retention -- 18:37:11,421 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.5 DROP TYPE aggregation_template -- 18:37:11,748 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 3.6 DROP TABLE tenants_by_time -- 18:37:13,224 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: set-keyspace USE hawkular_metrics -- 18:37:13,231 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 4.0 CREATE TABLE data_compressed ( tenant_id text, type tinyint, metric text, dpart bigint, time timestamp, c_value blob, ts_value blob, tags blob, PRIMARY KEY ((tenant_id, type, metric, dpart), time) ) WITH CLUSTERING ORDER BY (time DESC) -- 18:37:13,652 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 4.1 ALTER TABLE data WITH COMPRESSION = {'sstable_compression': 'LZ4Compressor'}; -- 18:37:13,975 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.0 ALTER TABLE scheduled_jobs_idx ADD job_name text -- 18:37:14,439 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.1 ALTER TABLE scheduled_jobs_idx ADD job_params frozen> -- 18:37:14,891 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.2 ALTER TABLE scheduled_jobs_idx ADD trigger frozen -- 18:37:15,342 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.3 ALTER TABLE scheduled_jobs_idx ADD job_type text -- 18:37:15,700 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.4 ALTER TABLE scheduled_jobs_idx ADD status tinyint -- 18:37:16,115 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.5 INSERT INTO sys_config (config_id, name, value) VALUES ('org.hawkular.metrics', 'gc_grace_seconds', '604800') -- 18:37:16,120 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 5.6 BEGIN UNLOGGED BATCH INSERT INTO sys_config (config_id, name, value) VALUES ('org.hawkular.metrics', 'ingestion.retry.max-retries', '5'); INSERT INTO sys_config (config_id, name, value) VALUES ('org.hawkular.metrics', 'ingestion.retry.max-delay', '30000'); APPLY BATCH; -- 18:37:16,130 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 6.0 ALTER TABLE data_compressed WITH compaction = { 'class': 'TimeWindowCompactionStrategy', 'compaction_window_unit': 'DAYS', 'compaction_window_size': '1' } -- 18:37:16,391 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 6.1 ALTER TABLE data WITH compaction = {'class': 'SizeTieredCompactionStrategy'} -- 18:37:16,650 INFO [org.cassalog.core.CassalogImpl] (metricsservice-lifecycle-thread) Applying ChangeSet -- version: 7.0 CREATE TABLE metrics_expiration_idx ( tenant_id text, type tinyint, metric text, time timestamp, PRIMARY KEY ((tenant_id, type), metric) ) -- 18:37:17,375 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) The keyspace hawkular_metrics is using a replication_factor of 1 18:37:17,674 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100001: Using a key space of 'hawkular_metrics' 18:37:17,787 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100008: Using max size of 2048 for string metrics 18:37:17,788 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100009: Using max number of retries 5 and max retry delay of 30000 ms for inserting data points 18:37:18,019 INFO [org.hawkular.metrics.core.service.MetricsServiceImpl] (metricsservice-lifecycle-thread) HAWKMETRICS100007: Using default data retention of 1209600 seconds 18:37:18,040 INFO [org.hawkular.metrics.core.dropwizard.MetricsInitializer] (metricsservice-lifecycle-thread) Creating metrics 18:37:18,326 INFO [org.hawkular.metrics.core.dropwizard.MetricsInitializer] (RxComputationScheduler-1) Finished creating metrics in 286 ms 18:37:18,462 WARN [com.datastax.driver.core.Cluster] (cluster2-worker-16) Re-preparing already prepared query is generally an anti-pattern and will likely affect performance. Consider preparing the statement only once. Query='SELECT DISTINCT tenant_id, tname FROM metrics_tags_idx' 18:37:18,479 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Preparing to create and schedule COMPRESS_DATA job 18:37:18,578 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Created and scheduled JobDetails{jobId=cfceef37-77e8-49e0-b8e5-6e9f49fbe139, jobType=COMPRESS_DATA, jobName=COMPRESS_DATA, parameters={}, trigger=RepeatingTrigger{triggerTime=1509044400000, interval=7200000, delay=60000}, status=NONE} 18:37:18,589 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Preparing to create and schedule DELETE_EXPIRED_METRICS job 18:37:18,602 INFO [org.hawkular.metrics.core.jobs.JobsServiceImpl] (metricsservice-lifecycle-thread) Created and scheduled JobDetails{jobId=e0a710f2-828b-4698-b4c8-b53b7795b534, jobType=DELETE_EXPIRED_METRICS, jobName=DELETE_EXPIRED_METRICS, parameters={}, trigger=RepeatingTrigger{triggerTime=1509062400000, interval=604800000, delay=60000}, status=NONE} 18:37:18,619 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (metricsservice-lifecycle-thread) Checking tables in hawkular_metrics to see if gc_grace_seconds needs to be updated 18:37:18,630 INFO [org.hawkular.metrics.api.jaxrs.MetricsServiceLifecycle] (metricsservice-lifecycle-thread) HAWKMETRICS200005: Metrics service started 18:37:18,631 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-10) gc_grace_seconds for active_time_slices is set to 864000. Resetting to 0 18:37:18,636 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-10) gc_grace_seconds for cassalog is set to 864000. Resetting to 0 18:37:18,636 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-10) gc_grace_seconds for data is set to 86400. Resetting to 0 18:37:18,854 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-18) gc_grace_seconds for data_compressed is set to 864000. Resetting back to zero. 18:37:19,072 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-16) gc_grace_seconds for finished_jobs_idx is set to 864000. Resetting to 0 18:37:19,360 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-1) gc_grace_seconds for jobs is set to 864000. Resetting to 0 18:37:19,674 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-4) gc_grace_seconds for leases is set to 864000. Resetting to 0 18:37:20,089 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-3) gc_grace_seconds for locks is set to 864000. Resetting to 0 18:37:20,386 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-5) gc_grace_seconds for metrics_expiration_idx is set to 864000. Resetting to 0 18:37:20,702 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-6) gc_grace_seconds for metrics_idx is set to 864000. Resetting to 0 18:37:21,034 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-7) gc_grace_seconds for metrics_tags_idx is set to 864000. Resetting to 0 18:37:21,297 INFO [org.hawkular.agent.monitor.service.AgentCoreEngine] (Hawkular Java Agent Start Thread) Hawkular Metrics is ready: {"MetricsService":"STARTED","Implementation-Version":"0.26.1.Final","Built-From-Git-SHA1":"45b148c834ed62018f153c23187b4436ae4208fe","Cassandra":"up"} 18:37:21,332 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-19) gc_grace_seconds for retentions_idx is set to 864000. Resetting to 0 18:37:21,490 INFO [org.hawkular.agent.monitor.service.AgentCoreEngine] (Hawkular Java Agent Start Thread) Hawkular Inventory is ready: {"Implementation-Version":"0.9.8.Final-SRC-revision-ce866b1ade2d7413f055f461eb36d004ff2888d8","Built-From-Git-SHA1":"ce866b1ade2d7413f055f461eb36d004ff2888d8","status":"UP"} 18:37:21,561 INFO [org.hawkular.agent.monitor.cmd.FeedCommProcessor] (Hawkular Java Agent Start Thread) HAWKAGENT010039: The command-gateway URL is [wss://127.0.0.1:8443/hawkular/command-gateway/feed/hawkular-services-2-7z0xc] 18:37:21,644 INFO [org.hawkular.agent.monitor.scheduler.SchedulerService] (Hawkular Java Agent Start Thread) HAWKAGENT010012: Starting scheduler 18:37:21,671 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-20) gc_grace_seconds for scheduled_jobs_idx is set to 864000. Resetting to 0 18:37:21,672 INFO [org.hawkular.agent.monitor.protocol.ProtocolServices] (Hawkular Java Agent Start Thread) HAWKAGENT010019: Managed server [Local DMR] is disabled. It will not be monitored. 18:37:21,718 INFO [org.hawkular.agent.monitor.cmd.FeedCommProcessor] (OkHttp https://127.0.0.1:8443/...) HAWKAGENT010034: Opened feed WebSocket connection to endpoint [wss://127.0.0.1:8443/hawkular/command-gateway/feed/hawkular-services-2-7z0xc] 18:37:21,812 INFO [org.hawkular.cmdgw.command.ws.server.FeedWebSocket] (default task-48) HAWKFEEDCOMM000013: WebSocket session [hawkular-services-2-7z0xc] opened for endpoint [/feed/{feedId}] 18:37:21,929 INFO [org.xnio] (Hawkular Java Agent Start Thread) XNIO version 3.3.4.Final 18:37:21,968 INFO [org.xnio.nio] (Hawkular Java Agent Start Thread) XNIO NIO Implementation Version 3.3.4.Final 18:37:21,996 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-24) gc_grace_seconds for sys_config is set to 864000. Resetting to 0 18:37:22,052 INFO [org.jboss.remoting] (Hawkular Java Agent Start Thread) JBoss Remoting version 4.0.18.Final 18:37:22,274 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-21) gc_grace_seconds for tasks is set to 864000. Resetting to 0 18:37:22,592 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-23) gc_grace_seconds for tenants is set to 864000. Resetting to 0 18:37:22,864 INFO [org.hawkular.agent.monitor.protocol.ProtocolServices] (Hawkular Java Agent Start Thread) HAWKAGENT010057: Auto-discovery scans will be performed every [600] seconds 18:37:22,911 INFO [org.hawkular.agent.monitor.util.Util] (Hawkular WildFly Agent Full Discovery Scan-platform-1) Machine ID was explicitly set to [hawkular-services-2-7z0xc] 18:37:23,660 INFO [org.hawkular.metrics.core.util.GCGraceSecondsManager] (RxComputationScheduler-13) Finished gc_grace_seconds updates in 5040 ms 18:42:29,337 INFO [org.jboss.as.protocol] (management task-4) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 2,5,management-handler-thread] 18:57:06,338 INFO [org.jboss.as.protocol] (management task-2) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 1,5,management-handler-thread] 19:00:18,645 INFO [org.hawkular.metrics.core.jobs.CompressData] (RxIoScheduler-3) Starting compression of timestamps (UTC) between 1509033600000 - 1509040799999 19:00:21,066 INFO [org.hawkular.metrics.core.jobs.CompressData] (RxComputationScheduler-18) Finished compressing data in 2421 ms 19:00:40,207 INFO [org.jboss.as.protocol] (management task-3) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 4,5,management-handler-thread] 19:06:42,010 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Avail-3) HAWKAGENT010009: An availability check failed: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [server-state] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureAvails(EndpointService.java:516) at org.hawkular.agent.monitor.scheduler.AvailsCollector.run(AvailsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 19:52:28,022 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Metrics-3) HAWKAGENT010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[https-remoting://127.0.0.1:9993]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [messages-added] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:586) at org.hawkular.agent.monitor.scheduler.MetricsCollector.run(MetricsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 20:01:31,114 INFO [org.jboss.as.protocol] (management task-5) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 4,5,management-handler-thread] 20:09:03,120 INFO [org.jboss.as.protocol] (management task-10) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 1,5,management-handler-thread] 20:09:28,138 INFO [org.jboss.as.protocol] (management task-2) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 1,5,management-handler-thread] 20:13:51,072 INFO [org.jboss.as.protocol] (management task-10) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 3,5,management-handler-thread] 20:18:28,017 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Metrics-3) HAWKAGENT010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[https-remoting://127.0.0.1:9993]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [AverageGetTime] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:586) at org.hawkular.agent.monitor.scheduler.MetricsCollector.run(MetricsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 20:18:37,018 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Metrics-3) HAWKAGENT010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[https-remoting://127.0.0.1:9993]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [scheduled-count] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:586) at org.hawkular.agent.monitor.scheduler.MetricsCollector.run(MetricsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 20:18:37,206 INFO [org.jboss.as.protocol] (management task-7) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 4,5,management-handler-thread] 20:26:33,121 INFO [org.jboss.as.protocol] (management task-9) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 3,5,management-handler-thread] 20:27:32,163 INFO [org.jboss.as.protocol] (management task-5) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 4,5,management-handler-thread] 20:39:51,007 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Avail-3) HAWKAGENT010009: An availability check failed: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [status] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureAvails(EndpointService.java:516) at org.hawkular.agent.monitor.scheduler.AvailsCollector.run(AvailsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 21:00:18,631 INFO [org.hawkular.metrics.core.jobs.CompressData] (RxIoScheduler-3) Starting compression of timestamps (UTC) between 1509040800000 - 1509047999999 21:00:18,665 INFO [com.datastax.driver.core.Native] (RxComputationScheduler-17) Could not load JNR POSIX Library, native system calls through this library will not be available (set this logger level to DEBUG to see the full stack trace). 21:00:18,666 INFO [com.datastax.driver.core.utils.UUIDs] (RxComputationScheduler-17) PID obtained through JMX: 362 21:00:20,936 INFO [org.hawkular.metrics.core.jobs.CompressData] (RxComputationScheduler-6) Finished compressing data in 2304 ms 21:12:09,087 INFO [org.jboss.as.protocol] (management task-9) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 3,5,management-handler-thread] 21:13:52,156 INFO [org.jboss.as.protocol] (management task-9) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 4,5,management-handler-thread] 21:23:00,149 INFO [org.jboss.as.protocol] (management task-1) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 3,5,management-handler-thread] 21:25:26,123 INFO [org.jboss.as.protocol] (management task-5) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 3,5,management-handler-thread] 21:43:47,107 INFO [org.jboss.as.protocol] (management task-7) WFLYPRT0057: cancelled task by interrupting thread Thread[management-handler-thread - 1,5,management-handler-thread] 21:44:22,012 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Avail-3) HAWKAGENT010009: An availability check failed: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [server-state] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureAvails(EndpointService.java:516) at org.hawkular.agent.monitor.scheduler.AvailsCollector.run(AvailsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 21:57:08,159 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Metrics-3) HAWKAGENT010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[https-remoting://127.0.0.1:9993]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [message-count] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:586) at org.hawkular.agent.monitor.scheduler.MetricsCollector.run(MetricsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 22:03:16,502 ERROR [io.undertow.request] (default task-47) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:13,289 ERROR [io.undertow.request] (default task-63) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: Java heap space 22:00:22,444 ERROR [io.undertow.request] (default task-302) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:47,514 ERROR [io.undertow.request] (default task-290) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:00:42,695 ERROR [io.undertow.request] (default task-306) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:47,515 ERROR [io.undertow.request] (default task-106) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:00:27,809 ERROR [io.undertow.request] (default task-191) UT005023: Exception handling request to /hawkular/inventory/resources/feed-1bf3e4de-fcec-49dd-953b-3ba8ad0a99d9-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:54,392 ERROR [io.undertow.request] (default task-107) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:00:24,423 ERROR [io.undertow.request] (default task-263) UT005023: Exception handling request to /hawkular/inventory/resources/feed-79bb97ed-65fe-4c09-998a-fb6e2ea84103-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:05:34,051 ERROR [io.undertow.request] (default task-91) UT005023: Exception handling request to /hawkular/inventory/resources/feed-04911657-8bcc-4031-9a62-12c79a430808-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:30,534 ERROR [io.undertow.request] (default task-297) UT005023: Exception handling request to /hawkular/metrics/metrics/data: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:15,717 ERROR [io.undertow.request] (default task-6) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:12:21,544 ERROR [io.undertow.request] (default task-193) UT005023: Exception handling request to /hawkular/inventory/resources/feed-0d22198b-dbe0-4f4e-9e7d-19901cb698f1-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:09,999 ERROR [io.undertow.request] (default task-27) UT005023: Exception handling request to /hawkular/inventory/resources/feed-8940eaf8-3ea6-45f0-8c9c-9f88319d97b4-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:10:13,951 ERROR [io.undertow.request] (default task-294) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:10:23,246 ERROR [io.undertow.request] (default task-72) UT005023: Exception handling request to /hawkular/inventory/resources/feed-c8f14a24-5411-4135-81c8-8405b7f3f3b9-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:47,514 ERROR [io.undertow.request] (default task-177) UT005023: Exception handling request to /hawkular/inventory/resources/feed-7ebf68ef-a6c4-4587-b031-b0d4ca45f845-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:05:41,852 ERROR [io.undertow.request] (default task-211) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:47,515 ERROR [io.undertow.request] (default task-75) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: Java heap space 22:03:09,517 ERROR [io.undertow.request] (default task-268) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:49,382 ERROR [io.undertow.request] (default task-160) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:12,369 ERROR [io.undertow.request] (default task-237) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:13,289 ERROR [io.undertow.request] (default task-280) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:47,515 ERROR [io.undertow.request] (default task-169) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.80;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.80], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:44,534 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-99) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-41653637-418d-4d89-b823-c6dfc404fcc1-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:44,533 ERROR [io.undertow.request] (default task-205) UT005023: Exception handling request to /hawkular/inventory/resources/feed-dc8b083b-922c-409d-94d1-87a4718a8bf7-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:02:10,106 ERROR [io.undertow.request] (default task-363) UT005023: Exception handling request to /hawkular/inventory/resources/feed-515f1d20-c7a1-4c7d-9616-3696311623b7-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:02:31,274 ERROR [io.undertow.request] (default task-35) UT005023: Exception handling request to /hawkular/inventory/resources/feed-f386352e-58e4-44b4-9ef0-75d99fc14a48-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:44,293 ERROR [io.undertow.request] (default task-109) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:16:46,920 ERROR [org.xnio.listener] (default I/O-40) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:29,260 ERROR [io.undertow.request] (default task-378) UT005023: Exception handling request to /hawkular/inventory/resources/feed-b5e4b6e2-d53c-4fad-b25d-c65ef1158fd2-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:14,624 ERROR [io.undertow.request] (default task-167) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:20,595 ERROR [io.undertow.request] (default task-74) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: Java heap space 22:06:55,253 ERROR [io.undertow.request] (default task-261) UT005023: Exception handling request to /hawkular/inventory/resources/feed-5263a3c4-015c-40c6-90ab-1db556e219f3-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:11,054 ERROR [io.undertow.request] (default task-51) UT005023: Exception handling request to /hawkular/inventory/resources/feed-d07d7f1b-b4ae-437d-8ea1-9ea60b2cad14-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:54,392 ERROR [io.undertow.request] (default task-186) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-32dd17eb-9f5e-49aa-b854-f3d20eba7081-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:01,419 ERROR [io.undertow.request] (default task-83) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:11:54,018 ERROR [io.undertow.request] (default task-150) UT005023: Exception handling request to /hawkular/inventory/resources/feed-1f9332e0-ec29-4a3b-982e-0687ad7bbf21-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:16:40,010 ERROR [io.undertow.request] (default task-151) UT005023: Exception handling request to /hawkular/inventory/resources/feed-bca14c09-f9c8-43b7-b617-5e3ae1ab479f-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:29,029 ERROR [org.jboss.as.ejb3.invocation] (default task-184) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:16:47,626 ERROR [io.undertow.request] (default task-368) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:31,608 ERROR [org.xnio.listener] (default task-358) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:42,225 ERROR [io.undertow.request] (default task-17) UT005023: Exception handling request to /hawkular/inventory/resources/feed-a339b374-4d46-492a-a86d-b941196eeb10-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:21,628 ERROR [io.undertow.request] (default task-174) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:43,964 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-55) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-5e57e401-85a0-4d84-9877-bf6ae2bfebf9-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:16:40,010 ERROR [io.undertow.request] (default task-155) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:54,589 ERROR [io.undertow.request] (default task-36) UT005023: Exception handling request to /hawkular/inventory/resources/feed-d2506440-34c5-4389-9218-1b9deb8fbf80-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:43,400 ERROR [io.undertow.request] (default task-319) UT005023: Exception handling request to /hawkular/inventory/resources/feed-9fea767c-f9d7-44bd-b164-b34156f5e08a-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:11:37,177 ERROR [io.undertow.request] (default task-220) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:49,237 ERROR [org.apache.activemq.artemis.core.client] (Thread-16 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@1d6c5de7-649995816)) AMQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:47,249 ERROR [io.undertow.request] (default task-257) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.103;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.103], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:05:12,487 ERROR [io.undertow.request] (default task-245) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:24,624 ERROR [io.undertow.request] (default task-357) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:27,212 ERROR [io.undertow.request] (default task-312) UT005023: Exception handling request to /hawkular/inventory/resources/feed-631ae748-c44a-4259-9998-b688252449eb-NewApp.war: java.lang.OutOfMemoryError: Java heap space: failed reallocation of scalar replaced objects 22:08:42,266 ERROR [io.undertow.request] (default task-264) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:50,534 ERROR [io.undertow.request] (default task-383) UT005023: Exception handling request to /hawkular/inventory/resources/feed-80dc6289-c6b3-493d-8d42-3e4fb5d7c591-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:29,959 ERROR [io.undertow.request] (default task-377) UT005023: Exception handling request to /hawkular/inventory/resources/feed-8ac92236-ac23-4018-a269-f69e5980dec2-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:19,286 ERROR [io.undertow.request] (default task-333) UT005023: Exception handling request to /hawkular/inventory/resources/feed-eb55f32e-5eca-400c-8a7c-5fe48980cbed-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:21,816 ERROR [io.undertow.request] (default task-318) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:13:38,026 ERROR [io.undertow.request] (default task-90) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:25,345 ERROR [io.undertow.request] (default task-365) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:13:42,931 ERROR [io.undertow.request] (default task-39) UT005023: Exception handling request to /hawkular/inventory/resources/feed-964d783f-b6cb-4d0d-9876-d3f4554aea87-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:37,645 ERROR [io.undertow.request] (default task-61) UT005023: Exception handling request to /hawkular/inventory/resources/feed-4519b0f1-513e-4cf6-bb2d-7b0d85a431c6-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:46,973 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-166) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-295b578c-9fab-4e07-8913-2c7579369547-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:16,566 ERROR [io.undertow.request] (default task-41) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:11,826 ERROR [io.undertow.request] (default task-255) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:27,212 ERROR [io.undertow.request] (default task-326) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:08,425 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-102) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-5a05454b-3eea-4ff6-8184-7bd8b984e497-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:31,608 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:43,394 ERROR [io.undertow.request] (default task-367) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:12:48,268 ERROR [io.undertow.request] (default task-218) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-246e13e6-6883-4a7b-824f-dc385cc847e1-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.122;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.122], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:23,731 ERROR [io.undertow.request] (default task-132) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:05:21,216 ERROR [io.undertow.request] (default task-168) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:10,966 ERROR [io.undertow.request] (default task-252) UT005023: Exception handling request to /hawkular/inventory/resources/feed-6ae785c4-2e48-4576-a3ab-e15913fb998d-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:00,281 ERROR [io.undertow.request] (default task-225) UT005023: Exception handling request to /hawkular/inventory/resources/feed-6d8c1852-7149-4524-b04a-8d4a1a0fbb80-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:45,211 ERROR [io.undertow.request] (default task-22) UT005023: Exception handling request to /hawkular/inventory/resources/feed-dbd1333e-786b-4051-9459-bfa36b689506-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:44,192 ERROR [io.undertow.request] (default task-123) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:07:10,323 ERROR [io.undertow.request] (default task-50) UT005023: Exception handling request to /hawkular/inventory/resources/feed-3d826948-fc25-4285-8a73-6f8c044b92ad-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:29,664 ERROR [io.undertow.request] (default task-207) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.117;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.117], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:09,189 ERROR [io.undertow.request] (default task-291) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:14,065 ERROR [io.undertow.request] (default task-23) UT005023: Exception handling request to /hawkular/inventory/resources/feed-0548a8f9-125b-4223-9d4e-9e6fef0d623a-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:38,314 ERROR [io.undertow.request] (default task-34) UT005023: Exception handling request to /hawkular/inventory/resources/feed-f7f489c2-43d9-480c-8829-673f22093841-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:30,612 ERROR [io.undertow.request] (default task-228) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:07:32,668 ERROR [io.undertow.request] (default task-158) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:09,667 ERROR [io.undertow.request] (default task-93) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-7ca72196-45f8-4bb4-942b-1fa6cbdbf126-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.117;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.117], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:24,746 ERROR [io.undertow.request] (default task-79) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:29,045 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-134) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-41006cf5-939a-4e9d-ba33-f812bf33e333-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:11,661 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-143) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-c95e87fb-669c-4814-8b3a-8a54fe5580fb-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:22,507 ERROR [io.undertow.request] (default task-361) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:35:11,817 ERROR [io.undertow.request] (default task-197) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:35:10,728 ERROR [io.undertow.request] (default task-61) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:24,624 ERROR [io.undertow.request] (default task-372) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:35:15,529 ERROR [org.jboss.as.ejb3.invocation] (default task-166) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:32:25,547 ERROR [org.xnio.listener] (default I/O-25) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:31:15,654 ERROR [org.xnio.listener] (default I/O-7) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:32:42,136 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-199) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-20, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-9, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-22, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-7, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-21, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-8, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-24, feed-12c038bb-5cc0-4ea5-a3e4-ac5027ac877c-Server-0-child-5...<92 other elements>]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:29:33,622 ERROR [io.undertow.request] (default task-107) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Connection=[keep-alive], Content-Type=[text/html;charset=UTF-8], Content-Length=[80], Date=[Thu, 26 Oct 2017 22:28:47 GMT]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:35:24,558 ERROR [io.undertow.request] (default task-41) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:35:12,750 ERROR [io.undertow.request] (default task-326) UT005023: Exception handling request to /hawkular/inventory/resources/feed-3d69cf00-f3a9-4ba7-a48c-8b62f5e458be-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:34:03,883 ERROR [io.undertow.request] (default task-191) UT005023: Exception handling request to /hawkular/inventory/resources/feed-9b1ebbbb-5852-4460-a96d-dd89b73ff728-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:32:48,982 ERROR [org.xnio.listener] (default I/O-9) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:20,270 ERROR [io.undertow.request] (default task-21) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-f7ec64b8-15dd-49aa-8796-a03806a3a12b-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.108;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.108], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:21,400 ERROR [io.undertow.request] (default task-301) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:37:07,139 ERROR [io.undertow.request] (default task-154) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[271719], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:36:33,801 ERROR [io.undertow.request] (default task-228) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:30:53,700 WARN [org.hawkular.agent.monitor.cmd.FeedCommProcessor] (OkHttp https://127.0.0.1:8443/...) HAWKAGENT010036: Feed communications channel encountered a failure. Response=[]: javax.net.ssl.SSLException: Unsupported record version Unknown-104.97 at sun.security.ssl.InputRecord.checkRecordVersion(InputRecord.java:552) at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:565) at sun.security.ssl.InputRecord.read(InputRecord.java:532) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940) at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) at okio.shaded.Okio$2.read(Okio.java:138) at okio.shaded.AsyncTimeout$2.read(AsyncTimeout.java:238) at okio.shaded.RealBufferedSource.request(RealBufferedSource.java:66) at okio.shaded.RealBufferedSource.require(RealBufferedSource.java:59) at okio.shaded.RealBufferedSource.readByte(RealBufferedSource.java:72) at okhttp3.shaded.internal.ws.WebSocketReader.readHeader(WebSocketReader.java:115) at okhttp3.shaded.internal.ws.WebSocketReader.processNextFrame(WebSocketReader.java:104) at okhttp3.shaded.internal.ws.RealWebSocket.readMessage(RealWebSocket.java:97) at okhttp3.shaded.ws.WebSocketCall.createWebSocket(WebSocketCall.java:152) at okhttp3.shaded.ws.WebSocketCall.access$000(WebSocketCall.java:41) at okhttp3.shaded.ws.WebSocketCall$1.onResponse(WebSocketCall.java:97) at okhttp3.shaded.RealCall$AsyncCall.execute(RealCall.java:126) at okhttp3.shaded.internal.NamedRunnable.run(NamedRunnable.java:32) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:38:15,517 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-199) ISPN000255: Error while processing prepare: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:30,172 ERROR [io.undertow.request] (default task-87) UT005023: Exception handling request to /hawkular/inventory/resources/feed-16886c82-8720-4e1d-8333-5c777bab6668-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:11:45,991 ERROR [io.undertow.request] (default task-157) UT005023: Exception handling request to /hawkular/inventory/resources/feed-b6230112-4322-4d42-b6f6-7a43b47a644f-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:57,828 ERROR [org.hawkular.agent.monitor.protocol.Discovery] (Hawkular WildFly Agent Full Discovery Scan-Local-2) HAWKAGENT010061: Failed to discover resources in [Endpoint[Local]:[https-remoting://127.0.0.1:9993]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching nodes for query [/] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:224) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:55) at org.hawkular.agent.monitor.protocol.Discovery.discoverChildren(Discovery.java:75) at org.hawkular.agent.monitor.protocol.EndpointService.discoverChildren(EndpointService.java:450) at org.hawkular.agent.monitor.protocol.EndpointService.access$900(EndpointService.java:83) at org.hawkular.agent.monitor.protocol.EndpointService$1.run(EndpointService.java:397) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:221) ... 9 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 10 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 12 more 22:33:53,890 ERROR [io.undertow.request] (default task-368) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:53,891 ERROR [io.undertow.request] (default task-325) UT005023: Exception handling request to /hawkular/inventory/resources/feed-967a5820-0193-4b7e-8494-e0c1dafc46df-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:34:01,865 ERROR [io.undertow.request] (default task-169) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,520 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular WildFly Agent Full Discovery Scan-Local-2) HAWKAGENT010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[https-remoting://127.0.0.1:9993]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching nodes for query [/] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:224) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:55) at org.hawkular.agent.monitor.protocol.Discovery.discoverChildren(Discovery.java:75) at org.hawkular.agent.monitor.protocol.EndpointService.discoverChildren(EndpointService.java:450) at org.hawkular.agent.monitor.protocol.EndpointService.access$900(EndpointService.java:83) at org.hawkular.agent.monitor.protocol.EndpointService$1.run(EndpointService.java:397) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:221) ... 9 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 10 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 12 more 22:38:14,163 ERROR [io.undertow.request] (default task-412) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.122;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.122], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:55,407 ERROR [io.undertow.request] (default task-211) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:55,407 ERROR [io.undertow.request] (default task-428) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:37:29,756 ERROR [io.undertow.request] (default task-56) UT005023: Exception handling request to /hawkular/inventory/resources/feed-c3a869b1-5ca6-41fa-9f03-3d42e2054661-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:32:48,262 ERROR [io.undertow.request] (default task-10) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-2ce6c102-6799-4a4e-9cb7-30233b1fe0c6-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:27,016 ERROR [io.undertow.request] (default task-189) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[271719], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:37:53,848 ERROR [io.undertow.request] (default task-426) UT005023: Exception handling request to /hawkular/inventory/resources/feed-3ff1a8ee-2dcf-4d8b-adf0-75a75b1aa795-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:31:01,376 ERROR [org.xnio.listener] (default I/O-47) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:30,947 ERROR [org.xnio.listener] (default I/O-17) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:51,408 ERROR [io.undertow.request] (default task-210) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:51,409 ERROR [org.xnio.listener] (default I/O-39) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,379 ERROR [io.undertow.request] (default task-34) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:51,408 ERROR [io.undertow.request] (default task-257) UT005023: Exception handling request to /hawkular/inventory/resources/feed-9de3d56b-8da2-4e63-a68c-4dc529101da7-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:51,408 ERROR [org.xnio.listener] (default I/O-41) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:37:29,162 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-396) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [JDG, EAP]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key JDG and requestor GlobalTx:local:60566. Lock is held by GlobalTx:local:60126 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$CompositeLockPromise.lock(DefaultLockManager.java:321) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResourceType(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:236) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:31:37,786 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-390) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-6eab1c86-b4ce-4e80-8761-3ede30834242-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:31:13,574 ERROR [org.xnio.listener] (default I/O-23) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:37:55,730 ERROR [io.undertow.request] (default task-420) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Date=[Thu, 26 Oct 2017 22:28:47 GMT], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,382 ERROR [io.undertow.request] (default task-436) UT005023: Exception handling request to /hawkular/inventory/resources/feed-9fb896ad-1cea-4641-a431-77f52d43374e-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,536 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-390) ISPN000255: Error while processing prepare: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,536 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-396) ISPN000255: Error while processing prepare: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key JDG and requestor GlobalTx:local:60566. Lock is held by GlobalTx:local:60126 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResourceType(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:236) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key JDG and requestor GlobalTx:local:60566. Lock is held by GlobalTx:local:60126 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$CompositeLockPromise.lock(DefaultLockManager.java:321) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:14:21,527 ERROR [io.undertow.request] (default task-104) UT005071: Undertow request failed HttpServerExchange{ HEAD /hawkular/alerts/status request {Accept=[*/*], User-Agent=[curl/7.29.0], Host=[hawkular-services-2-7z0xc:8080]} response {X-Powered-By=[Undertow/1], Server=[WildFly/10]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:33:00,945 ERROR [io.undertow.request] (default task-38) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[271719], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,537 WARN [com.arjuna.ats.arjuna] (default task-199) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:305f5, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@ed19} org.infinispan.transaction.synchronization.SynchronizationAdapter@ed38 >: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:38:15,539 WARN [com.arjuna.ats.arjuna] (default task-390) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:3060c, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@ed42} org.infinispan.transaction.synchronization.SynchronizationAdapter@ed61 >: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) 22:20:27,887 ERROR [org.xnio.listener] (default I/O-11) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:26:55,247 ERROR [io.undertow.request] (default task-156) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:04,669 ERROR [org.jboss.as.ejb3.invocation] (default task-40) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:31,167 ERROR [io.undertow.request] (default task-54) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:28:10,688 ERROR [io.undertow.request] (default task-130) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-58b6e003-f48e-4c59-a96d-40e538090265-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.76;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.76], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:28:09,358 ERROR [io.undertow.request] (default task-9) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:21:37,285 ERROR [io.undertow.request] (default task-219) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:12,665 ERROR [io.undertow.request] (default task-271) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:28:07,858 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-349) ISPN000136: Error executing command PutMapCommand, writing keys [feed-81caf8f0-03fc-4801-a5d8-725db047fd9c-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:14,644 ERROR [io.undertow.request] (default task-179) UT005023: Exception handling request to /hawkular/inventory/resources/feed-283097ef-e841-4c66-b971-a2b974c6da76-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:12,123 ERROR [io.undertow.request] (default task-28) UT005023: Exception handling request to /hawkular/inventory/resources/feed-530295bf-9280-4cf9-87b8-25790d67744a-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:52,712 WARN [com.arjuna.ats.arjuna] (default task-284) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30086, org.hibernate.search.backend.impl.PostTransactionWorkQueueSynchronization@73818dd >: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:28:02,800 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-142) ISPN000136: Error executing command PutMapCommand, writing keys [feed-8ef9c7cc-fe72-41b4-8820-66e8c15a3005-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:28:03,951 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-221) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-c1b94475-9405-4283-8816-a3406e218975-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:27:32,373 ERROR [org.jboss.as.ejb3.invocation] (default task-303) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:27:44,672 WARN [org.hawkular.agent.monitor.scheduler.MetricsCollector] (Hawkular-WildFly-Agent-Scheduler-Metrics-2) Unexpected error caught in MetricsCollector for endpoint [PlatformEndpointService[Endpoint[platform]:[null]]]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:23,229 ERROR [org.jboss.as.ejb3.invocation] (default task-117) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): javax.ejb.EJBException: WFLYEJB0442: Unexpected Error at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:184) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:277) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:46,038 ERROR [io.undertow.request] (default task-286) UT005023: Exception handling request to /hawkular/inventory/resources/feed-ff2daa16-f709-4dd0-88cb-8fbe663037b5-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:46,038 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-203) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-718e370d-0d29-445d-817c-aadb091283d6-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:59,852 ERROR [org.jboss.as.ejb3.invocation] (default task-88) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:39,180 ERROR [io.undertow.request] (default task-281) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:07:20,623 ERROR [io.undertow.request] (default task-370) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:47,989 ERROR [io.undertow.request] (default task-76) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:48,715 ERROR [io.undertow.request] (default task-360) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:47,253 ERROR [io.undertow.request] (default task-246) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:47,253 ERROR [io.undertow.request] (default task-73) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:48,715 ERROR [io.undertow.request] (default task-33) UT005023: Exception handling request to /hawkular/inventory/resources/feed-946addc0-ddb8-46b8-905e-0a3ed2f2f340-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:47,253 ERROR [io.undertow.request] (default task-64) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:17:34,679 ERROR [io.undertow.request] (default task-380) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-944dcc41-4435-4a15-a158-44fe35abe2e4-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:51,527 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-362) ISPN000136: Error executing command PutMapCommand, writing keys [feed-6be0321e-1eaa-401b-b00f-1b364c8ae395-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:52,702 ERROR [io.undertow.request] (default task-3) UT005023: Exception handling request to /hawkular/inventory/resources/feed-9c27166a-2797-48be-a22d-ee6d6ab5d20c-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:52,702 ERROR [io.undertow.request] (default task-147) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-65bff9de-298c-48b1-b45c-1d59469c0507-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.94;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.94], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:29,158 ERROR [org.jboss.remoting.resource] (Hawkular-WildFly-Agent-Scheduler-Avail-3) Close action for Remoting remote connection provider 19d713ad for endpoint "management-client" <78c2f706> failed to execute (resource may be left in an indeterminate state): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:18:42,143 WARNING [io.netty.util.HashedWheelTimer] (cluster2-timeouter-0) An exception was thrown by TimerTask.: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:07:10,700 ERROR [io.undertow.request] (default task-300) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-619b0ef7-1378-45ed-8c0f-b461d5a3bf96-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Date=[Thu, 26 Oct 2017 21:59:14 GMT], Connection=[close], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:39:47,793 ERROR [io.undertow.request] (default task-316) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:11:27,625 ERROR [io.undertow.request] (default task-162) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-a21ab1c8-d4dd-4dca-88b3-38e477178ae3-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:51,382 ERROR [io.undertow.request] (default task-173) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.147;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.147], Content-type=[application/json], Content-Length=[271719], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:26,360 ERROR [io.undertow.request] (default task-202) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:20,440 ERROR [io.undertow.request] (default task-2) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:18:27,207 ERROR [io.undertow.request] (default task-224) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:41,231 ERROR [io.undertow.request] (default task-267) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:12,987 ERROR [io.undertow.request] (default task-16) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:21:13,049 ERROR [io.undertow.request] (default task-244) UT005023: Exception handling request to /hawkular/inventory/resources/feed-406f5450-8379-4680-8d60-650a29295d72-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:07:58,906 ERROR [io.undertow.request] (default task-251) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:34,533 ERROR [io.undertow.request] (default task-366) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-56be4603-b781-4c35-80d0-972da26abd69-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.141;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.141], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:18:11,394 ERROR [io.undertow.request] (default task-194) UT005023: Exception handling request to /hawkular/inventory/resources/feed-ddf7f2c7-9dd5-4712-906b-5a17630e767b-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:22,805 WARN [org.hibernate.search.backend.impl.lucene.IndexWriterHolder] (Hibernate Search sync consumer thread for index resource.5) HSEARCH000052: Going to force release of the IndexWriter lock 22:20:47,346 ERROR [io.undertow.request] (default task-305) UT005023: Exception handling request to /hawkular/inventory/resources/feed-b9d9bab6-d9c4-40eb-b870-76ff72af60e5-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:51,651 ERROR [io.undertow.request] (default task-317) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:49,522 ERROR [org.apache.activemq.artemis.core.client] (Thread-23 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@1d6c5de7-649995816)) AMQ214017: Caught unexpected Throwable: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:52,916 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-353) ISPN000136: Error executing command VersionedPrepareCommand, writing keys []: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:22,805 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-95) ISPN000136: Error executing command RemoveCommand, writing keys [feed-203352e1-c559-4cc3-a2a6-e107af130c03-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:21:11,903 ERROR [io.undertow.request] (default task-139) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.104;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.104], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:46,874 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-67) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-34ed8615-6b76-46c1-b8aa-1344c547c1fa-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:09:25,886 ERROR [io.undertow.request] (default task-116) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-6a09a488-e5f5-491f-9e02-10625cb121b7-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:11,721 ERROR [io.undertow.request] (default task-346) UT005023: Exception handling request to /hawkular/inventory/resources/feed-c84c1e26-b8db-478b-8787-099cf056a8bc-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:48,281 ERROR [io.undertow.request] (default task-101) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:11:52,493 WARNING [io.netty.channel.epoll.EpollEventLoop] (cluster1-nio-worker-0) Unexpected exception in the selector loop.: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:51,645 WARN [org.jboss.weld.Servlet] (default task-188) WELD-000717: Unable to deactivate context org.jboss.weld.context.http.HttpRequestContextImpl@579ea20f when destroying request HttpServletRequestImpl [ POST /hawkular/inventory/import ] 22:21:40,547 ERROR [io.undertow.request] (default task-69) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:12:53,196 ERROR [io.undertow.request] (default task-78) UT005023: Exception handling request to /hawkular/inventory/resources/feed-e05b9ede-0dc5-4eb6-8603-2b5595ed27f2-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:10,642 ERROR [org.jboss.as.ejb3.invocation] (default task-131) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:15,687 ERROR [io.undertow.request] (default task-240) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-fe3e7ed6-8383-4e2b-8e47-9f826dfc3d52-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:24:35,486 ERROR [io.undertow.request] (default task-112) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:18:03,161 ERROR [io.undertow.request] (default task-71) UT005023: Exception handling request to /hawkular/inventory/resources/feed-464b542d-9cca-4de3-92dc-16ed9888d2fc-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:32,337 ERROR [io.undertow.request] (default task-124) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:24:38,518 ERROR [io.undertow.request] (default task-335) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:38,518 ERROR [io.undertow.request] (default task-308) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-5484dadd-b347-4590-80a0-56b609b1361b-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:49,441 ERROR [io.undertow.request] (default task-374) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-0041ea7d-724b-449b-8c48-30a6e9ab1dd7-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:24:41,107 ERROR [io.undertow.request] (default task-398) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:03,489 ERROR [io.undertow.request] (default task-243) UT005023: Exception handling request to /hawkular/inventory/resources/feed-1fb99241-3f3f-4e6b-9ba9-5f39f92075e6-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:20,640 WARN [org.jboss.modules] (default task-63) Failed to define class io.undertow.servlet.spec.ServletPrintWriter in Module "io.undertow.servlet:main" from local module loader @436e852b (finder: local module finder @32d2fa64 (roots: /opt/jboss/wildfly/modules,/opt/jboss/wildfly/modules/system/layers/hawkular,/opt/jboss/wildfly/modules/system/layers/base)): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:27:48,020 ERROR [io.undertow.request] (default task-212) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-c1fd5dbf-7f2f-4ca9-943c-f2b7f38ac79f-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:27:54,238 ERROR [org.xnio.listener] (default I/O-35) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:16,699 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-265) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-31299912-b4e8-40d1-915f-d835a17bc20a-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:24:42,548 ERROR [io.undertow.request] (default task-138) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:12,023 ERROR [io.undertow.request] (default task-100) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:07,747 ERROR [io.undertow.request] (default task-141) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:07,248 ERROR [io.undertow.request] (default task-375) UT005023: Exception handling request to /hawkular/inventory/resources/feed-1b8a1fa8-c1dd-4e8a-84d9-2251a2d0682b-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:10,059 ERROR [io.undertow.request] (default task-14) UT005023: Exception handling request to /hawkular/inventory/resources/feed-234c7d88-265f-4d6b-882d-ab33ed266de0-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:01,445 ERROR [io.undertow.request] (default task-65) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-9ba360da-b2cb-4cea-80d7-239f76575c84-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:23:18,800 ERROR [io.undertow.request] (default task-20) UT005023: Exception handling request to /hawkular/metrics/metrics/data: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:25:09,489 ERROR [io.undertow.request] (default task-129) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:49:23,779 WARN [com.arjuna.ats.arjuna] (default task-202) ARJUNA012129: TwoPhaseCoordinator.afterCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30b78, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[feed-b41a35b9-d5d7-4f73-b671-6656f621d7f9-NewApp.war], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@ef05} org.infinispan.transaction.synchronization.SynchronizationAdapter@ef24 > with error: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:49:39,140 ERROR [io.undertow.request] (default task-16) UT005023: Exception handling request to /hawkular/inventory/resources/feed-af0f2b37-fd1a-4551-bd6c-4d66a67bad44-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:49:50,263 ERROR [io.undertow.request] (default task-71) UT005023: Exception handling request to /hawkular/inventory/resources/feed-d96d82f6-ea71-493c-9a1b-40edd0c22c0b-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:27,402 ERROR [org.jboss.as.ejb3.invocation] (default task-110) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:15:05,948 ERROR [io.undertow.request] (default task-135) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:20:37,070 ERROR [io.undertow.request] (default task-336) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:18:03,161 ERROR [io.undertow.request] (default task-381) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:14:49,877 ERROR [io.undertow.request] (default task-254) UT005023: Exception handling request to /hawkular/inventory/resources/feed-d608a014-06b7-4be0-a023-40ede6fe1567-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:21:10,373 ERROR [io.undertow.request] (default task-146) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Type=[application/json], Content-Length=[0]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:17:51,551 ERROR [io.undertow.request] (default task-176) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:08:37,582 ERROR [io.undertow.request] (default task-121) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:26,958 ERROR [org.xnio.listener] (default I/O-2) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:51,654 ERROR [io.undertow.request] (default task-31) UT005023: Exception handling request to /hawkular/inventory/resources/feed-8c958d12-b6ce-4d4b-a1e0-5bf99174b44e-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:10:53,319 ERROR [io.undertow.request] (default task-338) UT005023: Exception handling request to /hawkular/inventory/resources/feed-6eec55f6-cf8e-40d4-8134-196c21240539-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:06:53,014 WARNING [io.netty.channel.epoll.EpollEventLoop] (cluster2-nio-worker-0) Unexpected exception in the selector loop.: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:51,645 ERROR [org.hawkular.agent.monitor.storage.HawkularStorageAdapter] (Hawkular-WildFly-Scheduler-Ping-1) HAWKAGENT010011: Failed to store avail data: ?: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:13,169 ERROR [io.undertow.request] (default task-321) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:50,075 ERROR [org.jboss.as.ejb3.invocation] (default task-44) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:17:14,871 ERROR [io.undertow.request] (default task-161) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:03:50,409 ERROR [io.undertow.request] (default task-299) UT005023: Exception handling request to /hawkular/inventory/resources/feed-128198aa-b63c-4580-8a52-fb8cda81f026-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:04:06,192 ERROR [io.undertow.request] (default task-379) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:48,572 ERROR [io.undertow.request] (default task-250) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:07,081 ERROR [io.undertow.request] (default task-309) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-02a3d84b-98fe-4e23-bd39-3809088446f0-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:43,857 WARN [org.hawkular.agent.monitor.protocol.ProtocolServices] (Hawkular WildFly Agent Auto-Discovery Scan-1) HAWKAGENT010058: Auto-discovery scan failed: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:19:25,940 ERROR [io.undertow.request] (default task-133) UT005023: Exception handling request to /hawkular/inventory/resources/feed-a3d35041-7bb1-409f-9928-6a881dc8ef8b-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:11,878 ERROR [org.jboss.resteasy.resteasy_jaxrs.i18n] (default task-347) RESTEASY002005: Failed executing POST /import: org.jboss.resteasy.spi.ReaderException: com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class org.hawkular.inventory.api.model.Metric] value failed (java.lang.OutOfMemoryError): GC overhead limit exceeded at [Source: io.undertow.servlet.spec.ServletInputStreamImpl@179fa13e; line: 1, column: 27625] (through reference chain: org.hawkular.inventory.api.model.Inventory["resources"]->java.util.ArrayList[10]->org.hawkular.inventory.api.model.RawResource["metrics"]->java.util.ArrayList[3]) at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:184) at org.jboss.resteasy.core.MethodInjectorImpl.injectArguments(MethodInjectorImpl.java:91) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:114) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: com.fasterxml.jackson.databind.JsonMappingException: Instantiation of [simple type, class org.hawkular.inventory.api.model.Metric] value failed (java.lang.OutOfMemoryError): GC overhead limit exceeded at [Source: io.undertow.servlet.spec.ServletInputStreamImpl@179fa13e; line: 1, column: 27625] (through reference chain: org.hawkular.inventory.api.model.Inventory["resources"]->java.util.ArrayList[10]->org.hawkular.inventory.api.model.RawResource["metrics"]->java.util.ArrayList[3]) at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:262) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.wrapAsJsonMappingException(StdValueInstantiator.java:445) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.rewrapCtorProblem(StdValueInstantiator.java:464) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:258) at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:385) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1123) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:277) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:249) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:465) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:380) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1123) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:277) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:249) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:465) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:380) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1123) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133) at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:1535) at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:944) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.readFrom(ResteasyJackson2Provider.java:121) at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.readFrom(AbstractReaderInterceptorContext.java:61) at org.jboss.resteasy.core.interception.ServerReaderInterceptorContext.readFrom(ServerReaderInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:53) at org.jboss.resteasy.security.doseta.DigitalVerificationInterceptor.aroundReadFrom(DigitalVerificationInterceptor.java:34) at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:55) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPDecodingInterceptor.aroundReadFrom(GZIPDecodingInterceptor.java:59) at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.proceed(AbstractReaderInterceptorContext.java:55) at org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:151) ... 51 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at sun.reflect.GeneratedConstructorAccessor172.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at com.fasterxml.jackson.databind.introspect.AnnotatedConstructor.call(AnnotatedConstructor.java:114) at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromObjectWith(StdValueInstantiator.java:256) at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:135) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:385) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1123) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:277) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:249) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:465) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:380) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1123) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:277) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:249) at com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:26) at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:465) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:380) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1123) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:298) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:133) at com.fasterxml.jackson.databind.ObjectReader._bind(ObjectReader.java:1535) at com.fasterxml.jackson.databind.ObjectReader.readValue(ObjectReader.java:944) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.readFrom(ResteasyJackson2Provider.java:121) at org.jboss.resteasy.core.interception.AbstractReaderInterceptorContext.readFrom(AbstractReaderInterceptorContext.java:61) 22:22:13,179 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-348) ISPN000136: Error executing command PutMapCommand, writing keys [feed-039ec64a-6eb1-4ab5-a741-96c850789560-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:05:21,801 INFO [org.apache.activemq.artemis.ra] (default-threads - 4) AMQ151000: awaiting topic/queue creation HawkularCommandEvent 22:22:30,176 ERROR [io.undertow.request] (default task-45) UT005023: Exception handling request to /hawkular/inventory/resources/feed-b89b9977-60fd-4a6d-9254-d858f24ad096-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:09,409 ERROR [io.undertow.request] (default task-140) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:22:43,095 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-153) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-d7cb1656-2240-44ee-8f33-f960755a82ce-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:33,119 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-156) ISPN000136: Error executing command VersionedCommitCommand, writing keys [feed-b316eb29-2e28-44e7-a936-dba2692e65e1-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:42:50,248 ERROR [org.xnio.listener] (default I/O-11) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:51,922 ERROR [io.undertow.request] (default task-76) UT005023: Exception handling request to /hawkular/inventory/resources/feed-34b96c20-837d-400d-9cf6-181b240642af-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:21:10,373 ERROR [io.undertow.request] (default task-248) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:20,074 ERROR [org.xnio.listener] (default I/O-36) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:56,868 WARN [com.arjuna.ats.arjuna] (default task-67) ARJUNA012129: TwoPhaseCoordinator.afterCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:2ffc1, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@eb59} org.infinispan.transaction.synchronization.SynchronizationAdapter@eb78 > with error: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:49:10,958 ERROR [org.jboss.as.ejb3.invocation] (default task-116) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:45,939 ERROR [org.jboss.as.ejb3.invocation] (default task-74) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:16:48,373 ERROR [io.undertow.request] (default task-258) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:23,986 ERROR [org.xnio.listener] (default I/O-9) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:24,035 ERROR [org.xnio.listener] (default I/O-37) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:44,594 ERROR [org.jboss.as.ejb3.invocation] (default task-310) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:12,750 ERROR [io.undertow.request] (default task-267) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Content-Type=[text/html;charset=UTF-8]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:53:29,093 ERROR [org.xnio.listener] (default I/O-26) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:53:13,040 ERROR [io.undertow.request] (default task-224) UT005023: Exception handling request to /hawkular/inventory/resources/feed-3d511407-f7cd-4b5e-9a40-57c2caa928f7-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:11,515 ERROR [org.xnio.listener] (default I/O-16) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:42:36,906 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-238) ISPN000255: Error while processing prepare: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:263) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:42:24,082 ERROR [org.jboss.as.ejb3.invocation] (default task-285) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:45,839 ERROR [org.xnio.listener] (default I/O-39) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:45,078 WARN [org.hawkular.metrics.scheduler.impl.LockManager] (locks-thread-pool-0) There was an error trying to renew locks: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:53:09,258 ERROR [io.undertow.request] (default task-71) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:52:09,722 ERROR [io.undertow.request] (default task-442) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:24,028 ERROR [io.undertow.request] (default task-281) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {X-Powered-By=[Undertow/1], Server=[WildFly/10]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:53,959 ERROR [org.xnio.listener] (default task-63) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:22,264 WARNING [io.netty.util.HashedWheelTimer] (cluster1-timeouter-0) An exception was thrown by TimerTask.: java.lang.OutOfMemoryError: Java heap space 22:48:42,610 ERROR [io.undertow.request] (default task-332) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:33,983 ERROR [org.jboss.as.ejb3.invocation] (default task-362) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBException: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleExceptionInOurTx(CMTTxInterceptor.java:187) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:277) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679) at org.infinispan.cache.impl.CacheImpl.putAll(CacheImpl.java:1405) at org.infinispan.cache.impl.DecoratedCache.putAll(DecoratedCache.java:490) at org.infinispan.cache.impl.DecoratedCache.putAll(DecoratedCache.java:495) at org.infinispan.cache.impl.AbstractDelegatingCache.putAll(AbstractDelegatingCache.java:368) at org.infinispan.cache.impl.EncoderCache.putAll(EncoderCache.java:670) at org.hawkular.inventory.service.InventoryServiceIspn.addResource(InventoryServiceIspn.java:121) at sun.reflect.GeneratedMethodAccessor162.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:82) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64) at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:275) ... 90 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:43,275 ERROR [org.xnio.listener] (default I/O-38) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:55:59,794 ERROR [io.undertow.request] (default task-310) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:174) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:478) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:422) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:162) at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:247) at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:588) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.write(AbstractFixedLengthStreamSinkConduit.java:106) at org.xnio.conduits.Conduits.writeFinalBasic(Conduits.java:132) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.writeFinal(AbstractFixedLengthStreamSinkConduit.java:175) at org.xnio.conduits.ConduitStreamSinkChannel.writeFinal(ConduitStreamSinkChannel.java:104) at io.undertow.channels.DetachableStreamSinkChannel.writeFinal(DetachableStreamSinkChannel.java:195) at io.undertow.server.HttpServerExchange$WriteDispatchChannel.writeFinal(HttpServerExchange.java:2007) at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:561) at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:596) at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.close(HttpServletResponseWrapper.java:58) at org.jboss.resteasy.util.CommitHeaderOutputStream.close(CommitHeaderOutputStream.java:87) at org.jboss.resteasy.util.DelegatingOutputStream.close(DelegatingOutputStream.java:60) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1036) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.writeTo(ResteasyJackson2Provider.java:209) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131) at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120) at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:473) ... 46 more 22:55:59,796 ERROR [org.jboss.as.txn] (default task-310) WFLYTX0003: APPLICATION ERROR: transaction still active in request with status 0 22:43:37,388 ERROR [org.xnio.listener] (default I/O-4) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:43,515 ERROR [io.undertow.request] (default task-101) UT005023: Exception handling request to /hawkular/inventory/resources/feed-01989340-914a-4e35-aae7-831dd4a725fb-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:32,397 ERROR [io.undertow.request] (default task-451) UT005023: Exception handling request to /hawkular/inventory/resources/feed-0d68c937-d719-43e8-a8f0-8292afce13ee-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:55:59,795 ERROR [io.undertow.request] (default task-285) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:174) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:478) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:422) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:162) at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:247) at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:588) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.write(AbstractFixedLengthStreamSinkConduit.java:106) at org.xnio.conduits.Conduits.writeFinalBasic(Conduits.java:132) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.writeFinal(AbstractFixedLengthStreamSinkConduit.java:175) at org.xnio.conduits.ConduitStreamSinkChannel.writeFinal(ConduitStreamSinkChannel.java:104) at io.undertow.channels.DetachableStreamSinkChannel.writeFinal(DetachableStreamSinkChannel.java:195) at io.undertow.server.HttpServerExchange$WriteDispatchChannel.writeFinal(HttpServerExchange.java:2007) at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:561) at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:596) at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.close(HttpServletResponseWrapper.java:58) at org.jboss.resteasy.util.CommitHeaderOutputStream.close(CommitHeaderOutputStream.java:87) at org.jboss.resteasy.util.DelegatingOutputStream.close(DelegatingOutputStream.java:60) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1036) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.writeTo(ResteasyJackson2Provider.java:209) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131) at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120) at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:473) ... 46 more 22:55:59,796 ERROR [org.jboss.as.txn] (default task-285) WFLYTX0003: APPLICATION ERROR: transaction still active in request with status 0 22:51:42,439 ERROR [org.hawkular.agent.monitor.storage.AsyncInventoryStorage] (Hawkular WildFly Agent Full Discovery Scan-platform-1) HAWKAGENT010024: Failed to store inventory data: java.net.SocketTimeoutException: Read timed out at java.net.SocketInputStream.socketRead0(Native Method) at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) at java.net.SocketInputStream.read(SocketInputStream.java:171) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397) at okhttp3.shaded.internal.connection.RealConnection.connectTls(RealConnection.java:241) at okhttp3.shaded.internal.connection.RealConnection.establishProtocol(RealConnection.java:198) at okhttp3.shaded.internal.connection.RealConnection.buildConnection(RealConnection.java:174) at okhttp3.shaded.internal.connection.RealConnection.connect(RealConnection.java:114) at okhttp3.shaded.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:193) at okhttp3.shaded.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:129) at okhttp3.shaded.internal.connection.StreamAllocation.newStream(StreamAllocation.java:98) at okhttp3.shaded.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.shaded.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:109) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.shaded.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.shaded.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:124) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92) at okhttp3.shaded.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67) at okhttp3.shaded.RealCall.getResponseWithInterceptorChain(RealCall.java:170) at okhttp3.shaded.RealCall.execute(RealCall.java:60) at org.hawkular.agent.monitor.storage.AsyncInventoryStorage.sendImportRestRequest(AsyncInventoryStorage.java:252) at org.hawkular.agent.monitor.storage.AsyncInventoryStorage.importInventoryData(AsyncInventoryStorage.java:215) at org.hawkular.agent.monitor.storage.AsyncInventoryStorage.receivedEvent(AsyncInventoryStorage.java:160) at org.hawkular.agent.monitor.storage.HawkularStorageAdapter.receivedEvent(HawkularStorageAdapter.java:396) at org.hawkular.agent.monitor.storage.InventoryStorageProxy.receivedEvent(InventoryStorageProxy.java:41) at org.hawkular.agent.monitor.protocol.EndpointService$InventoryListenerSupport.fireDiscoveryComplete(EndpointService.java:119) at org.hawkular.agent.monitor.protocol.EndpointService$DiscoveryResults.discoveryFinished(EndpointService.java:203) at org.hawkular.agent.monitor.protocol.EndpointService$1.run(EndpointService.java:408) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) 22:48:29,135 ERROR [org.xnio.listener] (default I/O-23) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:47:53,073 ERROR [io.undertow.request] (default task-223) UT005023: Exception handling request to /hawkular/inventory/resources/feed-fa329353-8b33-4d80-9e91-a5414a48e9cd-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:05,272 ERROR [io.undertow.request] (default task-452) UT005023: Exception handling request to /hawkular/inventory/resources/feed-3241d44b-1527-4f92-b59f-9556efdb73f9-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:40,178 ERROR [io.undertow.request] (default task-444) UT005023: Exception handling request to /hawkular/inventory/resources/feed-4f911609-f9ee-4dc6-993e-2146f3c18c11-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:35,019 ERROR [io.undertow.request] (default task-450) UT005023: Exception handling request to /hawkular/inventory/resources/feed-b56b6a17-e473-4155-98ec-0d6edbae31bf-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:42,609 ERROR [org.xnio.listener] (default I/O-3) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:47:44,646 ERROR [org.xnio.listener] (default I/O-41) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:33,568 ERROR [io.undertow.request] (default task-380) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:47:45,714 ERROR [org.jboss.as.ejb3.invocation] (default task-324) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): java.lang.OutOfMemoryError: GC overhead limit exceeded 22:47:31,488 ERROR [io.undertow.request] (default task-64) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-40fdb59a-79e9-4659-b23e-c6f77dd13ba8-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {X-Powered-By=[Undertow/1], Server=[WildFly/10]}}: java.lang.RuntimeException: WFLYUT0052: Security context creation failed at org.wildfly.extension.undertow.security.SecurityActions.createSecurityContext(SecurityActions.java:74) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.setup(SecurityContextThreadSetupAction.java:87) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:42) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.GeneratedConstructorAccessor117.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.jboss.security.SecurityContextFactory.createSecurityContext(SecurityContextFactory.java:199) at org.jboss.security.SecurityContextFactory.createSecurityContext(SecurityContextFactory.java:108) at org.wildfly.extension.undertow.security.SecurityActions.createSecurityContext(SecurityActions.java:72) ... 14 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:48:15,427 ERROR [io.undertow.request] (default task-445) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:45:50,345 ERROR [org.xnio.listener] (default I/O-42) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:47:44,646 ERROR [io.undertow.request] (default task-2) UT005023: Exception handling request to /hawkular/inventory/resources/feed-e2701838-a82a-4ae9-902d-ffd8277e33ea-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:46:39,299 ERROR [org.xnio.nio] (default I/O-25) XNIO000011: Task org.xnio.nio.QueuedNioTcpServer$1@5743d574 failed with an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:46:55,796 ERROR [org.xnio.listener] (default I/O-21) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:46:45,998 ERROR [io.undertow.request] (default I/O-30) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-c9b56f20-54ab-4b5c-b315-31c74bc9dc44-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {X-Powered-By=[Undertow/1], Server=[WildFly/10]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:22,324 ERROR [io.undertow.request] (default task-454) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-80e6cbfa-f959-49b6-a5ed-0a57c869f281-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:46:40,254 ERROR [io.undertow.request] (default task-360) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-c3759400-798b-4cca-90f4-fcb927e665bf-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:45:56,688 ERROR [org.xnio.listener] (default task-246) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:27,416 ERROR [io.undertow.request] (default task-95) UT005023: Exception handling request to /hawkular/inventory/resources/feed-203352e1-c559-4cc3-a2a6-e107af130c03-NewApp.war: org.jboss.resteasy.spi.UnhandledException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.jboss.resteasy.core.ExceptionHandler.handleApplicationException(ExceptionHandler.java:77) at org.jboss.resteasy.core.ExceptionHandler.handleException(ExceptionHandler.java:220) at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:175) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:418) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:55:59,799 ERROR [io.undertow.request] (default task-324) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:174) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:478) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:422) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:162) at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:247) at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:588) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.write(AbstractFixedLengthStreamSinkConduit.java:106) at org.xnio.conduits.Conduits.writeFinalBasic(Conduits.java:132) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.writeFinal(AbstractFixedLengthStreamSinkConduit.java:175) at org.xnio.conduits.ConduitStreamSinkChannel.writeFinal(ConduitStreamSinkChannel.java:104) at io.undertow.channels.DetachableStreamSinkChannel.writeFinal(DetachableStreamSinkChannel.java:195) at io.undertow.server.HttpServerExchange$WriteDispatchChannel.writeFinal(HttpServerExchange.java:2007) at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:561) at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:596) at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.close(HttpServletResponseWrapper.java:58) at org.jboss.resteasy.util.CommitHeaderOutputStream.close(CommitHeaderOutputStream.java:87) at org.jboss.resteasy.util.DelegatingOutputStream.close(DelegatingOutputStream.java:60) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1036) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.writeTo(ResteasyJackson2Provider.java:209) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131) at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120) at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:473) ... 46 more 22:46:43,818 ERROR [org.xnio.listener] (default I/O-43) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:46:36,183 ERROR [io.undertow.request] (default task-147) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:29,789 ERROR [org.jboss.as.txn] (default task-131) WFLYTX0003: APPLICATION ERROR: transaction still active in request with status 0 22:45:15,857 ERROR [io.undertow.request] (default task-300) UT005023: Exception handling request to /hawkular/inventory/resources/feed-a630c1c4-4ea0-479b-a1a3-6cb9c54bc2d9-NewApp.war: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:40:58,326 ERROR [io.undertow.request] (default task-303) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:174) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:478) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:422) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:162) at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:247) at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:588) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.write(AbstractFixedLengthStreamSinkConduit.java:106) at org.xnio.conduits.Conduits.writeFinalBasic(Conduits.java:132) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.writeFinal(AbstractFixedLengthStreamSinkConduit.java:175) at org.xnio.conduits.ConduitStreamSinkChannel.writeFinal(ConduitStreamSinkChannel.java:104) at io.undertow.channels.DetachableStreamSinkChannel.writeFinal(DetachableStreamSinkChannel.java:195) at io.undertow.server.HttpServerExchange$WriteDispatchChannel.writeFinal(HttpServerExchange.java:2007) at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:561) at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:596) at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.close(HttpServletResponseWrapper.java:58) at org.jboss.resteasy.util.CommitHeaderOutputStream.close(CommitHeaderOutputStream.java:87) at org.jboss.resteasy.util.DelegatingOutputStream.close(DelegatingOutputStream.java:60) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1036) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.writeTo(ResteasyJackson2Provider.java:209) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131) at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120) at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:473) ... 46 more 22:39:53,903 ERROR [stderr] (Thread-29 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@1d6c5de7-649995816)) ava.lang.OutOfMemoryError: GC overhead limit exceeded1" -ActiveMQServerImpl::serverUUID=91cf1c8a-ba7c-11e7-8f26-c9b2ac1e0bc7-166211658-584545138)" Exception in thread "OkHttp Dispatcher" Exception in thread "default task-81" Exception in thread "management task-4" Exception in thread "default task-403" Exception in thread "ServerService Thread Pool -- 245" Exception in thread "OkHttp Dispatcher" Exception in thread "default task-198" Exception in thread "default task-215" Exception in thread "default task-42" Exception in thread "default task-311" Exception in thread "default task-230" Exception in thread "default task-148" Exception in thread "default task-204" Exception in thread "Hawkular WildFly Agent Full Discovery Scan-Local-1" Exception in thread "default task-328" Exception in thread "cluster2-worker-24" Exception in thread "default task-80" Exception in thread "default task-96" Exception in thread "default task-342" Exception in thread "default task-222" Exception in thread "default task-77" Exception in thread "default task-125" Exception in thread "default task-344" Exception in thread "AsyncStoreProcessor--p12-t11" Exception in thread "AsyncStoreProcessor--p12-t13" Exception in thread "AsyncStoreProcessor--p12-t9" Exception in thread "AsyncStoreProcessor--p12-t14" Exception in thread "default task-249" Exception in thread "AsyncStoreProcessor--p12-t12" Exception in thread "default task-132" Exception in thread "default task-158" Exception in thread "default task-79" Exception in thread "default task-432" Exception in thread "default I/O-6" Exception in thread "default task-39" Exception in thread "default task-341" Exception in thread "default I/O-10" Exception in thread "ServerService Thread Pool -- 81" Exception in thread "OkHttp Dispatcher" Exception in thread "default task-205" Exception in thread "default task-294" Exception in thread "default task-388" Exception in thread "default task-253" Exception in thread "Hibernate Search sync consumer thread for index resource.4" Exception in thread "Thread-16 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@1d6c5de7-649995816)" Exception in thread "default task-19" Exception in thread "default task-376" Exception in thread "default task-165" Exception in thread "default task-82" Exception in thread "default task-247" Exception in thread "default task-127" Exception in thread "default task-7" Exception in thread "default task-337" Exception in thread "default task-226" Exception in thread "default task-163" Exception in thread "Hawkular-WildFly-Agent-Storage-Dispatcher-Metric" Exception in thread "default task-52" java.lang.OutOfMemoryError: GC overhead limit exceeded 22:43:02,519 ERROR [org.xnio.listener] (default I/O-45) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:55:59,802 ERROR [io.undertow.request] (default task-131) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-5d6ba7bb-7274-4ab2-b1a2-276b6f43d327-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.129.0.120;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.129.0.120], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Connection=[keep-alive], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache], Content-Length=[0], Date=[Thu, 26 Oct 2017 22:28:47 GMT]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:55:59,802 ERROR [stderr] (Thread-17 (ActiveMQ-server-org.apache.activemq.artemis.core.server.impl.ActiveMQServerImpl$2@1d6c5de7-649995816)) Exception in thread "OkHttp Dispatcher" Exception in thread "Hawkular WildFly Agent Full Discovery Scan-Local JMX-2" Exception in thread "OkHttp Dispatcher" java.lang.OutOfMemoryError: GC overhead limit exceeded 22:55:59,801 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-324) ISPN000136: Error executing command RemoveCommand, writing keys [feed-c0de849c-d5cb-4404-ac9e-889e9b026f5b-NewApp.war]: java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: 0:ffff0a810049:1bf44a5d:59f22b27:30af8 status: ActionStatus.COMMITTED > is not in a valid state to be invoking cache operations on. at org.infinispan.interceptors.impl.TxInterceptor.enlist(TxInterceptor.java:485) at org.infinispan.interceptors.impl.TxInterceptor.handleWriteCommand(TxInterceptor.java:451) at org.infinispan.interceptors.impl.TxInterceptor.visitRemoveCommand(TxInterceptor.java:249) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:63) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:154) at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitRemoveCommand(CacheMgmtInterceptor.java:217) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:63) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitRemoveCommand(IsMarshallableInterceptor.java:68) at org.infinispan.commands.write.RemoveCommand.acceptVisitor(RemoveCommand.java:63) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679) at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:661) at org.infinispan.cache.impl.DecoratedCache.remove(DecoratedCache.java:485) at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:363) at org.infinispan.cache.impl.EncoderCache.remove(EncoderCache.java:664) at java.util.Collections$SingletonSet.forEach(Collections.java:4767) at org.hawkular.inventory.service.InventoryServiceIspn.deleteResources(InventoryServiceIspn.java:145) at sun.reflect.GeneratedMethodAccessor165.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:82) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) 22:56:23,107 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-435) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-619cc88d-afd2-4482-89c7-b9197e47c27d-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-619cc88d-afd2-4482-89c7-b9197e47c27d-NewApp.war and requestor GlobalTx:local:61626. Lock is held by GlobalTx:local:61015 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:50:04,681 WARN [com.arjuna.ats.arjuna] (default task-353) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30044, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@eb85} org.infinispan.transaction.synchronization.SynchronizationAdapter@eba4 >: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:37,007 ERROR [io.undertow.request] (default task-196) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:49:53,769 ERROR [io.undertow.request] (default task-305) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:48,325 ERROR [io.undertow.request] (default task-375) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-1b8a1fa8-c1dd-4e8a-84d9-2251a2d0682b-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.31;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Connection=[close], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.31], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:51:26,962 ERROR [io.undertow.request] (default task-448) UT005071: Undertow request failed HttpServerExchange{ POST /hawkular/inventory/import request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[498], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:20,589 WARN [com.arjuna.ats.arjuna] (default task-265) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30079, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[feed-31299912-b4e8-40d1-915f-d835a17bc20a-NewApp.war], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@eb98} org.infinispan.transaction.synchronization.SynchronizationAdapter@ebb7 >: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:36,373 ERROR [io.undertow.request] (default task-374) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:52:25,331 WARNING [io.netty.util.HashedWheelTimer] (cluster2-timeouter-0) An exception was thrown by TimerTask.: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:12,582 ERROR [io.undertow.request] (default task-398) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:52:36,233 ERROR [io.undertow.request] (default task-3) UT005071: Undertow request failed HttpServerExchange{ DELETE /hawkular/inventory/resources/feed-03297229-cbe0-460d-ba82-7754bdc5b2b1-NewApp.war request {Accept=[application/json], Accept-Encoding=[gzip, deflate], User-Agent=[python-requests/2.18.4], Forwarded=[for=10.16.23.44;host=hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com;proto=http], Authorization=[Basic amRvZTpwYXNzd29yZA==], X-Forwarded-Proto=[http], X-Forwarded-Port=[80], X-Forwarded-For=[10.16.23.44], Content-type=[application/json], Content-Length=[0], Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com], X-Forwarded-Host=[hs-downstream-web-inventory-scalability-inventory-scalability.cloud2.jonqe.lab.eng.bos.redhat.com]} response {Expires=[0], Cache-Control=[no-cache, no-store, must-revalidate], X-Powered-By=[Undertow/1], Server=[WildFly/10], Pragma=[no-cache]}}: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:14,661 ERROR [io.undertow.request] (default task-316) UT005023: Exception handling request to /hawkular/inventory/import: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:52:16,975 ERROR [org.xnio.listener] (default I/O-2) XNIO001007: A channel event listener threw an exception: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:50:32,723 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-138) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-1114ae5a-f531-44ce-adf9-fb0bf7f5a968-NewApp.war]: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:56:26,607 ERROR [org.hawkular.agent.monitor.protocol.EndpointService] (Hawkular-WildFly-Agent-Scheduler-Metrics-3) HAWKAGENT010049: Could not access resources of endpoint [DMREndpointService[Endpoint[Local]:[https-remoting://127.0.0.1:9993]]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching DMR attribute [delivering-count] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:149) at org.hawkular.agent.monitor.protocol.EndpointService.measureMetrics(EndpointService.java:586) at org.hawkular.agent.monitor.scheduler.MetricsCollector.run(MetricsCollector.java:60) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchAttribute(DMRDriver.java:146) ... 8 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 9 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 11 more 22:56:26,609 ERROR [org.jboss.as.ejb3.invocation] (default task-265) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) ... 91 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:56:27,767 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-138) ISPN000255: Error while processing prepare: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:56:26,608 ERROR [org.jboss.as.ejb3.invocation] (default task-353) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) ... 91 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:56:26,626 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-411) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-d8099298-8ac3-4e41-9c3a-9e605b64dbc7-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-d8099298-8ac3-4e41-9c3a-9e605b64dbc7-NewApp.war and requestor GlobalTx:local:61693. Lock is held by GlobalTx:local:60674 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:56:26,625 ERROR [org.hawkular.agent.monitor.protocol.Discovery] (Hawkular WildFly Agent Full Discovery Scan-Local-2) HAWKAGENT010061: Failed to discover resources in [Endpoint[Local]:[https-remoting://127.0.0.1:9993]]: org.hawkular.agent.monitor.protocol.ProtocolException: Error fetching nodes for query [/] at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:224) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:55) at org.hawkular.agent.monitor.protocol.Discovery.discoverChildren(Discovery.java:75) at org.hawkular.agent.monitor.protocol.EndpointService.discoverChildren(EndpointService.java:450) at org.hawkular.agent.monitor.protocol.EndpointService.access$900(EndpointService.java:83) at org.hawkular.agent.monitor.protocol.EndpointService$1.run(EndpointService.java:397) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at org.shaded.jboss.threads.JBossThread.run(JBossThread.java:320) Caused by: org.hawkular.dmr.api.DmrApiException: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:86) at org.hawkular.agent.monitor.protocol.dmr.DMRDriver.fetchNodes(DMRDriver.java:221) ... 9 more Caused by: java.io.IOException: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:149) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:75) at org.hawkular.dmr.api.OperationBuilder$AbstractOperationBuilder.execute(OperationBuilder.java:84) ... 10 more Caused by: java.net.ConnectException: WFLYPRT0023: Could not connect to https-remoting://127.0.0.1:9993. The connection timed out at org.shaded.jboss.as.protocol.ProtocolConnectionUtils.connectSync(ProtocolConnectionUtils.java:124) at org.shaded.jboss.as.protocol.ProtocolConnectionManager$EstablishingConnection.connect(ProtocolConnectionManager.java:257) at org.shaded.jboss.as.protocol.ProtocolConnectionManager.connect(ProtocolConnectionManager.java:71) at org.shaded.jboss.as.protocol.mgmt.FutureManagementChannel$Establishing.getChannel(FutureManagementChannel.java:212) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient.getOrCreateChannel(RemotingModelControllerClient.java:146) at org.shaded.jboss.as.controller.client.impl.RemotingModelControllerClient$1.getChannel(RemotingModelControllerClient.java:65) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:147) at org.shaded.jboss.as.protocol.mgmt.ManagementChannelHandler.executeRequest(ManagementChannelHandler.java:122) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeRequest(AbstractModelControllerClient.java:263) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.execute(AbstractModelControllerClient.java:168) at org.shaded.jboss.as.controller.client.impl.AbstractModelControllerClient.executeForResult(AbstractModelControllerClient.java:147) ... 12 more 22:56:27,769 WARN [com.arjuna.ats.arjuna] (default task-138) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30ba1, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@ef11} org.infinispan.transaction.synchronization.SynchronizationAdapter@ef30 >: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.commons.CacheException: java.lang.OutOfMemoryError: GC overhead limit exceeded at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded 22:56:25,020 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-200) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-54e84e1d-c6df-411b-8b68-6a0620251825-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-54e84e1d-c6df-411b-8b68-6a0620251825-NewApp.war and requestor GlobalTx:local:61662. Lock is held by GlobalTx:local:60872 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:56:25,020 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-390) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-be490209-6fa7-4050-a6a5-feec88c4c2a6-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-be490209-6fa7-4050-a6a5-feec88c4c2a6-NewApp.war and requestor GlobalTx:local:61647. Lock is held by GlobalTx:local:60315 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-aea31206-8631-4903-ae55-d9d1742ecd8f-NewApp.war and requestor GlobalTx:local:61558. Lock is held by GlobalTx:local:60789 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-aea31206-8631-4903-ae55-d9d1742ecd8f-NewApp.war and requestor GlobalTx:local:61558. Lock is held by GlobalTx:local:60789 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:56:34,891 WARN [com.arjuna.ats.arjuna] (default task-327) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30faf, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@f06f} org.infinispan.transaction.synchronization.SynchronizationAdapter@f08e >: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-930a39bb-51ac-40c9-a946-f17f62d2c611-NewApp.war and requestor GlobalTx:local:61551. Lock is held by GlobalTx:local:61168 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-930a39bb-51ac-40c9-a946-f17f62d2c611-NewApp.war and requestor GlobalTx:local:61551. Lock is held by GlobalTx:local:61168 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:56:38,196 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-314) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-558a4ce7-f81a-4c36-9f2e-ce46943df3a0-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-558a4ce7-f81a-4c36-9f2e-ce46943df3a0-NewApp.war and requestor GlobalTx:local:61858. Lock is held by GlobalTx:local:61233 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:56:38,196 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-243) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-eb55f32e-5eca-400c-8a7c-5fe48980cbed-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-eb55f32e-5eca-400c-8a7c-5fe48980cbed-NewApp.war and requestor GlobalTx:local:61831. Lock is held by GlobalTx:local:61238 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:56:38,197 ERROR [org.jboss.as.ejb3.invocation] (default task-119) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: 0:ffff0a810049:1bf44a5d:59f22b27:2ffe9 status: ActionStatus.COMMITTED > is not in a valid state to be invoking cache operations on. at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleInCallerTx(CMTTxInterceptor.java:159) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:256) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:329) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.infinispan.commons.CacheException: java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: 0:ffff0a810049:1bf44a5d:59f22b27:2ffe9 status: ActionStatus.COMMITTED > is not in a valid state to be invoking cache operations on. at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:140) at org.infinispan.interceptors.impl.InvocationContextInterceptor.access$000(InvocationContextInterceptor.java:43) at org.infinispan.interceptors.impl.InvocationContextInterceptor$1.apply(InvocationContextInterceptor.java:60) at org.infinispan.interceptors.InvocationExceptionFunction.apply(InvocationExceptionFunction.java:21) at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.addCallback(SimpleAsyncInvocationStage.java:67) at org.infinispan.interceptors.InvocationStage.andExceptionally(InvocationStage.java:34) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:132) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.cache.impl.CacheImpl.executeCommandAndCommitIfNeeded(CacheImpl.java:1679) at org.infinispan.cache.impl.CacheImpl.putAll(CacheImpl.java:1405) at org.infinispan.cache.impl.DecoratedCache.putAll(DecoratedCache.java:490) at org.infinispan.cache.impl.DecoratedCache.putAll(DecoratedCache.java:495) at org.infinispan.cache.impl.AbstractDelegatingCache.putAll(AbstractDelegatingCache.java:368) at org.infinispan.cache.impl.EncoderCache.putAll(EncoderCache.java:670) at org.hawkular.inventory.service.InventoryServiceIspn.addResource(InventoryServiceIspn.java:121) at sun.reflect.GeneratedMethodAccessor162.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:82) at org.jboss.as.weld.ejb.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:93) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.jpa.interceptor.SBInvocationInterceptor.processInvocation(SBInvocationInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:437) at org.jboss.weld.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:64) at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:83) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.pool.PooledInstanceInterceptor.processInvocation(PooledInstanceInterceptor.java:51) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:254) ... 90 more Caused by: java.lang.IllegalStateException: Transaction TransactionImple < ac, BasicAction: 0:ffff0a810049:1bf44a5d:59f22b27:2ffe9 status: ActionStatus.COMMITTED > is not in a valid state to be invoking cache operations on. at org.infinispan.interceptors.impl.TxInterceptor.enlist(TxInterceptor.java:485) at org.infinispan.interceptors.impl.TxInterceptor.handleWriteCommand(TxInterceptor.java:451) at org.infinispan.interceptors.impl.TxInterceptor.visitPutMapCommand(TxInterceptor.java:274) at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:78) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndFinally(BaseAsyncInterceptor.java:154) at org.infinispan.interceptors.impl.CacheMgmtInterceptor.visitPutMapCommand(CacheMgmtInterceptor.java:148) at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:78) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.IsMarshallableInterceptor.visitPutMapCommand(IsMarshallableInterceptor.java:60) at org.infinispan.commands.write.PutMapCommand.acceptVisitor(PutMapCommand.java:78) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) ... 127 more 22:56:55,522 ERROR [org.jboss.as.ejb3.invocation] (default task-327) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) ... 91 more Caused by: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) ... 94 more Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-930a39bb-51ac-40c9-a946-f17f62d2c611-NewApp.war and requestor GlobalTx:local:61551. Lock is held by GlobalTx:local:61168 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-930a39bb-51ac-40c9-a946-f17f62d2c611-NewApp.war and requestor GlobalTx:local:61551. Lock is held by GlobalTx:local:61168 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:56:55,522 ERROR [org.jboss.as.ejb3.invocation] (default task-239) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) ... 91 more Caused by: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) ... 94 more Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-aea31206-8631-4903-ae55-d9d1742ecd8f-NewApp.war and requestor GlobalTx:local:61558. Lock is held by GlobalTx:local:60789 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-aea31206-8631-4903-ae55-d9d1742ecd8f-NewApp.war and requestor GlobalTx:local:61558. Lock is held by GlobalTx:local:60789 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:56:38,198 WARN [com.arjuna.ats.arjuna] (default task-279) ARJUNA012127: TwoPhaseCoordinator.afterCompletion - returned failure for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:30b4f, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@eef8} org.infinispan.transaction.synchronization.SynchronizationAdapter@ef17 > 22:56:41,804 ERROR [io.undertow.request] (default task-324) UT005023: Exception handling request to /hawkular/inventory/resources/feed-c0de849c-d5cb-4404-ac9e-889e9b026f5b-NewApp.war: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:174) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:478) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:422) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:162) at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:247) at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:588) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.write(AbstractFixedLengthStreamSinkConduit.java:106) at org.xnio.conduits.Conduits.writeFinalBasic(Conduits.java:132) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.writeFinal(AbstractFixedLengthStreamSinkConduit.java:175) at org.xnio.conduits.ConduitStreamSinkChannel.writeFinal(ConduitStreamSinkChannel.java:104) at io.undertow.channels.DetachableStreamSinkChannel.writeFinal(DetachableStreamSinkChannel.java:195) at io.undertow.server.HttpServerExchange$WriteDispatchChannel.writeFinal(HttpServerExchange.java:2007) at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:561) at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:596) at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.close(HttpServletResponseWrapper.java:58) at org.jboss.resteasy.util.CommitHeaderOutputStream.close(CommitHeaderOutputStream.java:87) at org.jboss.resteasy.util.DelegatingOutputStream.close(DelegatingOutputStream.java:60) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1036) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.writeTo(ResteasyJackson2Provider.java:209) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131) at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120) at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:473) ... 46 more 22:56:55,523 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-243) ISPN000255: Error while processing prepare: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-eb55f32e-5eca-400c-8a7c-5fe48980cbed-NewApp.war and requestor GlobalTx:local:61831. Lock is held by GlobalTx:local:61238 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) 22:56:48,750 WARN [com.arjuna.ats.arjuna] (default task-76) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:3121f, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@f144} org.infinispan.transaction.synchronization.SynchronizationAdapter@f163 >: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-3a5aebea-e954-4746-a950-93bef268407e-NewApp.war and requestor GlobalTx:local:61764. Lock is held by GlobalTx:local:60702 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-3a5aebea-e954-4746-a950-93bef268407e-NewApp.war and requestor GlobalTx:local:61764. Lock is held by GlobalTx:local:60702 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:57:14,343 ERROR [io.undertow.request] (default task-84) UT005023: Exception handling request to /hawkular/inventory/import: org.jboss.resteasy.spi.UnhandledException: RESTEASY003770: Response is committed, can't handle exception at org.jboss.resteasy.core.SynchronousDispatcher.writeException(SynchronousDispatcher.java:174) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:478) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:422) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.writev0(Native Method) at sun.nio.ch.SocketDispatcher.writev(SocketDispatcher.java:51) at sun.nio.ch.IOUtil.write(IOUtil.java:148) at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:504) at org.xnio.nio.NioSocketConduit.write(NioSocketConduit.java:162) at io.undertow.server.protocol.http.HttpResponseConduit.processWrite(HttpResponseConduit.java:247) at io.undertow.server.protocol.http.HttpResponseConduit.write(HttpResponseConduit.java:588) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.write(AbstractFixedLengthStreamSinkConduit.java:106) at org.xnio.conduits.Conduits.writeFinalBasic(Conduits.java:132) at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.writeFinal(AbstractFixedLengthStreamSinkConduit.java:175) at org.xnio.conduits.ConduitStreamSinkChannel.writeFinal(ConduitStreamSinkChannel.java:104) at io.undertow.channels.DetachableStreamSinkChannel.writeFinal(DetachableStreamSinkChannel.java:195) at io.undertow.server.HttpServerExchange$WriteDispatchChannel.writeFinal(HttpServerExchange.java:2007) at io.undertow.servlet.spec.ServletOutputStreamImpl.writeBufferBlocking(ServletOutputStreamImpl.java:561) at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:596) at org.jboss.resteasy.plugins.server.servlet.HttpServletResponseWrapper$DeferredOutputStream.close(HttpServletResponseWrapper.java:58) at org.jboss.resteasy.util.CommitHeaderOutputStream.close(CommitHeaderOutputStream.java:87) at org.jboss.resteasy.util.DelegatingOutputStream.close(DelegatingOutputStream.java:60) at com.fasterxml.jackson.core.json.UTF8JsonGenerator.close(UTF8JsonGenerator.java:1036) at org.jboss.resteasy.plugins.providers.jackson.ResteasyJackson2Provider.writeTo(ResteasyJackson2Provider.java:209) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.writeTo(AbstractWriterInterceptorContext.java:131) at org.jboss.resteasy.core.interception.ServerWriterInterceptorContext.writeTo(ServerWriterInterceptorContext.java:60) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:120) at org.jboss.resteasy.security.doseta.DigitalSigningInterceptor.aroundWriteTo(DigitalSigningInterceptor.java:145) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.plugins.interceptors.encoding.GZIPEncodingInterceptor.aroundWriteTo(GZIPEncodingInterceptor.java:100) at org.jboss.resteasy.core.interception.AbstractWriterInterceptorContext.proceed(AbstractWriterInterceptorContext.java:124) at org.jboss.resteasy.core.ServerResponseWriter.writeNomapResponse(ServerResponseWriter.java:98) at org.jboss.resteasy.core.SynchronousDispatcher.writeResponse(SynchronousDispatcher.java:473) ... 46 more 22:57:26,087 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (default task-279) ISPN000136: Error executing command VersionedPrepareCommand, writing keys [feed-22e24ea3-73e1-4060-a740-e940ab968aa5-NewApp.war]: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-22e24ea3-73e1-4060-a740-e940ab968aa5-NewApp.war and requestor GlobalTx:local:62013. Lock is held by GlobalTx:local:60565 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 22:57:38,658 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-387) ISPN000255: Error while processing prepare: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-4dac0d50-0ece-44f6-bc43-f3232ae49590-NewApp.war and requestor GlobalTx:local:62033. Lock is held by GlobalTx:local:60824 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-4dac0d50-0ece-44f6-bc43-f3232ae49590-NewApp.war and requestor GlobalTx:local:62033. Lock is held by GlobalTx:local:60824 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:57:33,561 ERROR [org.jboss.as.ejb3.invocation] (default task-212) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) 22:57:38,658 ERROR [org.jboss.as.ejb3.invocation] (default task-279) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.deleteResources(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.deleteResources(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.deleteResource(InventoryHandlers.java:349) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.deleteResource(Unknown Source) at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.api.LegacyThreadSetupActionWrapper$1.call(LegacyThreadSetupActionWrapper.java:44) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:202) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:805) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.RollbackException: ARJUNA016053: Could not commit transaction. at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1212) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) ... 91 more Caused by: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) ... 94 more Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:150) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:113) at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:837) ... 100 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-22e24ea3-73e1-4060-a740-e940ab968aa5-NewApp.war and requestor GlobalTx:local:62013. Lock is held by GlobalTx:local:60565 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.interceptors.InterceptorChain.invoke(InterceptorChain.java:137) at org.infinispan.transaction.impl.TransactionCoordinator.prepare(TransactionCoordinator.java:130) ... 102 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000299: Unable to acquire lock after 10 seconds for key feed-22e24ea3-73e1-4060-a740-e940ab968aa5-NewApp.war and requestor GlobalTx:local:62013. Lock is held by GlobalTx:local:60565 at org.infinispan.util.concurrent.locks.impl.DefaultLockManager$KeyAwareExtendedLockPromise.lock(DefaultLockManager.java:253) at org.infinispan.interceptors.locking.AbstractLockingInterceptor.lockAllAndRecord(AbstractLockingInterceptor.java:278) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.checkPendingAndLockAllKeys(AbstractTxLockingInterceptor.java:168) at org.infinispan.interceptors.locking.AbstractTxLockingInterceptor.lockAllOrRegisterBackupLock(AbstractTxLockingInterceptor.java:134) at org.infinispan.interceptors.locking.OptimisticLockingInterceptor.visitPrepareCommand(OptimisticLockingInterceptor.java:40) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.impl.TxInterceptor.handlePrepareCommand(TxInterceptor.java:167) at org.infinispan.interceptors.impl.TxInterceptor.visitPrepareCommand(TxInterceptor.java:136) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNext(BaseAsyncInterceptor.java:58) at org.infinispan.interceptors.DDAsyncInterceptor.handleDefault(DDAsyncInterceptor.java:54) at org.infinispan.interceptors.DDAsyncInterceptor.visitPrepareCommand(DDAsyncInterceptor.java:132) at org.infinispan.commands.tx.PrepareCommand.acceptVisitor(PrepareCommand.java:185) at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:127) at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:96) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:248) ... 104 more 22:57:32,422 WARN [com.arjuna.ats.arjuna] (default task-333) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffff0a810049:1bf44a5d:59f22b27:31456, SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=null, isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=-1, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@f200} org.infinispan.transaction.synchronization.SynchronizationAdapter@f21f >: org.infinispan.commons.CacheException: Could not prepare. at org.infinispan.transaction.impl.TransactionTable.beforeCompletion(TransactionTable.java:839) at org.infinispan.transaction.synchronization.SynchronizationAdapter.beforeCompletion(SynchronizationAdapter.java:28) at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:368) at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91) at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162) at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200) at com.arjuna.ats.internal.jta.transaction.arjunacore.BaseTransaction.commit(BaseTransaction.java:126) at com.arjuna.ats.jbossatx.BaseTransactionManagerDelegate.commit(BaseTransactionManagerDelegate.java:89) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:91) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) 22:57:32,422 ERROR [org.jboss.as.ejb3.invocation] (default task-390) WFLYEJB0034: EJB Invocation failed on component InventoryServiceIspn for method public abstract void org.hawkular.inventory.api.InventoryService.addResource(java.util.Collection): javax.ejb.EJBTransactionRolledbackException: Transaction rolled back at org.jboss.as.ejb3.tx.CMTTxInterceptor.handleEndTransactionException(CMTTxInterceptor.java:137) at org.jboss.as.ejb3.tx.CMTTxInterceptor.endTransaction(CMTTxInterceptor.java:117) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInOurTx(CMTTxInterceptor.java:279) at org.jboss.as.ejb3.tx.CMTTxInterceptor.required(CMTTxInterceptor.java:327) at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:239) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:73) at org.hawkular.inventory.api.InventoryService$$$view2.addResource(Unknown Source) at org.hawkular.inventory.handlers.InventoryHandlers.importInventory(InventoryHandlers.java:235) at org.hawkular.inventory.handlers.InventoryHandlers$Proxy$_$$_WeldClientProxy.importInventory(Unknown Source) at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:295) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:249) at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:236) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:402) at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:209) at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:221) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:85) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:131) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.DisableCacheHandler.handleRequest(DisableCacheHandler.java:33) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)