-
Bug
-
Resolution: Done
-
Major
-
7.2.2.GA
-
None
There are many unexpected warnings in the following scenario:
- Start 3 EAP servers (3 pods) with deployed Servlet
- Start 64 clients which are load balanced to all servlets
- Scale down 1 pod and check that all clients are rebalancing to other 2 nodes
Expected result:
All clients failover to 2 remaining nodes without any warnings/errors in server logs.
Actual result:
In the moment when 3rd node leaves the cluster then following warning/errors occur in logs:
[0m[31m15:34:04,442 ERROR [org.jgroups.protocols.TCP] (TQ-Bundler-7,ee,hsc-1-pdgvd) JGRP000034: hsc-1-pdgvd: failure sending message to hsc-1-m7rbd: java.net.SocketTimeoutException: connect timed out ... [0m[31m15:34:14,014 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p18-t1) ISPN000136: Error executing command GetKeyValueCommand, writing keys []: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 596 from hsc-1-hht82 at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) ... [0m[31m15:34:14,027 ERROR [org.infinispan.transaction.impl.TransactionCoordinator] (default task-13) ISPN000097: Error while processing a prepare in a single-phase transaction: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 597 from hsc-1-hht82 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:156) at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:861) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33) at org.infinispan.commons.tx.TransactionImpl.notifyAfterCompletion(TransactionImpl.java:506) at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:338) at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:87) at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:748) Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 597 from hsc-1-hht82 at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Suppressed: org.infinispan.util.logging.TraceException at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:156) at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:861) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33) at org.infinispan.commons.tx.TransactionImpl.notifyAfterCompletion(TransactionImpl.java:506) at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:338) at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:87) at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) ... 1 more ... [0m[31m15:34:14,063 ERROR [org.infinispan.interceptors.impl.InvocationContextInterceptor] (timeout-thread--p18-t1) ISPN000136: Error executing command PrepareCommand, writing keys [SessionAccessMetaDataKey(DjnjcnyeUsQjJjjasUFAcbIrEFjWk3JfjeH1ydeG), SessionCreationMetaDataKey(DjnjcnyeUsQjJjjasUFAcbIrEFjWk3JfjeH1ydeG), SessionAttributesKey(DjnjcnyeUsQjJjjasUFAcbIrEFjWk3JfjeH1ydeG)]: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 602 from hsc-1-hht82 at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... [0m[33m15:34:24,066 WARN [org.infinispan.transaction.impl.TransactionCoordinator] (default task-5) ISPN000141: Could not rollback prepared 1PC transaction. This transaction will be rolled back by the recovery process, if enabled. Transaction: LocalTransaction{remoteLockedNodes=[hsc-1-pdgvd, hsc-1-hht82], isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=10, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@6c6: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 618 from hsc-1-hht82 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.transaction.impl.TransactionCoordinator.rollbackInternal(TransactionCoordinator.java:230) at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:201) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:158) at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:861) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33) at org.infinispan.commons.tx.TransactionImpl.notifyAfterCompletion(TransactionImpl.java:506) at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:338) at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:87) at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.lang.Thread.run(Thread.java:748) Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 618 from hsc-1-hht82 at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Suppressed: org.infinispan.util.logging.TraceException at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) at org.infinispan.transaction.impl.TransactionCoordinator.rollbackInternal(TransactionCoordinator.java:230) at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:201) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:158) at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:861) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33) at org.infinispan.commons.tx.TransactionImpl.notifyAfterCompletion(TransactionImpl.java:506) at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:338) at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:87) at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) ... 1 more ... [0m[31m15:34:24,082 ERROR [org.infinispan.commons.tx.TransactionImpl] (default task-18) ISPN000926: afterCompletion() failed for SynchronizationAdapter{localTransaction=LocalTransaction{remoteLockedNodes=[hsc-1-pdgvd, hsc-1-hht82], isMarkedForRollback=false, lockedKeys=[], backupKeyLocks=[], topologyId=10, stateTransferFlag=null} org.infinispan.transaction.synchronization.SyncLocalTransaction@6d5} org.infinispan.transaction.synchronization.SynchronizationAdapter@6f4: org.infinispan.commons.CacheException: Could not commit. at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:863) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33) at org.infinispan.commons.tx.TransactionImpl.notifyAfterCompletion(TransactionImpl.java:506) at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:338) at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:87) at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348) at java.lang.Thread.run(Thread.java:748) Caused by: javax.transaction.xa.XAException at org.infinispan.transaction.impl.TransactionCoordinator.handleCommitFailure(TransactionCoordinator.java:206) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:158) at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:861) ... 29 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 607 from hsc-1-hht82 at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:259) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:156) ... 30 more Caused by: org.infinispan.util.concurrent.TimeoutException: ISPN000476: Timed out waiting for responses for request 607 from hsc-1-hht82 at org.infinispan.remoting.transport.impl.MultiTargetRequest.onTimeout(MultiTargetRequest.java:167) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:87) at org.infinispan.remoting.transport.AbstractRequest.call(AbstractRequest.java:22) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ... 1 more Suppressed: org.infinispan.util.logging.TraceException at org.infinispan.interceptors.impl.SimpleAsyncInvocationStage.get(SimpleAsyncInvocationStage.java:41) at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:250) at org.infinispan.transaction.impl.TransactionCoordinator.commit(TransactionCoordinator.java:156) at org.infinispan.transaction.impl.TransactionTable.afterCompletion(TransactionTable.java:861) at org.infinispan.transaction.synchronization.SynchronizationAdapter.afterCompletion(SynchronizationAdapter.java:33) at org.infinispan.commons.tx.TransactionImpl.notifyAfterCompletion(TransactionImpl.java:506) at org.infinispan.commons.tx.TransactionImpl.runCommit(TransactionImpl.java:338) at org.infinispan.commons.tx.TransactionImpl.commit(TransactionImpl.java:110) at org.wildfly.clustering.ee.infinispan.InfinispanBatch.close(InfinispanBatch.java:97) at org.wildfly.clustering.web.undertow.session.DistributableSession.requestDone(DistributableSession.java:87) at io.undertow.servlet.spec.ServletContextImpl.updateSessionAccessTime(ServletContextImpl.java:945) at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:579) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348) ... 1 more
Test Sources: HAServletCounterDnsPingAsymEncryptTest#scaleDownFromThreeToTwo
https://gitlab.mw.lab.eng.bos.redhat.com/jbossqe-eap/openshift-eap-tests/blob/master/test-eap/src/test/java/com/redhat/xpaas/eap/ha/HAServletCounterDnsPingAsymEncryptTest.java
https://gitlab.mw.lab.eng.bos.redhat.com/jbossqe-eap/openshift-eap-tests/blob/master/test-eap/src/test/java/com/redhat/xpaas/eap/ha/AbstractHACounterTest.java
Test does not fail because of issue in clients but because it's checking server logs after the test and there are many new errors/warnings in the EAP pods. Attaching logs from test and all pods - JBEAP-14463-logs-eapcd-ocp311.zip
Configuration of EAP server:
<?xml version='1.0' encoding='UTF-8'?> <server xmlns="urn:jboss:domain:10.0"> <extensions> <extension module="org.jboss.as.clustering.infinispan"/> <extension module="org.jboss.as.clustering.jgroups"/> <extension module="org.jboss.as.connector"/> <extension module="org.jboss.as.deployment-scanner"/> <extension module="org.jboss.as.ee"/> <extension module="org.jboss.as.ejb3"/> <extension module="org.jboss.as.jaxrs"/> <extension module="org.jboss.as.jdr"/> <extension module="org.jboss.as.jmx"/> <extension module="org.jboss.as.jpa"/> <extension module="org.jboss.as.jsf"/> <extension module="org.jboss.as.logging"/> <extension module="org.jboss.as.mail"/> <extension module="org.jboss.as.modcluster"/> <extension module="org.jboss.as.naming"/> <extension module="org.jboss.as.pojo"/> <extension module="org.jboss.as.remoting"/> <extension module="org.jboss.as.sar"/> <extension module="org.jboss.as.security"/> <extension module="org.jboss.as.transactions"/> <extension module="org.jboss.as.webservices"/> <extension module="org.jboss.as.weld"/> <extension module="org.wildfly.extension.batch.jberet"/> <extension module="org.wildfly.extension.bean-validation"/> <extension module="org.wildfly.extension.clustering.singleton"/> <extension module="org.wildfly.extension.discovery"/> <extension module="org.wildfly.extension.ee-security"/> <extension module="org.wildfly.extension.elytron"/> <extension module="org.wildfly.extension.io"/> <extension module="org.wildfly.extension.messaging-activemq"/> <extension module="org.wildfly.extension.microprofile.config-smallrye"/> <extension module="org.wildfly.extension.microprofile.health-smallrye"/> <extension module="org.wildfly.extension.microprofile.metrics-smallrye"/> <!-- ##TRACING_EXTENSION## --> <extension module="org.wildfly.extension.request-controller"/> <extension module="org.wildfly.extension.security.manager"/> <extension module="org.wildfly.extension.undertow"/> <!-- ##KEYCLOAK_EXTENSION## --> </extensions> <management> <security-realms> <security-realm name="ManagementRealm"> <authentication> <local default-user="$local" skip-group-loading="true"/> <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/> </authentication> <authorization map-groups-to-roles="false"> <properties path="mgmt-groups.properties" relative-to="jboss.server.config.dir"/> </authorization> </security-realm> <security-realm name="ApplicationRealm"> <!-- No SSL configuration discovered --> <authentication> <local default-user="$local" allowed-users="*" skip-group-loading="true"/> <properties path="application-users.properties" relative-to="jboss.server.config.dir"/> </authentication> <authorization> <properties path="application-roles.properties" relative-to="jboss.server.config.dir"/> </authorization> </security-realm> </security-realms> <audit-log> <formatters> <json-formatter name="json-formatter"/> </formatters> <handlers> <file-handler name="file" formatter="json-formatter" path="audit-log.log" relative-to="jboss.server.data.dir"/> </handlers> <logger log-boot="true" log-read-only="false" enabled="false"> <handlers> <handler name="file"/> </handlers> </logger> </audit-log> <management-interfaces> <http-interface console-enabled="false"><!-- ##MGMT_IFACE_REALM## --> <http-upgrade enabled="true"/> <socket-binding http="management-http"/> </http-interface> </management-interfaces> <access-control provider="simple"> <role-mapping> <role name="SuperUser"> <include> <user name="$local"/> </include> </role> </role-mapping> </access-control> </management> <profile> <subsystem xmlns="urn:jboss:domain:logging:6.0"> <console-handler name="CONSOLE"> <formatter> <named-formatter name="COLOR-PATTERN"/> </formatter> </console-handler> <logger category="com.arjuna"> <level name="WARN"/> </logger> <logger category="io.jaegertracing.Configuration"> <level name="WARN"/> </logger> <logger category="org.jboss.as.config"> <level name="DEBUG"/> </logger> <logger category="sun.rmi"> <level name="WARN"/> </logger> <!-- ##LOGGER-CATEGORY## --> <root-logger> <level name="INFO"/> <handlers> <handler name="CONSOLE"/> </handlers> </root-logger> <formatter name="OPENSHIFT"> <json-formatter> <exception-output-type value="formatted"/> <key-overrides timestamp="@timestamp"/> <meta-data> <property name="@version" value="1"/> </meta-data> </json-formatter> </formatter> <formatter name="COLOR-PATTERN"> <pattern-formatter pattern="%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"/> </formatter> </subsystem> <subsystem xmlns="urn:jboss:domain:batch-jberet:2.0"> <default-job-repository name="in-memory"/> <default-thread-pool name="batch"/> <job-repository name="in-memory"> <in-memory/> </job-repository> <!-- ##JOB_REPOSITORY## --> <thread-pool name="batch"> <max-threads count="10"/> <keepalive-time time="30" unit="seconds"/> </thread-pool> </subsystem> <subsystem xmlns="urn:jboss:domain:bean-validation:1.0"/> <subsystem xmlns="urn:jboss:domain:datasources:5.0"> <datasources> <datasource jta="true" jndi-name="java:jboss/datasources/ExampleDS" pool-name="ExampleDS" enabled="true" use-java-context="true" statistics-enabled="${wildfly.datasources.statistics-enabled:${wildfly.statistics-enabled:false}}"> <connection-url>jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE</connection-url> <driver>h2</driver> <security> <user-name>sa</user-name> <password>sa</password> </security> </datasource><!-- ##DATASOURCES## --> <drivers> <driver name="h2" module="com.h2database.h2"> <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class> </driver> <driver name="mysql" module="com.mysql"> <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class> </driver> <driver name="postgresql" module="org.postgresql"> <xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class> </driver> <!-- ##DRIVERS## --> </drivers> </datasources> </subsystem> <subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0"> <deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}" auto-deploy-exploded="false"/> </subsystem> <subsystem xmlns="urn:jboss:domain:discovery:1.0"/> <subsystem xmlns="urn:jboss:domain:ee:4.0"> <spec-descriptor-property-replacement>false</spec-descriptor-property-replacement> <concurrent> <context-services> <context-service name="default" jndi-name="java:jboss/ee/concurrency/context/default" use-transaction-setup-provider="true"/> </context-services> <managed-thread-factories> <managed-thread-factory name="default" jndi-name="java:jboss/ee/concurrency/factory/default" context-service="default"/> </managed-thread-factories> <managed-executor-services> <managed-executor-service name="default" jndi-name="java:jboss/ee/concurrency/executor/default" context-service="default" hung-task-threshold="60000" keepalive-time="5000"/> </managed-executor-services> <managed-scheduled-executor-services> <managed-scheduled-executor-service name="default" jndi-name="java:jboss/ee/concurrency/scheduler/default" context-service="default" hung-task-threshold="60000" keepalive-time="3000"/> </managed-scheduled-executor-services> </concurrent> <default-bindings context-service="java:jboss/ee/concurrency/context/default" datasource="java:jboss/datasources/ExampleDS" jms-connection-factory="java:jboss/DefaultJMSConnectionFactory" managed-executor-service="java:jboss/ee/concurrency/executor/default" managed-scheduled-executor-service="java:jboss/ee/concurrency/scheduler/default" managed-thread-factory="java:jboss/ee/concurrency/factory/default"/> </subsystem> <subsystem xmlns="urn:jboss:domain:ee-security:1.0"/> <subsystem xmlns="urn:jboss:domain:ejb3:5.0"> <session-bean> <stateless> <bean-instance-pool-ref pool-name="slsb-strict-max-pool"/> </stateless> <stateful default-access-timeout="5000" cache-ref="distributable" passivation-disabled-cache-ref="simple"/> <singleton default-access-timeout="5000"/> </session-bean> <mdb> <resource-adapter-ref resource-adapter-name="${ejb.resource-adapter-name:activemq-ra.rar}"/> <bean-instance-pool-ref pool-name="mdb-strict-max-pool"/> </mdb> <pools> <bean-instance-pools> <strict-max-pool name="mdb-strict-max-pool" derive-size="from-cpu-count" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/> <strict-max-pool name="slsb-strict-max-pool" derive-size="from-worker-pools" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/> </bean-instance-pools> </pools> <caches> <cache name="simple"/> <cache name="distributable" passivation-store-ref="infinispan" aliases="passivating clustered"/> </caches> <passivation-stores> <passivation-store name="infinispan" cache-container="ejb" max-size="10000"/> </passivation-stores> <async thread-pool-name="default"/> <timer-service thread-pool-name="default" default-data-store="default-file-store"> <data-stores> <file-data-store name="default-file-store" path="timer-service-data" relative-to="jboss.server.data.dir"/> <!-- ##DATASTORES## --> </data-stores> </timer-service> <remote connector-ref="http-remoting-connector" thread-pool-name="default"> <channel-creation-options> <option name="READ_TIMEOUT" value="${prop.remoting-connector.read.timeout:20}" type="xnio"/> <option name="MAX_OUTBOUND_MESSAGES" value="1234" type="remoting"/> </channel-creation-options> </remote> <thread-pools> <thread-pool name="default"> <max-threads count="10"/> <keepalive-time time="100" unit="milliseconds"/> </thread-pool> </thread-pools> <default-security-domain value="other"/> <default-missing-method-permissions-deny-access value="true"/> <statistics enabled="${wildfly.ejb3.statistics-enabled:${wildfly.statistics-enabled:false}}"/> <log-system-exceptions value="true"/> </subsystem> <subsystem xmlns="urn:wildfly:elytron:6.0" final-providers="combined-providers" disallowed-providers="OracleUcrypto"> <providers> <aggregate-providers name="combined-providers"> <providers name="elytron"/> <providers name="openssl"/> </aggregate-providers> <provider-loader name="elytron" module="org.wildfly.security.elytron"/> <provider-loader name="openssl" module="org.wildfly.openssl"/> </providers> <audit-logging> <file-audit-log name="local-audit" path="audit.log" relative-to="jboss.server.log.dir" format="JSON"/> </audit-logging> <security-domains> <security-domain name="ApplicationDomain" default-realm="ApplicationRealm" permission-mapper="default-permission-mapper"> <realm name="ApplicationRealm" role-decoder="groups-to-roles"/> <realm name="local"/> </security-domain> <security-domain name="ManagementDomain" default-realm="ManagementRealm" permission-mapper="default-permission-mapper"> <realm name="ManagementRealm" role-decoder="groups-to-roles"/> <realm name="local" role-mapper="super-user-mapper"/> </security-domain> </security-domains> <security-realms> <identity-realm name="local" identity="$local"/> <properties-realm name="ApplicationRealm"> <users-properties path="application-users.properties" relative-to="jboss.server.config.dir" digest-realm-name="ApplicationRealm"/> <groups-properties path="application-roles.properties" relative-to="jboss.server.config.dir"/> </properties-realm> <properties-realm name="ManagementRealm"> <users-properties path="mgmt-users.properties" relative-to="jboss.server.config.dir" digest-realm-name="ManagementRealm"/> <groups-properties path="mgmt-groups.properties" relative-to="jboss.server.config.dir"/> </properties-realm> </security-realms> <mappers> <simple-permission-mapper name="default-permission-mapper" mapping-mode="first"> <permission-mapping> <principal name="anonymous"/> <permission-set name="default-permissions"/> </permission-mapping> <permission-mapping match-all="true"> <permission-set name="login-permission"/> <permission-set name="default-permissions"/> </permission-mapping> </simple-permission-mapper> <constant-realm-mapper name="local" realm-name="local"/> <simple-role-decoder name="groups-to-roles" attribute="groups"/> <constant-role-mapper name="super-user-mapper"> <role name="SuperUser"/> </constant-role-mapper> </mappers> <permission-sets> <permission-set name="login-permission"> <permission class-name="org.wildfly.security.auth.permission.LoginPermission"/> </permission-set> <permission-set name="default-permissions"> <permission class-name="org.wildfly.extension.batch.jberet.deployment.BatchPermission" module="org.wildfly.extension.batch.jberet" target-name="*"/> <permission class-name="org.wildfly.transaction.client.RemoteTransactionPermission" module="org.wildfly.transaction.client"/> <permission class-name="org.jboss.ejb.client.RemoteEJBPermission" module="org.jboss.ejb-client"/> </permission-set> </permission-sets> <http> <http-authentication-factory name="application-http-authentication" http-server-mechanism-factory="global" security-domain="ApplicationDomain"> <mechanism-configuration> <mechanism mechanism-name="BASIC"> <mechanism-realm realm-name="ApplicationRealm"/> </mechanism> <mechanism mechanism-name="FORM"/> </mechanism-configuration> </http-authentication-factory> <http-authentication-factory name="management-http-authentication" http-server-mechanism-factory="global" security-domain="ManagementDomain"> <mechanism-configuration> <mechanism mechanism-name="DIGEST"> <mechanism-realm realm-name="ManagementRealm"/> </mechanism> </mechanism-configuration> </http-authentication-factory> <provider-http-server-mechanism-factory name="global"/> </http> <sasl> <sasl-authentication-factory name="application-sasl-authentication" sasl-server-factory="configured" security-domain="ApplicationDomain"> <mechanism-configuration> <mechanism mechanism-name="JBOSS-LOCAL-USER" realm-mapper="local"/> <mechanism mechanism-name="DIGEST-MD5"> <mechanism-realm realm-name="ApplicationRealm"/> </mechanism> </mechanism-configuration> </sasl-authentication-factory> <sasl-authentication-factory name="management-sasl-authentication" sasl-server-factory="configured" security-domain="ManagementDomain"> <mechanism-configuration> <mechanism mechanism-name="JBOSS-LOCAL-USER" realm-mapper="local"/> <mechanism mechanism-name="DIGEST-MD5"> <mechanism-realm realm-name="ManagementRealm"/> </mechanism> </mechanism-configuration> </sasl-authentication-factory> <configurable-sasl-server-factory name="configured" sasl-server-factory="elytron"> <properties> <property name="wildfly.sasl.local-user.default-user" value="$local"/> </properties> </configurable-sasl-server-factory> <mechanism-provider-filtering-sasl-server-factory name="elytron" sasl-server-factory="global"> <filters> <filter provider-name="WildFlyElytron"/> </filters> </mechanism-provider-filtering-sasl-server-factory> <provider-sasl-server-factory name="global"/> </sasl> <tls> <key-stores> <key-store name="jgroups.jceks"> <credential-reference clear-text="xpaasQEjgroups"/> <implementation type="JCEKS"/> <file path="/etc/jgroups-encrypt-secret-volume/jgroups.jceks" /> </key-store><!-- ##ELYTRON_KEY_STORE## --> </key-stores> <key-managers> <!-- ##ELYTRON_KEY_MANAGER## --> </key-managers> <server-ssl-contexts> <!-- ##ELYTRON_SERVER_SSL_CONTEXT## --> </server-ssl-contexts> </tls> </subsystem> <subsystem xmlns="urn:jboss:domain:infinispan:8.0"> <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server"> <transport lock-timeout="60000"/> <replicated-cache name="default"> <transaction mode="BATCH"/> </replicated-cache> </cache-container> <cache-container name="web" default-cache="repl" module="org.wildfly.clustering.web.infinispan"> <transport lock-timeout="60000"/> <replicated-cache name="repl"> <locking isolation="REPEATABLE_READ"/> <transaction mode="BATCH"/> <file-store/> </replicated-cache> <distributed-cache name="dist"> <locking isolation="REPEATABLE_READ"/> <transaction mode="BATCH"/> <file-store/> </distributed-cache> </cache-container> <cache-container name="ejb" aliases="sfsb" default-cache="repl" module="org.wildfly.clustering.ejb.infinispan"> <transport lock-timeout="60000"/> <replicated-cache name="repl"> <locking isolation="REPEATABLE_READ"/> <object-memory size="10000"/> <transaction mode="BATCH"/> <file-store/> </replicated-cache> <distributed-cache name="dist"> <locking isolation="REPEATABLE_READ"/> <transaction mode="BATCH"/> <file-store/> </distributed-cache> </cache-container> <cache-container name="hibernate" default-cache="local-query" module="org.infinispan.hibernate-cache"> <transport lock-timeout="60000"/> <local-cache name="local-query"> <object-memory size="10000"/> <expiration max-idle="100000"/> </local-cache> <invalidation-cache name="entity"> <object-memory size="10000"/> <expiration max-idle="100000"/> </invalidation-cache> <replicated-cache name="timestamps"/> </cache-container> </subsystem> <subsystem xmlns="urn:jboss:domain:io:3.0"> <worker name="default"/> <buffer-pool name="default"/> </subsystem> <subsystem xmlns="urn:jboss:domain:jaxrs:1.0"/> <subsystem xmlns="urn:jboss:domain:jca:5.0"> <archive-validation enabled="true" fail-on-error="true" fail-on-warn="false"/> <bean-validation enabled="true"/> <default-workmanager> <short-running-threads> <core-threads count="50"/> <queue-length count="50"/> <max-threads count="50"/> <keepalive-time time="10" unit="seconds"/> </short-running-threads> <long-running-threads> <core-threads count="50"/> <queue-length count="50"/> <max-threads count="50"/> <keepalive-time time="10" unit="seconds"/> </long-running-threads> </default-workmanager> <cached-connection-manager/> </subsystem> <subsystem xmlns="urn:jboss:domain:jdr:1.0"/> <subsystem xmlns="urn:jboss:domain:jgroups:6.0"> <channels default="ee"> <channel name="ee" stack="tcp"/> </channels> <stacks> <stack name="udp"> <transport type="UDP" socket-binding="jgroups-udp"/> <protocol type="dns.DNS_PING" ><property name="dns_query">ping-service</property><property name="async_discovery_use_separate_thread_per_request">true</property></protocol> <protocol type="MERGE3"/> <protocol type="FD_SOCK"/> <protocol type="FD_ALL"/> <protocol type="VERIFY_SUSPECT"/> <encrypt-protocol type="SYM_ENCRYPT" key-store="jgroups.jceks" key-alias="xpaasqe"> <key-credential-reference clear-text="xpaasQEjgroups"/> </encrypt-protocol> <protocol type="pbcast.NAKACK2"/> <protocol type="UNICAST3"/> <protocol type="pbcast.STABLE"/> <!--WARNING: No password defined for JGroups cluster. AUTH protocol has been disabled. Please define JGROUPS_CLUSTER_PASSWORD. --> <protocol type="pbcast.GMS"/> <protocol type="UFC"/> <protocol type="MFC"/> <protocol type="FRAG3"/> </stack> <stack name="tcp"> <transport type="TCP" socket-binding="jgroups-tcp"/> <protocol type="dns.DNS_PING" ><property name="dns_query">ping-service</property><property name="async_discovery_use_separate_thread_per_request">true</property></protocol> <protocol type="MERGE3"/> <protocol type="FD_SOCK"/> <protocol type="FD_ALL"/> <protocol type="VERIFY_SUSPECT"/> <encrypt-protocol type="SYM_ENCRYPT" key-store="jgroups.jceks" key-alias="xpaasqe"> <key-credential-reference clear-text="xpaasQEjgroups"/> </encrypt-protocol> <protocol type="pbcast.NAKACK2"/> <protocol type="UNICAST3"/> <protocol type="pbcast.STABLE"/> <!--WARNING: No password defined for JGroups cluster. AUTH protocol has been disabled. Please define JGROUPS_CLUSTER_PASSWORD. --> <protocol type="pbcast.GMS"/> <protocol type="MFC"/> <protocol type="FRAG3"/> </stack> </stacks> </subsystem> <subsystem xmlns="urn:jboss:domain:jmx:1.3"> <expose-resolved-model/> <expose-expression-model/> <remoting-connector/> </subsystem> <subsystem xmlns="urn:jboss:domain:jpa:1.1"> <jpa default-datasource="" default-extended-persistence-inheritance="DEEP"/> </subsystem> <subsystem xmlns="urn:jboss:domain:jsf:1.1"/> <!-- ##KEYCLOAK_SUBSYSTEM## --> <!-- ##KEYCLOAK_SAML_SUBSYSTEM## --> <subsystem xmlns="urn:jboss:domain:mail:3.0"> <mail-session name="default" jndi-name="java:jboss/mail/Default"> <smtp-server outbound-socket-binding-ref="mail-smtp"/> </mail-session> </subsystem> <subsystem xmlns="urn:jboss:domain:messaging-activemq:6.0"> <server name="default"> <statistics enabled="${wildfly.messaging-activemq.statistics-enabled:${wildfly.statistics-enabled:false}}" /> <journal pool-files="10"/> <security-setting name="#"> <role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/> </security-setting> <address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10" redistribution-delay="1000"/> <http-connector name="http-connector" socket-binding="http-messaging" endpoint="http-acceptor"/> <http-connector name="http-connector-throughput" socket-binding="http-messaging" endpoint="http-acceptor-throughput"> <param name="batch-delay" value="50"/> </http-connector> <!-- ##AMQ_REMOTE_CONNECTOR## --> <in-vm-connector name="in-vm" server-id="0"> <param name="buffer-pooling" value="false"/> </in-vm-connector> <http-acceptor name="http-acceptor" http-listener="default"/> <http-acceptor name="http-acceptor-throughput" http-listener="default"> <param name="batch-delay" value="50"/> <param name="direct-deliver" value="false"/> </http-acceptor> <in-vm-acceptor name="in-vm" server-id="0"> <param name="buffer-pooling" value="false"/> </in-vm-acceptor> <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/> <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/> <connection-factory name="InVmConnectionFactory" connectors="in-vm" entries="java:/ConnectionFactory"/> <connection-factory name="RemoteConnectionFactory" connectors="http-connector" entries="java:jboss/exported/jms/RemoteConnectionFactory" reconnect-attempts="-1" /> <pooled-connection-factory name="activemq-ra" transaction="xa" connectors="in-vm" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory"/> <!-- ##AMQ_POOLED_CONNECTION_FACTORY## --> </server> </subsystem> <subsystem xmlns="urn:wildfly:microprofile-config-smallrye:1.0"> <!-- ##MICROPROFILE_CONFIG_SOURCE## --> </subsystem> <subsystem xmlns="urn:wildfly:microprofile-health-smallrye:1.0"/> <subsystem xmlns="urn:wildfly:microprofile-metrics-smallrye:2.0" security-enabled="false" exposed-subsystems="*" prefix="${wildfly.metrics.prefix:jboss}"/> <!-- ##TRACING_SUBSYSTEM## --> <subsystem xmlns="urn:jboss:domain:modcluster:5.0"> <proxy name="default" advertise-socket="modcluster" listener="ajp"> <dynamic-load-provider> <load-metric type="cpu"/> </dynamic-load-provider> </proxy> </subsystem> <subsystem xmlns="urn:jboss:domain:naming:2.0"> <!-- ##MESSAGING_REMOTE_BINDINGS## --> <!-- ##AMQ_REMOTE_CONTEXT## --> <remote-naming/> </subsystem> <subsystem xmlns="urn:jboss:domain:pojo:1.0"/> <subsystem xmlns="urn:jboss:domain:remoting:4.0"> <http-connector name="http-remoting-connector" connector-ref="default" security-realm="ApplicationRealm"/> </subsystem> <subsystem xmlns="urn:jboss:domain:request-controller:1.0"/> <subsystem xmlns="urn:jboss:domain:resource-adapters:5.0"> <resource-adapters> <!-- ##RESOURCE_ADAPTERS## --> </resource-adapters> </subsystem> <subsystem xmlns="urn:jboss:domain:sar:1.0"/> <subsystem xmlns="urn:jboss:domain:security:2.0"> <security-domains> <security-domain name="other" cache-type="default"> <authentication> <login-module code="Remoting" flag="optional"> <module-option name="password-stacking" value="useFirstPass"/> </login-module> <login-module code="RealmDirect" flag="required"> <module-option name="password-stacking" value="useFirstPass"/> </login-module> <!-- ##OTHER_LOGIN_MODULES## --> </authentication> </security-domain> <security-domain name="jboss-web-policy" cache-type="default"> <authorization> <policy-module code="Delegating" flag="required"/> </authorization> </security-domain> <security-domain name="jboss-ejb-policy" cache-type="default"> <authorization> <policy-module code="Delegating" flag="required"/> </authorization> </security-domain> <!-- ##KEYCLOAK_SECURITY_DOMAIN## --> <!-- no additional security domains configured --><!-- ##ADDITIONAL_SECURITY_DOMAINS## --> </security-domains> </subsystem> <subsystem xmlns="urn:jboss:domain:security-manager:1.0"> <deployment-permissions> <maximum-set> <permission class="java.security.AllPermission"/> </maximum-set> </deployment-permissions> </subsystem> <subsystem xmlns="urn:jboss:domain:singleton:1.0"> <singleton-policies default="default"> <singleton-policy name="default" cache-container="server"> <simple-election-policy/> </singleton-policy> </singleton-policies> </subsystem> <subsystem xmlns="urn:jboss:domain:transactions:5.0"> <core-environment node-identifier="${jboss.node.name}"> <process-id> <uuid/> </process-id> </core-environment> <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager" recovery-listener="true"/> <coordinator-environment statistics-enabled="${wildfly.transactions.statistics-enabled:${wildfly.statistics-enabled:false}}"/> <!-- ##JDBC_STORE## --> </subsystem> <subsystem xmlns="urn:jboss:domain:undertow:8.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other" statistics-enabled="${wildfly.undertow.statistics-enabled:${wildfly.statistics-enabled:false}}"> <buffer-cache name="default"/> <server name="default-server"> <ajp-listener name="ajp" socket-binding="ajp"/> <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true" proxy-address-forwarding="true"/> <!-- No HTTPS configuration discovered --> <host name="default-host" alias="localhost"> <location name="/" handler="welcome-content"/> <!-- ##ACCESS_LOG_VALVE## --> <!-- ##FILTER_REFS## --> <http-invoker security-realm="ApplicationRealm"/> </host> </server> <servlet-container name="default"> <jsp-config/> <websockets/> </servlet-container> <handlers> <file name="welcome-content" path="${jboss.home.dir}/welcome-content"/> </handlers> <!-- ##HTTP_FILTERS_MARKER## --> </subsystem> <subsystem xmlns="urn:jboss:domain:webservices:2.0" statistics-enabled="${wildfly.webservices.statistics-enabled:${wildfly.statistics-enabled:false}}"> <modify-wsdl-address>true</modify-wsdl-address> <wsdl-host>jbossws.undefined.host</wsdl-host> <endpoint-config name="Standard-Endpoint-Config"/> <endpoint-config name="Recording-Endpoint-Config"> <pre-handler-chain name="recording-handlers" protocol-bindings="##SOAP11_HTTP ##SOAP11_HTTP_MTOM ##SOAP12_HTTP ##SOAP12_HTTP_MTOM"> <handler name="RecordingHandler" class="org.jboss.ws.common.invocation.RecordingServerHandler"/> </pre-handler-chain> </endpoint-config> <client-config name="Standard-Client-Config"/> </subsystem> <subsystem xmlns="urn:jboss:domain:weld:4.0"/> </profile> <interfaces> <interface name="management"> <inet-address value="${jboss.bind.address.management:127.0.0.1}"/> </interface> <interface name="public"> <inet-address value="${jboss.bind.address:127.0.0.1}"/> </interface> <interface name="private"> <inet-address value="${jboss.bind.address.private:127.0.0.1}"/> </interface> <interface name="bindall"> <inet-address value="0.0.0.0"/> </interface> </interfaces> <socket-binding-group name="standard-sockets" default-interface="public" port-offset="0"> <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/> <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/> <socket-binding name="ajp" interface="bindall" port="${jboss.ajp.port:8009}"/> <socket-binding name="http" interface="bindall" port="${jboss.http.port:8080}"/> <socket-binding name="https" interface="bindall" port="${jboss.https.port:8443}"/> <socket-binding name="jgroups-mping" interface="private" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/> <socket-binding name="jgroups-tcp" interface="private" port="7600"/> <socket-binding name="jgroups-udp" interface="private" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/> <socket-binding name="modcluster" multicast-address="${jboss.modcluster.multicast.address:224.0.1.105}" multicast-port="23364"/> <outbound-socket-binding name="http-messaging"> <remote-destination host="${jboss.messaging.host:localhost}" port="${jboss.http.port:8080}"/> </outbound-socket-binding> <socket-binding name="messaging" port="5445"/><socket-binding name="messaging-throughput" port="5455"/> <!-- ##AMQ_MESSAGING_SOCKET_BINDING## --> <socket-binding name="txn-recovery-environment" port="4712"/> <socket-binding name="txn-status-manager" port="4713"/> <outbound-socket-binding name="mail-smtp"> <remote-destination host="localhost" port="25"/> </outbound-socket-binding> </socket-binding-group> </server>
- is incorporated by
-
JBEAP-17060 OpenShift HA Tests Stabilisation Tracker
-
- Closed
-
- is related to
-
-
- Closed
-
- relates to
-
-
- Closed
-