-
Bug
-
Resolution: Done
-
Blocker
-
7.3.0.CD16, 7.3.0.CD17
-
None
During testing of load balancing of HTTP clients and scale down of pod in cluster on Openshift 3.11 there were hit reset HTTP sessions.
Customer Impact: During scaling down of pod in EAP cluster there can be reset sessions of HTTP clients, loosing its state.
Test scenario:
- Start 3 EAP pods and wait until all are ready with deployed servlet. (there is no wait for forming cluster)
- Start 64 clients which are repeatedly (every 5 seconds) calling servlet and incrementing counter in its session, clients are load balanced to all pods.
- Every-time client calls servlet, it gets counter increased by one.
- Scale down one of pods
Expected result:
Clients will failover to remaining pods. Their session will not be reset.
Actual result:
Sometimes clients failover to remaining pods but their session is reset. For example client TestWorker-56 (see test.log in attached zip):
[2019-06-13 15:02:17,839] INFO [TestWorker-56]: counterVerifier thread TestWorker-56 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1
Investigation:
There are 3 pods with EAP:
hsc-1-jcclb - pod 1
hsc-1-mdxfd - pod 2
hsc-1-d7z2h - pod 3
Based on the logs from pods 1 and 2, it took ~1min for pod 3 to join the cluster. Pods 1 and 2 formed cluster during boot of EAP servers so immediately. Still from test pov all pods started (state "running") and clients were load balanced to all pods - starting at time:
14:56:29,519 INFO [stdout] (default task-1) requestedSessionId: null, queryString: null ...
After some time pod 3 joins the cluster, there are already clients on pod 3:
ESC[0mESC[0m14:57:34,072 INFO [org.infinispan.CLUSTER] (thread-15,ee,hsc-1-jcclb) ISPN100000: Node hsc-1-d7z2h joined the cluster
Then pod 3 is scaled down:
ESC[0mESC[0m14:58:41,408 INFO [org.infinispan.CLUSTER] (thread-15,ee,hsc-1-jcclb) ISPN100001: Node hsc-1-d7z2h left the cluster
and clients failover to other pods however all of them have reset session:
[2019-06-13 15:02:17,839] INFO [TestWorker-56]: counterVerifier thread TestWorker-56 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 [2019-06-13 15:02:17,843] INFO [TestWorker-58]: counterVerifier thread TestWorker-58 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-56 session unexpectedly reset (session reseted 0 times already out of expected 0 times java.lang.AssertionError: Thread TestWorker-58 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:18,220] INFO [TestWorker-52]: counterVerifier thread TestWorker-52 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-52 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:18,829] INFO [TestWorker-39]: counterVerifier thread TestWorker-39 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-39 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:18,833] INFO [TestWorker-0]: counterVerifier thread TestWorker-0 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-0 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:18,833] INFO [TestWorker-62]: counterVerifier thread TestWorker-62 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-62 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:19,443] INFO [TestWorker-57]: counterVerifier thread TestWorker-57 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-57 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:19,450] INFO [TestWorker-27]: counterVerifier thread TestWorker-27 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-27 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:19,451] INFO [TestWorker-37]: counterVerifier thread TestWorker-37 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-37 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:20,044] INFO [TestWorker-30]: counterVerifier thread TestWorker-30 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-30 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,268] INFO [TestWorker-43]: counterVerifier thread TestWorker-43 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-43 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,583] INFO [TestWorker-51]: counterVerifier thread TestWorker-51 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-51 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,583] INFO [TestWorker-47]: counterVerifier thread TestWorker-47 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-47 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,877] INFO [TestWorker-3]: counterVerifier thread TestWorker-3 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-3 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,877] INFO [TestWorker-50]: counterVerifier thread TestWorker-50 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-50 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,877] INFO [TestWorker-44]: counterVerifier thread TestWorker-44 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-44 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:21,878] INFO [TestWorker-15]: counterVerifier thread TestWorker-15 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-15 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:22,493] INFO [TestWorker-24]: counterVerifier thread TestWorker-24 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-24 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:22,494] INFO [TestWorker-53]: counterVerifier thread TestWorker-53 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-53 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:22,795] INFO [TestWorker-48]: counterVerifier thread TestWorker-48 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-48 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:24,028] INFO [TestWorker-9]: counterVerifier thread TestWorker-9 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-9 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:24,029] INFO [TestWorker-46]: counterVerifier thread TestWorker-46 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-46 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:24,032] INFO [TestWorker-21]: counterVerifier thread TestWorker-21 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-21 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:24,335] INFO [TestWorker-29]: counterVerifier thread TestWorker-29 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-jcclb-496ada08-a69e-4084-9999-95c96036278d 1 java.lang.AssertionError: Thread TestWorker-29 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:25,257] INFO [TestWorker-59]: counterVerifier thread TestWorker-59 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-59 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:25,873] INFO [TestWorker-35]: counterVerifier thread TestWorker-35 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-35 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:25,873] INFO [TestWorker-34]: counterVerifier thread TestWorker-34 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-34 session unexpectedly reset (session reseted 0 times already out of expected 0 times [2019-06-13 15:02:25,873] INFO [TestWorker-6]: counterVerifier thread TestWorker-6 session reset!, state: hsc-1-d7z2h-ff33b1ea-7fe7-4c42-89c5-504dadcbe9f2 25, 0 response: hsc-1-mdxfd-66b2eb07-4f8e-4073-961e-bc0b20e9a10a 1 java.lang.AssertionError: Thread TestWorker-6 session unexpectedly reset (session reseted 0 times already out of expected 0 times
Note that there is difference in times between OCP instance and Jenkins slave where is test running - it seems that clients have +3:23 min time.
Test sources - HAServletCounterUpstreamDnsPingTest.scaleDownFromThreeToTwo:
https://gitlab.mw.lab.eng.bos.redhat.com/jbossqe-eap/openshift-eap-tests/blob/master/test-eap/src/test/java/com/redhat/xpaas/eap/ha/HAServletCounterUpstreamDnsPingTest.java
https://gitlab.mw.lab.eng.bos.redhat.com/jbossqe-eap/openshift-eap-tests/blob/master/test-eap/src/test/java/com/redhat/xpaas/eap/ha/AbstractHACounterTest.java
Configuration of EAP server:
<?xml version='1.0' encoding='UTF-8'?> <server xmlns="urn:jboss:domain:10.0"> <extensions> <extension module="org.jboss.as.clustering.infinispan"/> <extension module="org.jboss.as.clustering.jgroups"/> <extension module="org.jboss.as.connector"/> <extension module="org.jboss.as.deployment-scanner"/> <extension module="org.jboss.as.ee"/> <extension module="org.jboss.as.ejb3"/> <extension module="org.jboss.as.jaxrs"/> <extension module="org.jboss.as.jdr"/> <extension module="org.jboss.as.jmx"/> <extension module="org.jboss.as.jpa"/> <extension module="org.jboss.as.jsf"/> <extension module="org.jboss.as.logging"/> <extension module="org.jboss.as.mail"/> <extension module="org.jboss.as.modcluster"/> <extension module="org.jboss.as.naming"/> <extension module="org.jboss.as.pojo"/> <extension module="org.jboss.as.remoting"/> <extension module="org.jboss.as.sar"/> <extension module="org.jboss.as.security"/> <extension module="org.jboss.as.transactions"/> <extension module="org.jboss.as.webservices"/> <extension module="org.jboss.as.weld"/> <extension module="org.wildfly.extension.batch.jberet"/> <extension module="org.wildfly.extension.bean-validation"/> <extension module="org.wildfly.extension.clustering.singleton"/> <extension module="org.wildfly.extension.discovery"/> <extension module="org.wildfly.extension.ee-security"/> <extension module="org.wildfly.extension.elytron"/> <extension module="org.wildfly.extension.io"/> <extension module="org.wildfly.extension.messaging-activemq"/> <extension module="org.wildfly.extension.microprofile.config-smallrye"/> <extension module="org.wildfly.extension.microprofile.health-smallrye"/> <extension module="org.wildfly.extension.microprofile.metrics-smallrye"/> <!-- ##TRACING_EXTENSION## --> <extension module="org.wildfly.extension.request-controller"/> <extension module="org.wildfly.extension.security.manager"/> <extension module="org.wildfly.extension.undertow"/> <!-- ##KEYCLOAK_EXTENSION## --> </extensions> <management> <security-realms> <security-realm name="ManagementRealm"> <authentication> <local default-user="$local" skip-group-loading="true"/> <properties path="mgmt-users.properties" relative-to="jboss.server.config.dir"/> </authentication> <authorization map-groups-to-roles="false"> <properties path="mgmt-groups.properties" relative-to="jboss.server.config.dir"/> </authorization> </security-realm> <security-realm name="ApplicationRealm"> <!-- No SSL configuration discovered --> <authentication> <local default-user="$local" allowed-users="*" skip-group-loading="true"/> <properties path="application-users.properties" relative-to="jboss.server.config.dir"/> </authentication> <authorization> <properties path="application-roles.properties" relative-to="jboss.server.config.dir"/> </authorization> </security-realm> </security-realms> <audit-log> <formatters> <json-formatter name="json-formatter"/> </formatters> <handlers> <file-handler name="file" formatter="json-formatter" path="audit-log.log" relative-to="jboss.server.data.dir"/> </handlers> <logger log-boot="true" log-read-only="false" enabled="false"> <handlers> <handler name="file"/> </handlers> </logger> </audit-log> <management-interfaces> <http-interface console-enabled="false"><!-- ##MGMT_IFACE_REALM## --> <http-upgrade enabled="true"/> <socket-binding http="management-http"/> </http-interface> </management-interfaces> <access-control provider="simple"> <role-mapping> <role name="SuperUser"> <include> <user name="$local"/> </include> </role> </role-mapping> </access-control> </management> <profile> <subsystem xmlns="urn:jboss:domain:logging:6.0"> <console-handler name="CONSOLE"> <formatter> <named-formatter name="COLOR-PATTERN"/> </formatter> </console-handler> <logger category="com.arjuna"> <level name="WARN"/> </logger> <logger category="io.jaegertracing.Configuration"> <level name="WARN"/> </logger> <logger category="org.jboss.as.config"> <level name="DEBUG"/> </logger> <logger category="sun.rmi"> <level name="WARN"/> </logger> <!-- ##LOGGER-CATEGORY## --> <root-logger> <level name="INFO"/> <handlers> <handler name="CONSOLE"/> </handlers> </root-logger> <formatter name="OPENSHIFT"> <json-formatter> <exception-output-type value="formatted"/> <key-overrides timestamp="@timestamp"/> <meta-data> <property name="@version" value="1"/> </meta-data> </json-formatter> </formatter> <formatter name="COLOR-PATTERN"> <pattern-formatter pattern="%K{level}%d{HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n"/> </formatter> </subsystem> <subsystem xmlns="urn:jboss:domain:batch-jberet:2.0"> <default-job-repository name="in-memory"/> <default-thread-pool name="batch"/> <job-repository name="in-memory"> <in-memory/> </job-repository> <!-- ##JOB_REPOSITORY## --> <thread-pool name="batch"> <max-threads count="10"/> <keepalive-time time="30" unit="seconds"/> </thread-pool> </subsystem> <subsystem xmlns="urn:jboss:domain:bean-validation:1.0"/> <subsystem xmlns="urn:jboss:domain:datasources:5.0"> <datasources> <datasource jta="true" jndi-name="java:jboss/datasources/ExampleDS" pool-name="ExampleDS" enabled="true" use-java-context="true" statistics-enabled="${wildfly.datasources.statistics-enabled:${wildfly.statistics-enabled:false}}"> <connection-url>jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE</connection-url> <driver>h2</driver> <security> <user-name>sa</user-name> <password>sa</password> </security> </datasource><!-- ##DATASOURCES## --> <drivers> <driver name="h2" module="com.h2database.h2"> <xa-datasource-class>org.h2.jdbcx.JdbcDataSource</xa-datasource-class> </driver> <driver name="mysql" module="com.mysql"> <xa-datasource-class>com.mysql.jdbc.jdbc2.optional.MysqlXADataSource</xa-datasource-class> </driver> <driver name="postgresql" module="org.postgresql"> <xa-datasource-class>org.postgresql.xa.PGXADataSource</xa-datasource-class> </driver> <!-- ##DRIVERS## --> </drivers> </datasources> </subsystem> <subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0"> <deployment-scanner path="deployments" relative-to="jboss.server.base.dir" scan-interval="5000" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}" auto-deploy-exploded="false"/> </subsystem> <subsystem xmlns="urn:jboss:domain:discovery:1.0"/> <subsystem xmlns="urn:jboss:domain:ee:4.0"> <spec-descriptor-property-replacement>false</spec-descriptor-property-replacement> <concurrent> <context-services> <context-service name="default" jndi-name="java:jboss/ee/concurrency/context/default" use-transaction-setup-provider="true"/> </context-services> <managed-thread-factories> <managed-thread-factory name="default" jndi-name="java:jboss/ee/concurrency/factory/default" context-service="default"/> </managed-thread-factories> <managed-executor-services> <managed-executor-service name="default" jndi-name="java:jboss/ee/concurrency/executor/default" context-service="default" hung-task-threshold="60000" keepalive-time="5000"/> </managed-executor-services> <managed-scheduled-executor-services> <managed-scheduled-executor-service name="default" jndi-name="java:jboss/ee/concurrency/scheduler/default" context-service="default" hung-task-threshold="60000" keepalive-time="3000"/> </managed-scheduled-executor-services> </concurrent> <default-bindings context-service="java:jboss/ee/concurrency/context/default" datasource="java:jboss/datasources/ExampleDS" jms-connection-factory="java:jboss/DefaultJMSConnectionFactory" managed-executor-service="java:jboss/ee/concurrency/executor/default" managed-scheduled-executor-service="java:jboss/ee/concurrency/scheduler/default" managed-thread-factory="java:jboss/ee/concurrency/factory/default"/> </subsystem> <subsystem xmlns="urn:jboss:domain:ee-security:1.0"/> <subsystem xmlns="urn:jboss:domain:ejb3:5.0"> <session-bean> <stateless> <bean-instance-pool-ref pool-name="slsb-strict-max-pool"/> </stateless> <stateful default-access-timeout="5000" cache-ref="distributable" passivation-disabled-cache-ref="simple"/> <singleton default-access-timeout="5000"/> </session-bean> <mdb> <resource-adapter-ref resource-adapter-name="${ejb.resource-adapter-name:activemq-ra.rar}"/> <bean-instance-pool-ref pool-name="mdb-strict-max-pool"/> </mdb> <pools> <bean-instance-pools> <strict-max-pool name="mdb-strict-max-pool" derive-size="from-cpu-count" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/> <strict-max-pool name="slsb-strict-max-pool" derive-size="from-worker-pools" instance-acquisition-timeout="5" instance-acquisition-timeout-unit="MINUTES"/> </bean-instance-pools> </pools> <caches> <cache name="simple"/> <cache name="distributable" passivation-store-ref="infinispan" aliases="passivating clustered"/> </caches> <passivation-stores> <passivation-store name="infinispan" cache-container="ejb" max-size="10000"/> </passivation-stores> <async thread-pool-name="default"/> <timer-service thread-pool-name="default" default-data-store="default-file-store"> <data-stores> <file-data-store name="default-file-store" path="timer-service-data" relative-to="jboss.server.data.dir"/> <!-- ##DATASTORES## --> </data-stores> </timer-service> <remote connector-ref="http-remoting-connector" thread-pool-name="default"> <channel-creation-options> <option name="READ_TIMEOUT" value="${prop.remoting-connector.read.timeout:20}" type="xnio"/> <option name="MAX_OUTBOUND_MESSAGES" value="1234" type="remoting"/> </channel-creation-options> </remote> <thread-pools> <thread-pool name="default"> <max-threads count="10"/> <keepalive-time time="100" unit="milliseconds"/> </thread-pool> </thread-pools> <default-security-domain value="other"/> <default-missing-method-permissions-deny-access value="true"/> <statistics enabled="${wildfly.ejb3.statistics-enabled:${wildfly.statistics-enabled:false}}"/> <log-system-exceptions value="true"/> </subsystem> <subsystem xmlns="urn:wildfly:elytron:6.0" final-providers="combined-providers" disallowed-providers="OracleUcrypto"> <providers> <aggregate-providers name="combined-providers"> <providers name="elytron"/> <providers name="openssl"/> </aggregate-providers> <provider-loader name="elytron" module="org.wildfly.security.elytron"/> <provider-loader name="openssl" module="org.wildfly.openssl"/> </providers> <audit-logging> <file-audit-log name="local-audit" path="audit.log" relative-to="jboss.server.log.dir" format="JSON"/> </audit-logging> <security-domains> <security-domain name="ApplicationDomain" default-realm="ApplicationRealm" permission-mapper="default-permission-mapper"> <realm name="ApplicationRealm" role-decoder="groups-to-roles"/> <realm name="local"/> </security-domain> <security-domain name="ManagementDomain" default-realm="ManagementRealm" permission-mapper="default-permission-mapper"> <realm name="ManagementRealm" role-decoder="groups-to-roles"/> <realm name="local" role-mapper="super-user-mapper"/> </security-domain> </security-domains> <security-realms> <identity-realm name="local" identity="$local"/> <properties-realm name="ApplicationRealm"> <users-properties path="application-users.properties" relative-to="jboss.server.config.dir" digest-realm-name="ApplicationRealm"/> <groups-properties path="application-roles.properties" relative-to="jboss.server.config.dir"/> </properties-realm> <properties-realm name="ManagementRealm"> <users-properties path="mgmt-users.properties" relative-to="jboss.server.config.dir" digest-realm-name="ManagementRealm"/> <groups-properties path="mgmt-groups.properties" relative-to="jboss.server.config.dir"/> </properties-realm> </security-realms> <mappers> <simple-permission-mapper name="default-permission-mapper" mapping-mode="first"> <permission-mapping> <principal name="anonymous"/> <permission-set name="default-permissions"/> </permission-mapping> <permission-mapping match-all="true"> <permission-set name="login-permission"/> <permission-set name="default-permissions"/> </permission-mapping> </simple-permission-mapper> <constant-realm-mapper name="local" realm-name="local"/> <simple-role-decoder name="groups-to-roles" attribute="groups"/> <constant-role-mapper name="super-user-mapper"> <role name="SuperUser"/> </constant-role-mapper> </mappers> <permission-sets> <permission-set name="login-permission"> <permission class-name="org.wildfly.security.auth.permission.LoginPermission"/> </permission-set> <permission-set name="default-permissions"> <permission class-name="org.wildfly.extension.batch.jberet.deployment.BatchPermission" module="org.wildfly.extension.batch.jberet" target-name="*"/> <permission class-name="org.wildfly.transaction.client.RemoteTransactionPermission" module="org.wildfly.transaction.client"/> <permission class-name="org.jboss.ejb.client.RemoteEJBPermission" module="org.jboss.ejb-client"/> </permission-set> </permission-sets> <http> <http-authentication-factory name="application-http-authentication" http-server-mechanism-factory="global" security-domain="ApplicationDomain"> <mechanism-configuration> <mechanism mechanism-name="BASIC"> <mechanism-realm realm-name="ApplicationRealm"/> </mechanism> <mechanism mechanism-name="FORM"/> </mechanism-configuration> </http-authentication-factory> <http-authentication-factory name="management-http-authentication" http-server-mechanism-factory="global" security-domain="ManagementDomain"> <mechanism-configuration> <mechanism mechanism-name="DIGEST"> <mechanism-realm realm-name="ManagementRealm"/> </mechanism> </mechanism-configuration> </http-authentication-factory> <provider-http-server-mechanism-factory name="global"/> </http> <sasl> <sasl-authentication-factory name="application-sasl-authentication" sasl-server-factory="configured" security-domain="ApplicationDomain"> <mechanism-configuration> <mechanism mechanism-name="JBOSS-LOCAL-USER" realm-mapper="local"/> <mechanism mechanism-name="DIGEST-MD5"> <mechanism-realm realm-name="ApplicationRealm"/> </mechanism> </mechanism-configuration> </sasl-authentication-factory> <sasl-authentication-factory name="management-sasl-authentication" sasl-server-factory="configured" security-domain="ManagementDomain"> <mechanism-configuration> <mechanism mechanism-name="JBOSS-LOCAL-USER" realm-mapper="local"/> <mechanism mechanism-name="DIGEST-MD5"> <mechanism-realm realm-name="ManagementRealm"/> </mechanism> </mechanism-configuration> </sasl-authentication-factory> <configurable-sasl-server-factory name="configured" sasl-server-factory="elytron"> <properties> <property name="wildfly.sasl.local-user.default-user" value="$local"/> </properties> </configurable-sasl-server-factory> <mechanism-provider-filtering-sasl-server-factory name="elytron" sasl-server-factory="global"> <filters> <filter provider-name="WildFlyElytron"/> </filters> </mechanism-provider-filtering-sasl-server-factory> <provider-sasl-server-factory name="global"/> </sasl> <tls> <key-stores> <key-store name="jgroups.jceks"> <credential-reference clear-text="xpaasQEjgroups"/> <implementation type="JCEKS"/> <file path="/etc/jgroups-encrypt-secret-volume/jgroups.jceks" /> </key-store><!-- ##ELYTRON_KEY_STORE## --> </key-stores> <key-managers> <!-- ##ELYTRON_KEY_MANAGER## --> </key-managers> <server-ssl-contexts> <!-- ##ELYTRON_SERVER_SSL_CONTEXT## --> </server-ssl-contexts> </tls> </subsystem> <subsystem xmlns="urn:jboss:domain:infinispan:8.0"> <cache-container name="server" aliases="singleton cluster" default-cache="default" module="org.wildfly.clustering.server"> <transport lock-timeout="60000"/> <replicated-cache name="default"> <transaction mode="BATCH"/> </replicated-cache> </cache-container> <cache-container name="web" default-cache="repl" module="org.wildfly.clustering.web.infinispan"> <transport lock-timeout="60000"/> <replicated-cache name="repl"> <locking isolation="REPEATABLE_READ"/> <transaction mode="BATCH"/> <file-store/> </replicated-cache> <distributed-cache name="dist"> <locking isolation="REPEATABLE_READ"/> <transaction mode="BATCH"/> <file-store/> </distributed-cache> </cache-container> <cache-container name="ejb" aliases="sfsb" default-cache="repl" module="org.wildfly.clustering.ejb.infinispan"> <transport lock-timeout="60000"/> <replicated-cache name="repl"> <locking isolation="REPEATABLE_READ"/> <object-memory size="10000"/> <transaction mode="BATCH"/> <file-store/> </replicated-cache> <distributed-cache name="dist"> <locking isolation="REPEATABLE_READ"/> <transaction mode="BATCH"/> <file-store/> </distributed-cache> </cache-container> <cache-container name="hibernate" default-cache="local-query" module="org.infinispan.hibernate-cache"> <transport lock-timeout="60000"/> <local-cache name="local-query"> <object-memory size="10000"/> <expiration max-idle="100000"/> </local-cache> <invalidation-cache name="entity"> <object-memory size="10000"/> <expiration max-idle="100000"/> </invalidation-cache> <replicated-cache name="timestamps"/> </cache-container> </subsystem> <subsystem xmlns="urn:jboss:domain:io:3.0"> <worker name="default"/> <buffer-pool name="default"/> </subsystem> <subsystem xmlns="urn:jboss:domain:jaxrs:1.0"/> <subsystem xmlns="urn:jboss:domain:jca:5.0"> <archive-validation enabled="true" fail-on-error="true" fail-on-warn="false"/> <bean-validation enabled="true"/> <default-workmanager> <short-running-threads> <core-threads count="50"/> <queue-length count="50"/> <max-threads count="50"/> <keepalive-time time="10" unit="seconds"/> </short-running-threads> <long-running-threads> <core-threads count="50"/> <queue-length count="50"/> <max-threads count="50"/> <keepalive-time time="10" unit="seconds"/> </long-running-threads> </default-workmanager> <cached-connection-manager/> </subsystem> <subsystem xmlns="urn:jboss:domain:jdr:1.0"/> <subsystem xmlns="urn:jboss:domain:jgroups:6.0"> <channels default="ee"> <channel name="ee" stack="tcp"/> </channels> <stacks> <stack name="udp"> <transport type="UDP" socket-binding="jgroups-udp"/> <protocol type="dns.DNS_PING" ><property name="dns_query">ping-service</property><property name="async_discovery_use_separate_thread_per_request">true</property></protocol> <protocol type="MERGE3"/> <protocol type="FD_SOCK"/> <protocol type="FD_ALL"/> <protocol type="VERIFY_SUSPECT"/> <encrypt-protocol type="SYM_ENCRYPT" key-store="jgroups.jceks" key-alias="xpaasqe"> <key-credential-reference clear-text="xpaasQEjgroups"/> </encrypt-protocol> <protocol type="pbcast.NAKACK2"/> <protocol type="UNICAST3"/> <protocol type="pbcast.STABLE"/> <!--WARNING: No password defined for JGroups cluster. AUTH protocol has been disabled. Please define JGROUPS_CLUSTER_PASSWORD. --> <protocol type="pbcast.GMS"/> <protocol type="UFC"/> <protocol type="MFC"/> <protocol type="FRAG3"/> </stack> <stack name="tcp"> <transport type="TCP" socket-binding="jgroups-tcp"/> <protocol type="dns.DNS_PING" ><property name="dns_query">ping-service</property><property name="async_discovery_use_separate_thread_per_request">true</property></protocol> <protocol type="MERGE3"/> <protocol type="FD_SOCK"/> <protocol type="FD_ALL"/> <protocol type="VERIFY_SUSPECT"/> <encrypt-protocol type="SYM_ENCRYPT" key-store="jgroups.jceks" key-alias="xpaasqe"> <key-credential-reference clear-text="xpaasQEjgroups"/> </encrypt-protocol> <protocol type="pbcast.NAKACK2"/> <protocol type="UNICAST3"/> <protocol type="pbcast.STABLE"/> <!--WARNING: No password defined for JGroups cluster. AUTH protocol has been disabled. Please define JGROUPS_CLUSTER_PASSWORD. --> <protocol type="pbcast.GMS"/> <protocol type="MFC"/> <protocol type="FRAG3"/> </stack> </stacks> </subsystem> <subsystem xmlns="urn:jboss:domain:jmx:1.3"> <expose-resolved-model/> <expose-expression-model/> <remoting-connector/> </subsystem> <subsystem xmlns="urn:jboss:domain:jpa:1.1"> <jpa default-datasource="" default-extended-persistence-inheritance="DEEP"/> </subsystem> <subsystem xmlns="urn:jboss:domain:jsf:1.1"/> <!-- ##KEYCLOAK_SUBSYSTEM## --> <!-- ##KEYCLOAK_SAML_SUBSYSTEM## --> <subsystem xmlns="urn:jboss:domain:mail:3.0"> <mail-session name="default" jndi-name="java:jboss/mail/Default"> <smtp-server outbound-socket-binding-ref="mail-smtp"/> </mail-session> </subsystem> <subsystem xmlns="urn:jboss:domain:messaging-activemq:6.0"> <server name="default"> <statistics enabled="${wildfly.messaging-activemq.statistics-enabled:${wildfly.statistics-enabled:false}}" /> <journal pool-files="10"/> <security-setting name="#"> <role name="guest" send="true" consume="true" create-non-durable-queue="true" delete-non-durable-queue="true"/> </security-setting> <address-setting name="#" dead-letter-address="jms.queue.DLQ" expiry-address="jms.queue.ExpiryQueue" max-size-bytes="10485760" page-size-bytes="2097152" message-counter-history-day-limit="10" redistribution-delay="1000"/> <http-connector name="http-connector" socket-binding="http-messaging" endpoint="http-acceptor"/> <http-connector name="http-connector-throughput" socket-binding="http-messaging" endpoint="http-acceptor-throughput"> <param name="batch-delay" value="50"/> </http-connector> <!-- ##AMQ_REMOTE_CONNECTOR## --> <in-vm-connector name="in-vm" server-id="0"> <param name="buffer-pooling" value="false"/> </in-vm-connector> <http-acceptor name="http-acceptor" http-listener="default"/> <http-acceptor name="http-acceptor-throughput" http-listener="default"> <param name="batch-delay" value="50"/> <param name="direct-deliver" value="false"/> </http-acceptor> <in-vm-acceptor name="in-vm" server-id="0"> <param name="buffer-pooling" value="false"/> </in-vm-acceptor> <jms-queue name="ExpiryQueue" entries="java:/jms/queue/ExpiryQueue"/> <jms-queue name="DLQ" entries="java:/jms/queue/DLQ"/> <connection-factory name="InVmConnectionFactory" connectors="in-vm" entries="java:/ConnectionFactory"/> <connection-factory name="RemoteConnectionFactory" connectors="http-connector" entries="java:jboss/exported/jms/RemoteConnectionFactory" reconnect-attempts="-1" /> <pooled-connection-factory name="activemq-ra" transaction="xa" connectors="in-vm" entries="java:/JmsXA java:jboss/DefaultJMSConnectionFactory"/> <!-- ##AMQ_POOLED_CONNECTION_FACTORY## --> </server> </subsystem> <subsystem xmlns="urn:wildfly:microprofile-config-smallrye:1.0"> <!-- ##MICROPROFILE_CONFIG_SOURCE## --> </subsystem> <subsystem xmlns="urn:wildfly:microprofile-health-smallrye:1.0"/> <subsystem xmlns="urn:wildfly:microprofile-metrics-smallrye:2.0" security-enabled="false" exposed-subsystems="*" prefix="${wildfly.metrics.prefix:jboss}"/> <!-- ##TRACING_SUBSYSTEM## --> <subsystem xmlns="urn:jboss:domain:modcluster:5.0"> <proxy name="default" advertise-socket="modcluster" listener="ajp"> <dynamic-load-provider> <load-metric type="cpu"/> </dynamic-load-provider> </proxy> </subsystem> <subsystem xmlns="urn:jboss:domain:naming:2.0"> <!-- ##MESSAGING_REMOTE_BINDINGS## --> <!-- ##AMQ_REMOTE_CONTEXT## --> <remote-naming/> </subsystem> <subsystem xmlns="urn:jboss:domain:pojo:1.0"/> <subsystem xmlns="urn:jboss:domain:remoting:4.0"> <http-connector name="http-remoting-connector" connector-ref="default" security-realm="ApplicationRealm"/> </subsystem> <subsystem xmlns="urn:jboss:domain:request-controller:1.0"/> <subsystem xmlns="urn:jboss:domain:resource-adapters:5.0"> <resource-adapters> <!-- ##RESOURCE_ADAPTERS## --> </resource-adapters> </subsystem> <subsystem xmlns="urn:jboss:domain:sar:1.0"/> <subsystem xmlns="urn:jboss:domain:security:2.0"> <security-domains> <security-domain name="other" cache-type="default"> <authentication> <login-module code="Remoting" flag="optional"> <module-option name="password-stacking" value="useFirstPass"/> </login-module> <login-module code="RealmDirect" flag="required"> <module-option name="password-stacking" value="useFirstPass"/> </login-module> <!-- ##OTHER_LOGIN_MODULES## --> </authentication> </security-domain> <security-domain name="jboss-web-policy" cache-type="default"> <authorization> <policy-module code="Delegating" flag="required"/> </authorization> </security-domain> <security-domain name="jboss-ejb-policy" cache-type="default"> <authorization> <policy-module code="Delegating" flag="required"/> </authorization> </security-domain> <!-- ##KEYCLOAK_SECURITY_DOMAIN## --> <!-- no additional security domains configured --><!-- ##ADDITIONAL_SECURITY_DOMAINS## --> </security-domains> </subsystem> <subsystem xmlns="urn:jboss:domain:security-manager:1.0"> <deployment-permissions> <maximum-set> <permission class="java.security.AllPermission"/> </maximum-set> </deployment-permissions> </subsystem> <subsystem xmlns="urn:jboss:domain:singleton:1.0"> <singleton-policies default="default"> <singleton-policy name="default" cache-container="server"> <simple-election-policy/> </singleton-policy> </singleton-policies> </subsystem> <subsystem xmlns="urn:jboss:domain:transactions:5.0"> <core-environment node-identifier="${jboss.node.name}"> <process-id> <uuid/> </process-id> </core-environment> <recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager" recovery-listener="true"/> <coordinator-environment statistics-enabled="${wildfly.transactions.statistics-enabled:${wildfly.statistics-enabled:false}}"/> <!-- ##JDBC_STORE## --> </subsystem> <subsystem xmlns="urn:jboss:domain:undertow:8.0" default-server="default-server" default-virtual-host="default-host" default-servlet-container="default" default-security-domain="other" statistics-enabled="${wildfly.undertow.statistics-enabled:${wildfly.statistics-enabled:false}}"> <buffer-cache name="default"/> <server name="default-server"> <ajp-listener name="ajp" socket-binding="ajp"/> <http-listener name="default" socket-binding="http" redirect-socket="https" enable-http2="true" proxy-address-forwarding="true"/> <!-- No HTTPS configuration discovered --> <host name="default-host" alias="localhost"> <location name="/" handler="welcome-content"/> <!-- ##ACCESS_LOG_VALVE## --> <!-- ##FILTER_REFS## --> <http-invoker security-realm="ApplicationRealm"/> </host> </server> <servlet-container name="default"> <jsp-config/> <websockets/> </servlet-container> <handlers> <file name="welcome-content" path="${jboss.home.dir}/welcome-content"/> </handlers> <!-- ##HTTP_FILTERS_MARKER## --> </subsystem> <subsystem xmlns="urn:jboss:domain:webservices:2.0" statistics-enabled="${wildfly.webservices.statistics-enabled:${wildfly.statistics-enabled:false}}"> <modify-wsdl-address>true</modify-wsdl-address> <wsdl-host>jbossws.undefined.host</wsdl-host> <endpoint-config name="Standard-Endpoint-Config"/> <endpoint-config name="Recording-Endpoint-Config"> <pre-handler-chain name="recording-handlers" protocol-bindings="##SOAP11_HTTP ##SOAP11_HTTP_MTOM ##SOAP12_HTTP ##SOAP12_HTTP_MTOM"> <handler name="RecordingHandler" class="org.jboss.ws.common.invocation.RecordingServerHandler"/> </pre-handler-chain> </endpoint-config> <client-config name="Standard-Client-Config"/> </subsystem> <subsystem xmlns="urn:jboss:domain:weld:4.0"/> </profile> <interfaces> <interface name="management"> <inet-address value="${jboss.bind.address.management:127.0.0.1}"/> </interface> <interface name="public"> <inet-address value="${jboss.bind.address:127.0.0.1}"/> </interface> <interface name="private"> <inet-address value="${jboss.bind.address.private:127.0.0.1}"/> </interface> <interface name="bindall"> <inet-address value="0.0.0.0"/> </interface> </interfaces> <socket-binding-group name="standard-sockets" default-interface="public" port-offset="0"> <socket-binding name="management-http" interface="management" port="${jboss.management.http.port:9990}"/> <socket-binding name="management-https" interface="management" port="${jboss.management.https.port:9993}"/> <socket-binding name="ajp" interface="bindall" port="${jboss.ajp.port:8009}"/> <socket-binding name="http" interface="bindall" port="${jboss.http.port:8080}"/> <socket-binding name="https" interface="bindall" port="${jboss.https.port:8443}"/> <socket-binding name="jgroups-mping" interface="private" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45700"/> <socket-binding name="jgroups-tcp" interface="private" port="7600"/> <socket-binding name="jgroups-udp" interface="private" port="55200" multicast-address="${jboss.default.multicast.address:230.0.0.4}" multicast-port="45688"/> <socket-binding name="modcluster" multicast-address="${jboss.modcluster.multicast.address:224.0.1.105}" multicast-port="23364"/> <outbound-socket-binding name="http-messaging"> <remote-destination host="${jboss.messaging.host:localhost}" port="${jboss.http.port:8080}"/> </outbound-socket-binding> <socket-binding name="messaging" port="5445"/><socket-binding name="messaging-throughput" port="5455"/> <!-- ##AMQ_MESSAGING_SOCKET_BINDING## --> <socket-binding name="txn-recovery-environment" port="4712"/> <socket-binding name="txn-status-manager" port="4713"/> <outbound-socket-binding name="mail-smtp"> <remote-destination host="localhost" port="25"/> </outbound-socket-binding> </socket-binding-group> </server>
- is caused by
-
-
- Closed
-
- is incorporated by
-
-
- Closed
-
-
JBEAP-17770 EAP CD 18.0 Release
-
- Closed
-