Loading...

XML

Word

Printable

Type: Bug
Resolution: Done
Priority: Major
Fix Version/s: 7.1.2.CR1, 7.1.2.GA
Affects Version/s: 7.1.0.GA
Component/s: mod_cluster
Labels:
- test_included

CDW devel_ack:
CDW docs_ack:
CDW pm_ack:
CDW qa_ack:
CDW release:
Target Release:

7.1.z.GA
Workaround:

Workaround Exists
Workaround Description:
Hide

Decrease socket-timeout in mod-cluster-config to minimize impact of down proxies (1-5 seconds):

<mod-cluster-config advertise-socket="modcluster" proxies="proxy1 proxy2" socket-timeout="5" connector="ajp">
Show
Decrease socket-timeout in mod-cluster-config to minimize impact of down proxies (1-5 seconds): <mod-cluster-config advertise-socket= "modcluster" proxies= "proxy1 proxy2" socket-timeout= "5" connector= "ajp" >
Git Pull Request:
https://github.com/modcluster/mod_cluster/pull/313
Steps to Reproduce:

Hide

1. Set up mod cluster with multiple proxies. You can further increase the socket-timeout if you wish to exacerbate the issue, but the default socket-timeout of 20 sufficed for me:

<mod-cluster-config advertise-socket="modcluster" proxies="proxy1 proxy2" socket-timeout="20" connector="ajp">

2. Add deployments. 5 was enough for concerns in my case.

3. Leave one or several of the configured mod_cluster proxies in a down/unavailable state. It would need to specifically be down in such a way where JBoss's connection attempts to it are not immediately refused. So you might have to set up some port that is listening but not responding.

4. Attempt to start JBoss and note slowness and likely the default 300 second blocking timeout will be exceeded.

Show
1. Set up mod cluster with multiple proxies. You can further increase the socket-timeout if you wish to exacerbate the issue, but the default socket-timeout of 20 sufficed for me: <mod-cluster-config advertise-socket="modcluster" proxies="proxy1 proxy2" socket-timeout="20" connector="ajp"> 2. Add deployments. 5 was enough for concerns in my case. 3. Leave one or several of the configured mod_cluster proxies in a down/unavailable state. It would need to specifically be down in such a way where JBoss's connection attempts to it are not immediately refused. So you might have to set up some port that is listening but not responding. 4. Attempt to start JBoss and note slowness and likely the default 300 second blocking timeout will be exceeded.

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

There's a timing issue between proxy reset requests and other MCMPs. This can allow for some severe start up delays if there is a configured proxy that is down or unresponsive. Start up is seen to be stalled as it tries to send enable-apps to the bad proxy:

"ServerService Thread Pool -- 72" #134 prio=5 os_prio=0 tid=0x00007f84581fa9f0 nid=0x6718 runnable [0x00007f83c1561000]
   java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
        at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
        at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
        - locked <0x00000000eb0a4700> (a java.io.InputStreamReader)
        at java.io.InputStreamReader.read(InputStreamReader.java:184)
        at java.io.BufferedReader.fill(BufferedReader.java:161)
        at java.io.BufferedReader.readLine(BufferedReader.java:324)
        - locked <0x00000000eb0a4700> (a java.io.InputStreamReader)
        at java.io.BufferedReader.readLine(BufferedReader.java:389)
        at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequest(DefaultMCMPHandler.java:529)
        at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequest(DefaultMCMPHandler.java:605)
        - locked <0x00000000fcaac598> (a org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler$Proxy)
        at org.jboss.modcluster.mcmp.impl.DefaultMCMPHandler.sendRequest(DefaultMCMPHandler.java:429)
        at org.jboss.modcluster.ModClusterService.enable(ModClusterService.java:375)
        at org.jboss.modcluster.ModClusterService.start(ModClusterService.java:361)
        at org.wildfly.mod_cluster.undertow.UndertowEventHandlerAdapter.onDeploymentStart(UndertowEventHandlerAdapter.java:136)
        - locked <0x00000000fcaac7a8> (a org.wildfly.mod_cluster.undertow.UndertowEventHandlerAdapter)
        at org.wildfly.extension.undertow.Host$1.invoke(Host.java:192)
        at org.wildfly.extension.undertow.UndertowService.fireEvent(UndertowService.java:249)
        - locked <0x00000000fcaa4598> (a java.util.Collections$SynchronizedList)
        at org.wildfly.extension.undertow.Host.registerDeployment(Host.java:189)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentService.startContext(UndertowDeploymentService.java:103)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentService$1.run(UndertowDeploymentService.java:82)

This can occur because a reset request places the proxy back in an OK state, which then allows the ENABLEs or any MCMP through.

incorporates

MODCLUSTER-639 proxy reset requests can allow for other MCMPs to bad proxy

Closed

is incorporated by

JBEAP-14269 [GSS](7.1.z) Update mod_cluster from 1.3.7.Final to 1.3.9.Final

Closed

Assignee:: Radoslav Husar

Reporter:: Aaron Ogburn

Tester:: Jiří Bílek (Inactive)

QA Contact:: Jiří Bílek (Inactive)

Votes:: 0 Vote for this issue

Watchers:: 6 Start watching this issue

Created:: 2018/01/24 3:48 PM

Updated:: 2021/10/24 5:20 AM

Resolved:: 2018/03/25 11:08 AM

Details

Description

Attachments

Issue Links

Easy Agile Planning Poker

Activity

People

Dates