Details
-
Bug
-
Resolution: Duplicate
-
Major
-
None
-
7.0.3.GA, 7.0.4.GA
Description
Scenario:
Solaris
1. Undertow as balancer (EAP-7.1.0.DR6)
2. 2 EAP workers (EAP-7.1.0.DR6)
3. App present on both workers
Issue
1.Send few requests via balancer to app
2. shutdown one of the workers
3. Send few request via balancer to the app
Expected
All requests are handled
Now
Few request did not get any response and timeout
Reproducer:
repro.py
is able to use few threads to sent multiple requests and count onces that were handled
Usage is simple, save reproducer ideally to EAP bin folder where jboss-cli.sh is.
Use command " python repro.py -u URL_TO_APP_VIA_BALANCER -s 100 -r 100 -a ADDRESS_OF_ONE_WORKER_MANAGEMENT"
Example:
python repro.py -u http://10.16.89.253:8484/clusterbench/jvmroute -s 100 -r 100 -a 10.16.89.253:10091
Reproducer will send few requests, then shutdown via jboss-cli.sh one of the selected workers and then send few requests one more time and always counting sent vs handled
+Verbose stuff, probably not relevant, but there is info about scenario where curl was handled correctly but testsuite request wasn't +
Undertow as balancer seems to not get message sent via htmlutil java library but new traffic suggest that message was sent. Curl request was correctly handled. There is no info about what happened with htmlutil request.
Curl:
curl, http://10.16.89.253:8484/clusterbench/requestinfo
Eap log:
07:17:38,332 DEBUG [io.undertow.server.handlers.proxy] (default I/O-5) Sending request ClientRequest{path='/clusterbench/requestinfo', method=GET, protocol=HTTP/1.1} to target 10.16.89.253 for exchange HttpServerExchange{ GET /clusterbench/requestinfo request {Accept=[*/*], User-Agent=[curl/7.21.2 (i386-pc-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.1m zlib/1.2.3-T4mods libidn/1.19], Host=[10.16.89.253:8484]} response {X-Powered-By=[Undertow/1], Server=[JBoss-EAP/7]}}
07:17:38,332 DEBUG [io.undertow.server.handlers.proxy] (default I/O-5) Sent request ClientRequest{path='/clusterbench/requestinfo', method=GET, protocol=HTTP/1.1} to target 10.16.89.253 for exchange HttpServerExchange{ GET /clusterbench/requestinfo request {Accept=[*/*], User-Agent=[curl/7.21.2 (i386-pc-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.1m zlib/1.2.3-T4mods libidn/1.19], Host=[10.16.89.253:8484]} response {X-Powered-By=[Undertow/1], Server=[JBoss-EAP/7]}}
07:17:38,333 DEBUG [io.undertow.request] (default I/O-5) suspending writes on io.undertow.protocols.ajp.AjpClientRequestClientStreamSinkChannel@1e436c4d to prevent listener runaway
Response:
Path info: null
Query string: null
Query string UTF-8 decoded: null
Remote user: null
Parameters [key=value]: {}
Headers: {Accept=*/*, User-Agent=curl/7.21.2 (i386-pc-solaris2.11) libcurl/7.21.2 OpenSSL/1.0.1m zlib/1.2.3-T4mods libidn/1.19, X-Forwarded-Proto=http, X-Forwarded-Host=10.16.89.253, X-Forwarded-For=10.16.89.253, Host=10.16.89.253:8484, Content-Length=0, X-Forwarded-Port=8484, X-Forwarded-Server=10.16.89.253}
Host header: 10.16.89.253:8484
Character encoding: null
JVM route: jboss-eap-7.1-2
Session ID: odmpazEJvW3afPK7qzJydUorRDgmp9EgQApLYkWf
Session isNew: true
Session ServletContext: io.undertow.servlet.spec.ServletContextImpl@d4d298c
Servlet specific info: Session tracking mode: [URL, COOKIE]
Traffic:
07:17:38.333273 IP 10.16.89.253.57963 > 10.16.89.253.8110: Flags [P.], seq 1:375, ack 1, win 34816, options [nop,nop,TS val 2923858362 ecr 2923858362], length 374
0x0000: 0102 0002 0000 01aa 0000 0002 0000 0000 ................
0x0010: 0000 0009 0000 0009 4500 01aa 0000 4000 ........E.....@.
0x0020: 4006 0000 0a10 59fd 0a10 59fd e26b 1fae @.....Y...Y..k..
0x0030: 68b5 96fa 43f1 a11d 8018 8800 0196 0000 h...C...........
0x0040: 0101 080a ae46 89ba ae46 89ba 1234 0172 .....F...F...4.r
0x0050: 0202 0008 4854 5450 2f31 2e31 0000 192f ....HTTP/1.1.../
0x0060: 636c 7573 7465 7262 656e 6368 2f72 6571 clusterbench/req
0x0070: 7565 7374 696e 666f 0000 0000 000c 3130 uestinfo......10
0x0080: 2e31 362e 3839 2e32 3533 0000 0c31 302e .16.89.253...10.
0x0090: 3136 2e38 392e 3235 3300 2124 0000 09a0 16.89.253.!$....
0x00a0: 0100 032a 2f2a 0000 1258 2d46 6f72 7761 ...*/*...X-Forwa
0x00b0: 7264 6564 2d53 6572 7665 7200 000c 3130 rded-Server...10
0x00c0: 2e31 362e 3839 2e32 3533 00a0 0e00 5d63 .16.89.253....]c
0x00d0: 7572 6c2f 372e 3231 2e32 2028 6933 3836 url/7.21.2.(i386
0x00e0: 2d70 632d 736f 6c61 7269 7332 2e31 3129 -pc-solaris2.11)
0x00f0: 206c 6962 6375 726c 2f37 2e32 312e 3220 .libcurl/7.21.2.
0x0100: 4f70 656e 5353 4c2f 312e 302e 316d 207a OpenSSL/1.0.1m.z
0x0110: 6c69 622f 312e 322e 332d 5434 6d6f 6473 lib/1.2.3-T4mods
0x0120: 206c 6962 6964 6e2f 312e 3139 0000 1158 .libidn/1.19...X
0x0130: 2d46 6f72 7761 7264 6564 2d50 726f 746f -Forwarded-Proto
0x0140: 0000 0468 7474 7000 0010 582d 466f 7277 ...http...X-Forw
0x0150: 6172 6465 642d 506f 7274 0000 0438 3438 arded-Port...848
0x0160: 3400 000f 582d 466f 7277 6172 6465 642d 4...X-Forwarded-
0x0170: 466f 7200 000c 3130 2e31 362e 3839 2e32 For...10.16.89.2
0x0180: 3533 00a0 0800 0130 00a0 0b00 1131 302e 53.....0.....10.
0x0190: 3136 2e38 392e 3235 333a 3834 3834 0000 16.89.253:8484..
0x01a0: 1058 2d46 6f72 7761 7264 6564 2d48 6f73 .X-Forwarded-Hos
0x01b0: 7400 000c 3130 2e31 362e 3839 2e32 3533 t...10.16.89.253
0x01c0: 00ff ..
continued with resend to worker and response
Java library
com.gargoylesoftware.htmlunit.WebClient
07:17:38.374 [FINE] Retrieving clusterbench/jvmroute content
Response: Nothing
Eap log about this event: Nothing
Traffic:
07:17:38.377374 IP 10.16.89.253.63827 > 10.16.89.253.8484: Flags [P.], seq 618:965, ack 1, win 34816, options [nop,nop,TS val 2923858366 ecr 2923855993], length 347
0x0000: 0102 0002 0000 018f 0000 0002 0000 0000 ................
0x0010: 0000 0009 0000 0009 4500 018f 0000 4000 ........E.....@.
0x0020: 3c06 0000 0a10 59fd 0a10 59fd f953 2124 <.....Y...Y..S!$
0x0030: 4790 ecb2 7f4b 4292 8018 8800 017b 0000 G....KB......{..
0x0040: 0101 080a ae46 89be ae46 8079 4745 5420 .....F...F.yGET.
0x0050: 2f63 6c75 7374 6572 6265 6e63 682f 6a76 /clusterbench/jv
0x0060: 6d72 6f75 7465 2048 5454 502f 312e 310d mroute.HTTP/1.1.
0x0070: 0a55 7365 722d 4167 656e 743a 204d 6f7a .User-Agent:.Moz
0x0080: 696c 6c61 2f34 2e30 2028 636f 6d70 6174 illa/4.0.(compat
0x0090: 6962 6c65 3b20 4d53 4945 2038 2e30 3b20 ible;.MSIE.8.0;.
0x00a0: 5769 6e64 6f77 7320 4e54 2036 2e30 290d Windows.NT.6.0).
0x00b0: 0a41 6363 6570 743a 2069 6d61 6765 2f67 .Accept:.image/g
0x00c0: 6966 2c20 696d 6167 652f 6a70 6567 2c20 if,.image/jpeg,.
0x00d0: 696d 6167 652f 706a 7065 672c 2069 6d61 image/pjpeg,.ima
0x00e0: 6765 2f70 6a70 6567 2c20 2a2f 2a0d 0a41 ge/pjpeg,.*/*..A
0x00f0: 6363 6570 742d 456e 636f 6469 6e67 3a20 ccept-Encoding:.
0x0100: 677a 6970 2c20 6465 666c 6174 650d 0a41 gzip,.deflate..A
0x0110: 6363 6570 742d 4c61 6e67 7561 6765 3a20 ccept-Language:.
0x0120: 656e 2d75 730d 0a48 6f73 743a 2031 302e en-us..Host:.10.
0x0130: 3136 2e38 392e 3235 333a 3834 3834 0d0a 16.89.253:8484..
0x0140: 436f 6e6e 6563 7469 6f6e 3a20 4b65 6570 Connection:.Keep
0x0150: 2d41 6c69 7665 0d0a 436f 6f6b 6965 3a20 -Alive..Cookie:.
0x0160: 4a53 4553 5349 4f4e 4944 3d6a 7244 6434 JSESSIONID=jrDd4
0x0170: 396f 4a73 2d71 3443 6a44 6b64 4b61 7876 9oJs-q4CjDkdKaxv
0x0180: 5838 646f 5252 3765 5879 3750 3955 6e49 X8doRR7eXy7P9UnI
0x0190: 4937 642e 6a62 6f73 732d 6561 702d 372e I7d.jboss-eap-7.
0x01a0: 312d 310d 0a0d 0a 1-1....
Continued with nothing
Attachments
Issue Links
- clones
-
-
- Closed
-
- is blocked by
-
WFLY-7392 Add max-retries to Undertow reverse proxy
-
- Closed
-
- is duplicated by
-
-
- Closed
-
