-
Bug
-
Resolution: Won't Do
-
Major
-
None
-
None
-
None
When running any of the qpid-interop-tests against the Artemis broker, it consistently works for the first test, but hangs on the second:
[kvdr@localhost RedHat]$ python -m qpid_interop_test.amqp_types_test WARNING: AMQP DotNetLite shims not installed Test Broker: apache-activemq-artemis v.2.5.0-SNAPSHOT on <platform not found> test_binary_ProtonCpp->ProtonCpp (__main__.BinaryTestCase) ... ok test_binary_ProtonCpp->ProtonPython (__main__.BinaryTestCase) ...
and behaves the same way for all of the interop tests.
In all qpid-interop-tests, the client receiver is first connected followed immediately by the client sender on separate threads. This means that the client is not waiting for the broker to respond to the receiver's connection before establishing the senders connection.
During a previous investigation of similar behavior, it was thought that the problem might lie in the auto-queue creation logic of the broker, and that pre-defining the queues would solve the problem (as a work-around). However, I have added all the queues used by qpid_interop_test.amqp_types_test to the broker.xml file (attached), and these are being deployed by the broker on startup:
2018-01-08 11:15:56,138 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying queue jms.queue.qpid-interop.amqp_types_test.binary.ProtonCpp.ProtonCpp on address jms.queue.qpid-interop.amqp_types_test.binary.ProtonCpp.ProtonCpp 2018-01-08 11:15:56,141 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying queue jms.queue.qpid-interop.amqp_types_test.binary.ProtonCpp.ProtonPython on address jms.queue.qpid-interop.amqp_types_test.binary.ProtonCpp.ProtonPython 2018-01-08 11:15:56,144 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying queue jms.queue.qpid-interop.amqp_types_test.binary.ProtonCpp.RheaJs on address jms.queue.qpid-interop.amqp_types_test.binary.ProtonCpp.RheaJs 2018-01-08 11:15:56,148 INFO [org.apache.activemq.artemis.core.server] AMQ221003: Deploying queue jms.queue.qpid-interop.amqp_types_test.binary.ProtonPython.ProtonCpp on address jms.queue.qpid-interop.amqp_types_test.binary.ProtonPython.ProtonCpp ...
but appears to have no effect on this issue.
These tests work correctly against all other brokers (ActiveMQ, Qpid-cpp and Disptach single-node).
An analysis of the AMQP traffic for the qpid-cpp shim (see attached wireshark capture) shows that in each case, on the second test (or occasionally a few more tests will complete successfully before the hang), after the receiver sends its AMQP flow performative, the broker stops responding. It should at this point start transferring messages to the receiver.
In the wireshark file attached, this can be clearly seen using only the Proton C++ shim. In this test, the first four tests succeeded, and the fifth hung. Looking at the fifth test (starting in frame 176):
Frame 176: Sender connects to broker with AMQP open, begin, attach
Frame 181: Receiver connects to broker with AMQP open, begin, attach
Frame 183: Broker responds to sender with AMQP open
Frame 185: Broker responds to sender with AMQP begin, attach, flow
Frame 187: Sender sends 4 test messages to broker (AMQP transfer x4)
Frame 188: Broker sends 4 dispositions (AMQP disposition x4)
Frame 189: Sender closes (AMQP close)
Frame 190: Broker closes (AMQP close)
The traffic between the sender and broker appears normal. However, the receiver traffic from frame 181 has not yet been responded to in any way.
Frame 194: Broker responds to receiver with AMQP open
Frame 196: Broker responds to receiver with AMQP begin, attach
Frame 198: Receiver sends broker AMQP flow
The receipt of flow should allow the broker to start the transfer of its messages, but instead, it does not produce any further traffic.
If I use the PN_TRACE_FRM=1 environment variable when running Artemis, I can see that the AMQP flow from the receiver to the broker is the last AMQP packet received at a Proton level.