Details
-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
-
AMQ 7.11.5.OPR.1.GA
-
None
-
False
-
None
-
False
Description
Initial descritpion:
1. Create a set of 4 brokers
2. Send 100000 messages to Broker #3 (last in statefulset)
3. Initiate scaledown
4. Disable Scaledown acceptor in Broker #0 (first in SS)
At this point after ENTMQBR-8580 is implemented what is supposed to happen is, all messages are migrated to Pod#1 (second in SS).
What happened in actuality was, 1000 messages were migrated to Pod#1, 2306 messages were migrated to Pod#2 and after that Drainer declared operation a success, deleting the PV in question, losing 97% of messages from the original Pod.
This seems to be caused by ENTMQBR-8640.
Drainer starts, sees that there are 0 messages to migrate (see 8640), but still starts Drainer broker. Drainer broker sends messages to Pod#1. If this process is interrupted, Drainer sees this as a completed operation,d espite stil having messages on AC+MC address. So it concludes that draining has been completed, even though not all messages have been migrated, therefore it finishes and removes the PVC, which resluts in message loss.
