-
Feature Request
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
-
False
-
-
PerDestinationBundler has 1 sender thread for each destination queue. Investigate whether replacing this with a single sender thread improves performance. The single sender thread iterates over all queues and sends messages where available.
This reduces the total number of threads and also reduces contention on the queues.
Comparison to TransferQueueBundler and existing PerDestinationBundler:
- P: number of producers, e.g. 100
- N: number of destinations, e.g. 10
| TQB | PDB | PDB2 | |
|---|---|---|---|
| contention on a single queue | P + 1: 101 | P/N+1: 11 | P/N + 1/N: 10 + 1/10 |
| total # of threads | P+1: 101 | P+N: 110 | P+1: 101 |
PDB2 reduces threads compared to PDB, as it only uses a single sender. Also, it only contends with producers roughly 10 percent of the time for the same queue.
The single sender thread in PDB2 tries to remove all messages from the queues, round-robing among them. When no message are available, it blocks and is awoken by a producer when adding new messages.
