Receive buffers are now reused in UDP, TCP and TCP_NIO2, to prevent high memory allocation rates.

Status

• TP.receive() now de-serializes both single messages and message batches
• TCP uses a pre-allocated buffer for each connection (TcpConnection), which grows on demand, rather than creating a new one for each message. This reduced memory allocation (UPerf, 4 nodes, 2M msgs/node) by ca. 1.8GB
• TCP_NIO2 now also pre-allocates a buffer (in NioConnection) and grows it on demand

UDP

• The same buffer (66000 bytes) is reused
• Message batches are de-serialized off of the UDP buffer and passed to a thread pool for further processing. No copy
• Single messages: the buffer is copied and passed to the thread pool where de-serialization happens
  • If we made processing of single messsages similar to that of message batches, we could get rid of the buffer copy: do the de-serialization directly in the TP.receive() method and pass the message to the thread pool for further processing
    • Look at whether this hampers performance
    • Should not be the case as we're already de-serializing message batches in the TP.receive() method
    • Most of the time (when receiving a lot of messages), messages will be received as message batches, not single messages anyway, so de-serialization of single messages shouldn't matter (perf-wise)

TCP

• Every connection creates a new buffer for every incoming message, based on the length which is sent first
• I think we can safely switch to reusable buffers as both single message and message batches get de-serialized in TP.receive() and therefore implicitly already make a copy.
  • Alternative 1: every TcpConnection has its own buffer. On reception of length, if the buffer is too small, we grow it. Downside: large buffers tend to hang around. Upside: idle connections are reaped, gc'ing the buffer. Also, buffer sizes are never more than TP.max_bundle_size (plus some constant overhead) anyway. The other downside is that we keep 1 buffer per connection, which means buffer-size * members overhead; not nice in a large cluster...
  • Alterntive 2: use a buffer pool. Buffers are only used for a brief period of time: to de-serialize single messages or message batches. A fixed-size pool might be a more memory efficient solution, especially in large clusters. On reception of length, a connection grabs a right-sized buffer from the pool and returns it after de-serialization and passing of the de-serialized message to a thread pool. When the buffer is exhausted, a new temp buffer will be created.

TCP_NIO2

• Every connection has a Buffers instance for reading. On reception of length, a new buffer for the message is created.
• Similar to TCP, we could hang on to that buffer and grow it if needed, or use a buffer pool (see above)