Uploaded image for project: 'JGroups'
  1. JGroups
  2. JGRP-1223

STREAMING_STATE_TRANSFER abysmally slow

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 2.10
    • 2.8
    • None
    • Low
    • Hide

      Run two instances of StateTransferTest.java (see attached file) with the included config. After one minute the second instance will (typically) end up with an empty map:

      INFO JChannel - JGroups version: 2.8.0.GA

      -------------------------------------------------------------------
      GMS: address=WZUR4269297-5387, cluster=state-transfer-test, physical address=165.222.17.200:2599
      -------------------------------------------------------------------
      INFO ReplicatedHashMap - state could not be retrieved (first member)
      Loading state took: 60135ms
      Shared map has 0 entries

      With the included patch the state is loaded successfully:

      INFO JChannel - JGroups version: 2.8.0.GA

      -------------------------------------------------------------------
      GMS: address=WZUR4269297-19455, cluster=state-transfer-test, physical address=165.222.17.200:2654
      -------------------------------------------------------------------
      INFO ReplicatedHashMap - state was retrieved successfully, waiting for setState()
      INFO ReplicatedHashMap - setState() was called
      Loading state took: 1484ms
      Shared map has 10000 entries

      Show
      Run two instances of StateTransferTest.java (see attached file) with the included config. After one minute the second instance will (typically) end up with an empty map: INFO JChannel - JGroups version: 2.8.0.GA ------------------------------------------------------------------- GMS: address=WZUR4269297-5387, cluster=state-transfer-test, physical address=165.222.17.200:2599 ------------------------------------------------------------------- INFO ReplicatedHashMap - state could not be retrieved (first member) Loading state took: 60135ms Shared map has 0 entries With the included patch the state is loaded successfully: INFO JChannel - JGroups version: 2.8.0.GA ------------------------------------------------------------------- GMS: address=WZUR4269297-19455, cluster=state-transfer-test, physical address=165.222.17.200:2654 ------------------------------------------------------------------- INFO ReplicatedHashMap - state was retrieved successfully, waiting for setState() INFO ReplicatedHashMap - setState() was called Loading state took: 1484ms Shared map has 10000 entries

      With a relatively large shared state (anything in excess of a few MBytes) STREAMING_STATE_TRANSFER is very slow, it can take minutes for the state transfer to complete. The underlying issue is that STREAMING_STATE_TRANSFER.StreamingOutputStreamWrapper subclasses java.io.FilterOutputStream which always calls write(int b), even if buffers are to be written to the underlying socket.
      As a consequence each byte to be transferred results in a call to write() to the underlying socket.
      The fix is simple, the byte array variants of StreamingOutputStreamWrapper.write() should delegate to the wrapped socket directly.

        1. JGRP-1223.zip
          2 kB

              vblagoje Vladimir Blagojevic (Inactive)
              kelstner Kornelius Elstner (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: