Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-11679

Bad uneven segment distributions can happen after the rebalancing with SyncConsistentHashFactory

    Details

    • Type: Bug
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 9.4.19.Final, 10.1.6.Final
    • Fix Version/s: 11.0.0.Final
    • Component/s: Core
    • Labels:
      None
    • Steps to Reproduce:
      Hide
      • Add a unit test "testRebalanceDistribution" in SyncConsistentHashFactoryKeyDistributionTest.

      https://github.com/infinispan/infinispan/compare/master...msfm:master_SyncCHFKeyDistTest

      • Execute unit test
      $ mvn -pl core test -Dtest=distribution.ch.SyncConsistentHashFactoryKeyDistributionTest#testRebalanceDistribution
      
      • Check the test result, especially "Primary: max", "Primary: min" and "Segments per node - max/min ratio". You will see "Primary: min" that indicates around half (around 0.500-0.600) and "Segments per node - max/min ratio" indicates around two times (around 1.7-2.0).
      Show
      Add a unit test "testRebalanceDistribution" in SyncConsistentHashFactoryKeyDistributionTest. https://github.com/infinispan/infinispan/compare/master...msfm:master_SyncCHFKeyDistTest Execute unit test $ mvn -pl core test -Dtest=distribution.ch.SyncConsistentHashFactoryKeyDistributionTest#testRebalanceDistribution Check the test result, especially "Primary: max", "Primary: min" and "Segments per node - max/min ratio". You will see "Primary: min" that indicates around half (around 0.500-0.600) and "Segments per node - max/min ratio" indicates around two times (around 1.7-2.0).

      Description

      Uneven segment distribution can happen after rebalancing with SyncConsistentHashFactory.

      Even if a larger number of segments than "20 * number of nodes" is configured, some nodes have only around 50-60% number of segments than other nodes after the rebalance in the worst case.

      I understand the documentation (like the following) states about the possibility of slight uneven distribution as a potential drawback of SyncConsistentHashFactory.

      Infinispan 10 configuration guide says:

      This implementation does have some negative points in that the load distribution is slightly uneven. It also moves more segments than strictly necessary on a join or leave.

      JDG 7 Developer Guide says:

      It has a potential drawback in that it can move a greater number of segments than necessary during re-balancing. This can be mitigated by using a larger number of segments.
      Another potential drawback is that the segments are not distributed as evenly as possible, and actually using a very large number of segments can make the distribution of segments worse.

      However, only half segments than others is not a slight uneven.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  dan.berindei Dan Berindei
                  Reporter:
                  mmiura Masafumi Miura
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: