Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-12172

Make the number of segments dynamic

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • 11.0.1.Final
    • Configuration, Core
    • None

    Description

      Now that the data container and stores store segment information, it's becoming harder and harder to change the number of segments to accommodate a growth in the number of nodes and preserve all the data.

      We should investigate removing the numSegments configuration attribute and using an effectiveNumSegments that is the first power of 2 that's bigger than 10 * <cluster size> (customizable, maybe?) and at least as big as the previous effectiveNumSegments.

      The number of segments increase would be decided by the coordinator on join and would require state transfer, as SyncConsistentHashFactory would not assign the split segments to exactly the same owners (although it would have to be pretty close in order to make this practical).

      We would like to avoid updating the segment of every persisted entry on effectiveNumSegments changes, so the segment stored in persisted entries should be computed based on the maximum value we will allow for effectiveNumSegments, e.g. 2^16. We would then use a shift+mask to obtain the "effective segment" of the key, and range queries when obtaining all the keys in a segment.

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            dberinde@redhat.com Dan Berindei (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: