Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-13576

SoftIndexFileStore segmentation can be made more performant

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Major
    • 14.0.0.Dev02
    • 13.0.0.Final
    • Loaders and Stores
    • None

    Description

      SoftIndexFileStore added segmentation support in ISPN-12822. However, the initial implementation is not the best and can use more memory that desired, see https://github.com/infinispan/infinispan/blob/4e317d144ea85667a8d95d5cbd407e236ab1034a/core/src/main/java/org/infinispan/persistence/sifs/IndexNode.java#L1118

      We should instead
      1. Prefix the key with the segment. Since the Index B+ tree is sorted, this means entries will be sorted by segment. This will allow iterating over only a small subset of nodes.

      2. Change the publish method to search for the first leaf that can have the given segment and allow for paging inside. That is that it should return entries sorted so it can start over in in the middle of iteration. This will reduce the amount of memory from buffering, since currently we have to fetch all entries for a given cache segment in the index segment.

      3. With #2 we can reduce the number of Index segments (note different than cache segments) to 3 as it was originally. This will reduce the number of files as well.

      4. Need to update the index initial bytes so upon restart they know to migrate to the new index.

      Attachments

        Activity

          People

            wburns@redhat.com Will Burns
            wburns@redhat.com Will Burns
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: