Uploaded image for project: 'ModeShape'
  1. ModeShape
  2. MODE-2387

MapDb configuration settings for LocalIndexProvider

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Major Major
    • 4.2.0.Final
    • 4.1.0.Final
    • Performance, Query
    • None

      See also discussion here https://developer.jboss.org/thread/250242

      MapDb configuration in LocalIndexProvider vastly (on some systems by 10k%) affects query and save performance. I created a performance test and changed LocalIndexProvider to include all tested settings. You may run the test after playing around with them with

      mvn -s ../settings.xml -Dcheckstyle.skip -Dtest=LocalIndexProviderTest#mapDbPerformanceTest test
      

      On a linux system having a SSD it looks like this:

      NUM_NODETYPES = 10;
      NUM_INDEXED_PROPERTIES = 10;
      NUM_ITERATIONS = 500;
      QUERIES_PER_ITERATION = 2;
      
      Ops per second based on settings combinations:
      
      standard
      62
      
      asyncWriteEnable()
      <1	
      
      maker = maker.asyncWriteEnable();
      maker = maker.asyncWriteFlushDelay(500);
      <<<1	
      
      maker = maker.cacheLRUEnable();
      61
      
      maker = maker.freeSpaceReclaimQ(1);
      63
      
      maker = maker.freeSpaceReclaimQ(10);
      61
      
      maker = maker.mmapFileEnable();
      119
      
      maker = maker.commitFileSyncDisable();
      99
      
      maker = maker.transactionDisable();
      6
      
      maker = maker.cacheLRUEnable();
      maker = maker.mmapFileEnable();
      maker = maker.commitFileSyncDisable();
      366
      

      The difference is already considerable but nothing compared to the performance difference on Windows. I couldn't run all tests there but performance boost using mmap and syncDisable was enormous. I guess Linux caches disk I/O way better.

      Things get even worse when not using an SSD but a HDD.

      Interestingly the transactionDisable and async settings only make things worse, which feels counterintuitive.

      mmapFileEnable might cause problems on 32 bit systems and commitFileSyncDisable may destroy the index if not properly shutdown.

      Both drawbacks might or might not be acceptable. For me they totally are, I already managed to corrupt the index without those settings in the past and simply deleting it will rebuild it anyway. IOW I don't care if the index gets corrupted after a crash as it is just an index. I could even delete it at every start.

      My suggestion is to add configurations settings in the json config for the LocalIndexProvider implementation so that everyone may decide what settings to use.

              hchiorean Horia Chiorean (Inactive)
              bes82 Bjoern Schmidt (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: