Uploaded image for project: 'Infinispan'
  1. Infinispan
  2. ISPN-14514

SIFS results in "Too many records for this key" exception

    XMLWordPrintable

Details

    • Hide

      Compile and run this to reproduce:

      import org.infinispan.Cache;
import org.infinispan.configuration.cache.ConfigurationBuilder;
import org.infinispan.manager.DefaultCacheManager;
import org.infinispan.manager.EmbeddedCacheManager;

public class InfinispanBugReproducer {

  public static void test(Cache<String, String> cache) {
    for (int n = 0; n < 50000; n++) {
      cache.put("KeyOne", "ValueOne");
      cache.get("KeyOne");
      cache.remove("KeyOne");
    }
  }

  public static void main(String[] args) throws Exception {
    System.out.println("This is an attempt to demonstrates a possible bug.");

    try (EmbeddedCacheManager cacheManager = new DefaultCacheManager()) {

      System.out.println("Begin: perform the test with a simple cache");
      ConfigurationBuilder builder1 = new ConfigurationBuilder();
      builder1.simpleCache();
      cacheManager.defineConfiguration("default", builder1.build());
      Cache<String, String> cache1 = cacheManager.getCache("default");
      test(cache1);
      System.out.println("End: test with simple cache\n");

      System.out.println("Begin:  perform the test with a SIFS cache");
      ConfigurationBuilder builder2 = new ConfigurationBuilder();
      builder2
          .persistence()
          .addSoftIndexFileStore()
          .dataLocation("data")
          .indexLocation("index")
          .shared(false);
      cacheManager.defineConfiguration("sifs", builder2.build());
      Cache<String, String> cache2 = cacheManager.getCache("sifs");
      test(cache2);
      System.out.println("End: test with SIFS cache\n");
    }
    System.out.println("Complete.");
  }
}
      Show
      Compile and run this to reproduce: import org.infinispan.Cache; import org.infinispan.configuration.cache.ConfigurationBuilder; import org.infinispan.manager.DefaultCacheManager; import org.infinispan.manager.EmbeddedCacheManager; public class InfinispanBugReproducer { public static void test(Cache<String, String> cache) { for (int n = 0; n < 50000; n++) { cache.put("KeyOne", "ValueOne"); cache.get("KeyOne"); cache.remove("KeyOne"); } } public static void main(String[] args) throws Exception { System.out.println("This is an attempt to demonstrates a possible bug."); try (EmbeddedCacheManager cacheManager = new DefaultCacheManager()) { System.out.println("Begin: perform the test with a simple cache"); ConfigurationBuilder builder1 = new ConfigurationBuilder(); builder1.simpleCache(); cacheManager.defineConfiguration("default", builder1.build()); Cache<String, String> cache1 = cacheManager.getCache("default"); test(cache1); System.out.println("End: test with simple cache\n"); System.out.println("Begin: perform the test with a SIFS cache"); ConfigurationBuilder builder2 = new ConfigurationBuilder(); builder2 .persistence() .addSoftIndexFileStore() .dataLocation("data") .indexLocation("index") .shared(false); cacheManager.defineConfiguration("sifs", builder2.build()); Cache<String, String> cache2 = cacheManager.getCache("sifs"); test(cache2); System.out.println("End: test with SIFS cache\n"); } System.out.println("Complete."); } }

    Description

      I am opening this on behalf of a colleague who cannot reset his RedHat account.

      Seems as if the "remove" is removing from the cache but not from the index. As a result the index grows to have too many keys.

      Full stack trace:

      ERROR InvocationContextInterceptor ISPN000136: Error executing command RemoveCommand on Cache 'sifs', writing keys [KeyOne]
 java.lang.IllegalStateException: Too many records for this key (short overflow)
	at org.infinispan.persistence.sifs.IndexNode.copyWith(IndexNode.java:653)
	at org.infinispan.persistence.sifs.IndexNode.setPosition(IndexNode.java:380)
	at org.infinispan.persistence.sifs.Index$Segment.accept(Index.java:372)
	at org.infinispan.persistence.sifs.Index$Segment.accept(Index.java:242)
	at io.reactivex.rxjava3.internal.subscribers.LambdaSubscriber.onNext(LambdaSubscriber.java:65)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:404)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178)
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324)
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289)
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.base/java.lang.Thread.run(Thread.java:833)
Exception in thread "main" org.infinispan.commons.CacheException: java.lang.IllegalStateException: Too many records for this key (short overflow)
	at org.infinispan.interceptors.impl.InvocationContextInterceptor.rethrowException(InvocationContextInterceptor.java:135)
	at org.infinispan.interceptors.impl.InvocationContextInterceptor.lambda$new$0(InvocationContextInterceptor.java:61)
	at org.infinispan.interceptors.ExceptionSyncInvocationStage.andExceptionally(ExceptionSyncInvocationStage.java:40)
	at org.infinispan.interceptors.BaseAsyncInterceptor.invokeNextAndExceptionally(BaseAsyncInterceptor.java:133)
	at org.infinispan.interceptors.impl.InvocationContextInterceptor.visitCommand(InvocationContextInterceptor.java:89)
	at org.infinispan.interceptors.impl.AsyncInterceptorChainImpl.invoke(AsyncInterceptorChainImpl.java:247)
	at org.infinispan.cache.impl.InvocationHelper.doInvoke(InvocationHelper.java:297)
	at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:101)
	at org.infinispan.cache.impl.InvocationHelper.invoke(InvocationHelper.java:83)
	at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:686)
	at org.infinispan.cache.impl.CacheImpl.remove(CacheImpl.java:680)
	at org.infinispan.cache.impl.AbstractDelegatingCache.remove(AbstractDelegatingCache.java:454)
	at org.infinispan.cache.impl.EncoderCache.remove(EncoderCache.java:707)
	at InfinispanBugReproducer.test(InfinispanBugReproducer.java:12)
	at InfinispanBugReproducer.main(InfinispanBugReproducer.java:39)
Caused by: java.lang.IllegalStateException: Too many records for this key (short overflow)
	at org.infinispan.persistence.sifs.IndexNode.copyWith(IndexNode.java:653)
	at org.infinispan.persistence.sifs.IndexNode.setPosition(IndexNode.java:380)
	at org.infinispan.persistence.sifs.Index$Segment.accept(Index.java:372)
	at org.infinispan.persistence.sifs.Index$Segment.accept(Index.java:242)
	at io.reactivex.rxjava3.internal.subscribers.LambdaSubscriber.onNext(LambdaSubscriber.java:65)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:404)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178)
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324)
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289)
	at io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.base/java.lang.Thread.run(Thread.java:833)

      Attachments

        Issue Links

          duplicates

          Bug - A problem that impairs or prevents the functions of the product. ISPN-13547 SIFS Compaction error (short overflow)

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Resolved
          is related to

          Bug - A problem that impairs or prevents the functions of the product. DBZ-6495 Exception using oracle connector with Infinispan embeded IllegalStateException: Too many records for this key (short overflow)

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Coding In Progress

          Activity

            People

              wburns@redhat.com Will Burns
              david.everly David Everly (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: