Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
1.7.0.Final
-
None
-
False
-
False
Description
When using the infinispan SPI with the Oracle connector against an NFS backed store it at some point in time crashes with the exception mentioned below. It's also not recommended by the Infinispan project to store the data in NFS: https://infinispan.org/docs/stable/titles/configuring/configuring.html#file_stores
The problem with the current implementation is that you are basically limiting the Debezium connector to a single node and it cannot run in a Kafkaconnect cluster which is not really what you want from an operational perspective.
UPDATE: This also occurs on an EBS backed volume
The only way to fix it is to delete the infinispan data and start with an empty directory the moment you hit this exception. The connector cannot restart once this error has occured.
[2021-10-07 08:10:20,727] INFO ISPN000556: Starting user marshaller 'org.infinispan.commons.marshall.ImmutableProtoStreamMarshaller' (org.infinispan.CONTAINER:36) [2021-10-07 08:10:20,822] ERROR Mining session stopped due to the {} (io.debezium.connector.oracle.logminer.LogMinerHelper:86) org.infinispan.commons.CacheConfigurationException: Error starting component org.infinispan.persistence.manager.PersistenceManager [snipped because the BasicComponentRegistryImpl repeats 5 times] at org.infinispan.factories.impl.BasicComponentRegistryImpl.startDependencies(BasicComponentRegistryImpl.java:622) at org.infinispan.factories.impl.BasicComponentRegistryImpl.doStartWrapper(BasicComponentRegistryImpl.java:586) at org.infinispan.factories.impl.BasicComponentRegistryImpl.startWrapper(BasicComponentRegistryImpl.java:564) at org.infinispan.factories.impl.BasicComponentRegistryImpl.access$700(BasicComponentRegistryImpl.java:30) at org.infinispan.factories.impl.BasicComponentRegistryImpl$ComponentWrapper.running(BasicComponentRegistryImpl.java:787) at org.infinispan.factories.AbstractComponentRegistry.internalStart(AbstractComponentRegistry.java:354) at org.infinispan.factories.AbstractComponentRegistry.start(AbstractComponentRegistry.java:250) at org.infinispan.factories.ComponentRegistry.start(ComponentRegistry.java:213) at org.infinispan.cache.impl.CacheImpl.start(CacheImpl.java:1015) at org.infinispan.cache.impl.AbstractDelegatingCache.start(AbstractDelegatingCache.java:512) at org.infinispan.manager.DefaultCacheManager.wireAndStartCache(DefaultCacheManager.java:698) at org.infinispan.manager.DefaultCacheManager.createCache(DefaultCacheManager.java:644) at org.infinispan.manager.DefaultCacheManager.internalGetCache(DefaultCacheManager.java:533) at org.infinispan.manager.DefaultCacheManager.getCache(DefaultCacheManager.java:511) at io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor.createCache(InfinispanLogMinerEventProcessor.java:136) at io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor.<init>(InfinispanLogMinerEventProcessor.java:115) at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.createProcessor(LogMinerStreamingChangeEventSource.java:205) at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:116) at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:54) at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:166) at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:127) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.util.concurrent.CompletionException: org.infinispan.persistence.spi.PersistenceException: ISPN000279: Failed to read stored entries from file. Error in file /bitnami/infinispan/transactions.dat at offset 140506 at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1739) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348) ... 1 more Caused by: org.infinispan.persistence.spi.PersistenceException: ISPN000279: Failed to read stored entries from file. Error in file /bitnami/infinispan/transactions.dat at offset 140506 at org.infinispan.persistence.file.SingleFileStore.rebuildIndex(SingleFileStore.java:254) at org.infinispan.persistence.file.SingleFileStore.start(SingleFileStore.java:177) at org.infinispan.persistence.support.NonBlockingStoreAdapter.lambda$start$0(NonBlockingStoreAdapter.java:107) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736) ... 5 more
Attachments
Issue Links
- is incorporated by
-
DBZ-4159 Store buffered events in separate Infinispan cache
-
- Closed
-
