Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4064

Infinispan SPI throws NPE with more than one connector configured to the same Oracle database

XMLWordPrintable

      We are using Oracle 19c with LOB disabled using the infinispan SPI. This runs fine with a single connector, but once we have more than one connector connecting to the same Oracle instance (to reduce the amount of tables handled by a single connector, thus by a single KafkaConnect instance), these random NPEs are appearing. Restarting the connector then resumes the work as normal up until it crashes again. This can happen like 20 times per day.

      We realize that running with more than one connector to the same database is not a recommended practice, this also made us decide to revert back to a single connector. This removes this exception altogether.

      It starts with this error:

      2021-09-22 10:24:47    java.lang.NullPointerException: Null keys are not supported!
      2021-09-22 10:24:47        at java.base/java.util.Objects.requireNonNull(Objects.java:246)
      2021-09-22 10:24:47        at org.infinispan.cache.impl.CacheImpl.assertKeyNotNull(CacheImpl.java:202)
      2021-09-22 10:24:47        at org.infinispan.cache.impl.CacheImpl.get(CacheImpl.java:541)
      2021-09-22 10:24:47        at org.infinispan.cache.impl.CacheImpl.containsKey(CacheImpl.java:526)
      2021-09-22 10:24:47        at org.infinispan.cache.impl.CacheImpl.containsKey(CacheImpl.java:522)
      2021-09-22 10:24:47        at org.infinispan.cache.impl.AbstractDelegatingCache.containsKey(AbstractDelegatingCache.java:384)
      2021-09-22 10:24:47        at org.infinispan.cache.impl.EncoderCache.containsKey(EncoderCache.java:601)
      2021-09-22 10:24:47        at io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor.processRow(InfinispanLogMinerEventProcessor.java:206)
      2021-09-22 10:24:47        at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processResults(AbstractLogMinerEventProcessor.java:150)
      2021-09-22 10:24:47        at io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor.process(InfinispanLogMinerEventProcessor.java:176)
      2021-09-22 10:24:47        at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:144)
      2021-09-22 10:24:47        at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:54)
      2021-09-22 10:24:47        at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:165)
      2021-09-22 10:24:47        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:127)
      2021-09-22 10:24:47        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      2021-09-22 10:24:47        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      2021-09-22 10:24:47        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      2021-09-22 10:24:47        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      2021-09-22 10:24:47        at java.base/java.lang.Thread.run(Thread.java:834)
      

      And 20 minutes or so later we get this exception:

      java.lang.NullPointerException
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processRow(AbstractLogMinerEventProcessor.java:162)
          at io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor.processRow(InfinispanLogMinerEventProcessor.java:210)
          at io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor.processResults(AbstractLogMinerEventProcessor.java:150)
          at io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor.process(InfinispanLogMinerEventProcessor.java:176)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:144)
          at io.debezium.connector.oracle.logminer.LogMinerStreamingChangeEventSource.execute(LogMinerStreamingChangeEventSource.java:54)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:165)
          at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:127)
          at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
          at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
          at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
          at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
          at java.base/java.lang.Thread.run(Thread.java:834)
      

      Our connector config is as follows:

      {
          "config": {
              "connector.class": "io.debezium.connector.oracle.OracleConnector",
              "snapshot.mode": "initial",
              "snapshot.locking.mode": "none",
              "tasks.max": "1",
              "database.server.name": "myserver",
              "database.url": "myjdbcstring",
              "database.dbname": "my_svc",
              "event.processing.failure.handling.mode": "warn",
              "topics": "mytopic",
              "log.mining.sleep.time.min.ms": "1000",
              "log.mining.sleep.time.max.ms": "10000",
              "log.mining.batch.size.min": "10000",
              "log.mining.batch.size.max": "200000",
              "log.mining.view.fetch.size": "20000",
              "log.mining.sleep.time.default.ms": "1000",
              "log.mining.sleep.time.increment.ms": "1000",
              "log.mining.archive.log.hours": "5",
              "log.mining.archive.log.only.mode": "false",
              "log.mining.archive.destination.name": "LOG_ARCHIVE_DEST_2",
              "log.mining.strategy": "online_catalog",
              "decimal.handling.mode": "string",
              "log.mining.buffer.type": "infinispan",
              "log.mining.buffer.location": "/data/infinispan/myconnector",
              "heartbeat.interval.ms": "60000",
              "provide.transaction.metadata": "true",
              "tombstones.on.delete": "false",
              "poll.interval.ms": "60000",
              "database.history.kafka.bootstrap.servers": "my-kafka-bootstrap-host.fqdn:9094",
              "database.history.kafka.topic": "myserver.history",
              "database.history.store.only.captured.tables.ddl": "true",
              "database.history.skip.unparseable.ddl": "true",
              "table.include.list": "MYSCHEMA.MYTABLE1,MYSCHEMA.MYTABLE2"
          }
      }
      

            ccranfor@redhat.com Chris Cranford
            jorn.argelo@mn.nl Jorn Argelo (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: