Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1985

Stop processing new commitlogs in cdc folder

    XMLWordPrintable

Details

    Description

      I was able to reproduce issues met by Ahmed and reported to the mailing list. Here are my findings to help solving the issue.

      First, I got an NPE

      15:05:33.383 [pool-4-thread-3] ERROR i.d.c.c.CassandraConnectorTask - Encountered exception while running Commit Log Post-Processor; stopping all processors in Cassandra Connector Task
      java.lang.NullPointerException: null
              at java.util.Arrays.sort(Arrays.java:1438) ~[na:1.8.0_242]
              at io.debezium.connector.cassandra.CommitLogPostProcessor.process(CommitLogPostProcessor.java:41) ~[debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.AbstractProcessor.start(AbstractProcessor.java:61) ~[debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.CassandraConnectorTask$ProcessorGroup.lambda$start$0(CassandraConnectorTask.java:200) ~[debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_242]
              at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_242]
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_242]
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_242]
              at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_242]
      

      Second, if you follow my procedure (based on Ahmed's with a few minor changes to make it work and be reproducible faster) you'll end up with some commitlogs processed and other sitting in the cdc folder and not being picked up by dbz. Then if debezium is restarted, those files are processed but it happens again with following commitlogs. See attached logs:

      • start1 for first start of dbz
      • start2 for second start of dbz

      Finally, I saw in the logs that the connector complains about Futures.transformAsync missing

      15:20:55.917 [main] DEBUG c.d.driver.core.GuavaCompatibility - Error while checking existence of method Futures.transformAsync
      java.lang.NoSuchMethodException: com.google.common.util.concurrent.Futures.transformAsync(com.google.common.util.concurrent.ListenableFuture, com.google.common.util.concurrent.AsyncFunction)
              at java.lang.Class.getMethod(Class.java:1786) ~[na:1.8.0_242]
              at com.datastax.driver.core.GuavaCompatibility.methodExists(GuavaCompatibility.java:253) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at com.datastax.driver.core.GuavaCompatibility.isGuava_19_0_OrHigher(GuavaCompatibility.java:225) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at com.datastax.driver.core.GuavaCompatibility.selectImplementation(GuavaCompatibility.java:131) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at com.datastax.driver.core.GuavaCompatibility.<clinit>(GuavaCompatibility.java:52) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at com.datastax.driver.core.Cluster.<clinit>(Cluster.java:71) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.CassandraClient.<init>(CassandraClient.java:53) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.CassandraClient.<init>(CassandraClient.java:48) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.CassandraConnectorContext.<init>(CassandraConnectorContext.java:39) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.CassandraConnectorTask.run(CassandraConnectorTask.java:72) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
              at io.debezium.connector.cassandra.CassandraConnectorTask.main(CassandraConnectorTask.java:61) [debezium-connector-cassandra-1.1.0.Final-jar-with-dependencies.jar:1.1.0.Final]
      

      I don't think it relates to the current issue but having them might be important for performance (if it's being used somehow) so I prefer adding it here too just in case.

      Attachments

        1. debezium.stdout-start1.log.gz
          25 kB
        2. debezium.stdout-start2.log.gz
          22 kB
        3. local_log
          664 kB

        Issue Links

          Activity

            People

              zhou.bing@husky.neu.edu Bingqin Zhou
              cscetbon cyril Scetbon (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: