Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7946

JdbcSinkTask doesn't clear offsets on stop

XMLWordPrintable

    • Important

       

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      debezium-connector-jdbc latest

      What is the connector configuration?

      Any

      What is the captured database version and mode of depoyment?

      Any

      What behaviour do you expect?

      I expect the close method in JdbcSinkConnectorTask to clear the topic partitions offsets when it it closed as it might get re-assigned with different set of topic partitions. The re-assignment can happen due to re-balance. The task shouldn't be holding on to old topic partitions. 

      SinkTask doc reference

      Other sink connectors which handle this:

      1. Confluent S3 Sink connector
      2. Adobe S3 Sink connector

      What behaviour do you see?

      If the topic partitions weren't cleared, it results in below error,

      Ignoring invalid task provided offset topicXYZ-1 OffsetAndMetadata{offset=4966, leaderEpoch=null, metadata=''} -- partition not assigned, assignment=[topicXYZ-1] (org.apache.kafka.connect.runtime.WorkerSinkTask:434) 

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes

      Do you have the connector logs, ideally from start till finish?

      Added above

      How to reproduce the issue using our tutorial deployment?

       

      Reproduced using unit-test by asserting the contents of sinkTask.offsets.

              ccranfor@redhat.com Chris Cranford
              blacktooth1234 Ravindranath Kakarla (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: