Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8453

Debezium db2i CDC source connector does not seem to pickup JOURNAL_ENTRY_TYPES => 'DR' records

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 3.0.4.Final
    • None
    • ibmi-connector
    • None
    • False
    • None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      3.0.1.Final

      What is the connector configuration?

      {
          "connector.class" : "io.debezium.connector.db2as400.As400RpcConnector",
          "name" : "connector-name",
          "sanitize.field.names" : "true",
          "tasks.max" : "1",
          "database.hostname" : "AS400",
          "database.user" : "user",
          "database.password" : "pw",
          "database.port" : "8471",
          "database.dbname" : "DB",
          "database.schema" : "SCHEMA",
          "driver.secure" : false,
          "poll.interval.ms" : "5000",
          "table.include.list" : "SCHEMA.TABLE",
          "snapshot.mode" : "schema_only",
          "key.converter.schema.registry.url" : "http://schema-registry.prod.com/",
          "value.converter.schema.registry.url" : "http://schema-registry.prod.com/",
          "key.converter" : "io.confluent.connect.avro.AvroConverter",
          "value.converter" : "io.confluent.connect.avro.AvroConverter",
          "socket timeout" : "300000",
          "keep.alive" : "true",
          "topic.prefix" : "db2.cdc",
          "topic.creation.default.partitions" : 12,
          "topic.creation.default.replication.factor" : 4,
          "topic.creation.default.cleanup.policy" : "delete,compact",
          "transforms" : "InsertField",
          "transforms.InsertField.type" : "org.apache.kafka.connect.transforms.InsertField$Value",
          "transforms.InsertField.static.field" : "event_source",
          "transforms.InsertField.static.value" : "DB.SCHEMA.TABLE",
          "max.journal.timeout" : "300000",
          "header.converter": "org.apache.kafka.connect.storage.StringConverter"
        }

      What is the captured database version and mode of deployment?

      7.4, on-premise

      What behavior do you expect?

      All delete events get picked up

      What behavior do you see?

      After analysis of some kafka topics created with the Debezium db2i cdc connector, I have noticed some missing events in the kafka topic, which are actually some delete events.

      The events that were not picked up are the JOURNAL_ENTRY_TYPE = 'DR' events (the DL events get picked up successfully).

      https://www.ibm.com/docs/en/i/7.4?topic=information-all-journal-entries-by-code-type

      Do you see the same behaviour using the latest released Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      Yes

      Do you have the connector logs, ideally from start till finish?

      Not yet at least. No error appears in the logs as far as i can tell

       

       

              Unassigned Unassigned
              wltmlx Lukas Langegger
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: