Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7958

ibmi as400 connector does not support incremental snapshots

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Unresolved
    • Icon: Major Major
    • Backlog
    • None
    • ibmi-connector
    • None
    • False
    • None
    • False

      What Debezium connector do you use and what version?

      debezium-connector-ibmi

      connector 2.7.0-snapshot

      What is the connector configuration?

      {

      "name": "F63QULDVES-2",

      "config":

      { "connector.class": "io.debezium.connector.db2as400.As400RpcConnector", "tasks.max": "1", "database.hostname": "databasehost", "database.dbname": "dbname", "port": "", "database.user": "user", "database.password": "password", "database.schema": "schema", "poll.interval.ms": 2000, "snapshot.mode": "initial", "table.include.list": "schema.table1", "transforms": "unwrap,schemarename", "transforms.schemarename.type": "com.fnz.kakfa.smt.RepackageJavaFriendlySchemaRenamer", "transforms.schemarename.package": "com.fnz.debezium.schema", "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState", "transforms.unwrap.drop.tombstones": "false", "transforms.unwrap.delete.handling.mode": "rewrite", "key.converter.schema.registry.url": "http://mskafka:8081", "value.converter.schema.registry.url": "http://mskafka:8081", "key.converter": "io.confluent.connect.avro.AvroConverter", "value.converter": "io.confluent.connect.avro.AvroConverter", "snapshot.max.threads": 4, "topic.creation.default.cleanup.policy": "compact", "topic.creation.default.retention.ms": -1, "topic.creation.default.replication.factor": 1, "topic.creation.default.partitions": 1, "field.name.adjustment.mode": "avro", "topic.prefix": "databasehost", "signal.enabled.channels": "source,kafka", "signal.kafka.topic": "scehma-1-signal", "signal.data.collection": "schema.DEBEZIUM_SIGNAL", "signal.kafka.bootstrap.servers": "mskafka:9092" }

      }

       

       

      What is the captured database version and mode of depoyment?

      gcp

      What behaviour do you expect?

      snapshot tables as per signal

      What behaviour do you see?

      exception:
      io.debezium.DebeziumException: Incremental snapshot is not properly configured, either sinalling data collection is not provided or connector-specific snapshotting not set

      Do you see the same behaviour using the latest relesead Debezium version?

      yes

      Do you have the connector logs, ideally from start till finish?

      can be provided

      How to reproduce the issue using our tutorial deployment?

      send following to kafla

      key: databsehost

      value: {
          "type": "execute-snapshot",
          "data":

      {         "data-collections": [             "schema.table1"         ],         "type": "INCREMENTAL"     }

      }

      Feature request or enhancement

      Which use case/requirement will be addressed by the proposed feature?

      snapshot support

       

      The connector currently uses the old depricated way that I found in mysql where the table list is stored in the offsets

              Unassigned Unassigned
              msillencefnz Martin Sillence
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: