Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7249

Notifications are Missing the ID field in log channel

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Major Major
    • 2.6.0.Alpha1
    • None
    • core-library
    • None

      Bug report

      I found that the id field is missing from the log entries via the notifications channel. This field is useful for incremental snapshots to track/link the notification to the snapshot signal that was sent. Please see the attached screenshots of the id field appearing a topic (via the sink notification channel) but not in the log (via the log notification channel). 

      See forum post here. https://groups.google.com/g/debezium/c/UcSWv_bIjJo

      What Debezium connector do you use and what version?

      MySqlConnector version 2.4

      What is the connector configuration?

      {     "connector.class": "io.debezium.connector.mysql.MySqlConnector",     "snapshot.locking.mode": "none",     "errors.log.include.messages": "false",     "transforms.unwrap.delete.handling.mode": "rewrite",     "incremental.snapshot.chunk.size": "1024",     "transforms": "unwrap",     "signal.enabled.channels": "kafka",     "schema.history.internal.store.only.captured.databases.ddl": "false",     "schema.history.internal.store.only.captured.tables.ddl": "false",     "topic.prefix": "abc",     "binary.handling.mode": "bytes",     "schema.history.internal.kafka.topic": "abc.schema_changes",     "transforms.unwrap.drop.tombstones": "true",     "transforms.unwrap.type": "io.debezium.transforms.ExtractNewRecordState",     "value.converter": "io.confluent.connect.avro.AvroConverter",     "errors.log.enable": "true",     "key.converter": "io.confluent.connect.avro.AvroConverter",     "database.user": "-",     "signal.kafka.bootstrap.servers": "-",     "database.server.id": "-",     "signal.kafka.topic": "abc.signal",     "read.only": "true",     "notification.enabled.channels": "sink,log",     "schema.history.internal.kafka.bootstrap.servers": "-",     "transforms.unwrap.field.name.adjustment.mode": "avro",     "transforms.unwrap.add.headers": "op",     "database.port": "-",     "notification.sink.topic.name": "default.snapshot_notifications",     "value.converter.schema.registry.url": "-",     "producer.override.max.request.size": "16777216",     "errors.max.retries": "5",     "database.connectionTimeZone": "SERVER",     "database.hostname": "-",     "database.password": "-",     "schema.name.adjustment.mode": "avro",     "name": "abc",     "errors.tolerance": "all",     "max.batch.size": "2048",     "table.include.list": "-",     "key.converter.schema.registry.url": "-",     "snapshot.mode": "when_needed" }

      What is the captured database version and mode of depoyment?

      AWS RDS - MySQL Community 8.0.28

      What behaviour do you expect?

      The notification id should appear in the notification logs.

      What behaviour do you see?

      The notification logs are generated but they do not contain the id field.

      Do you have the connector logs, ideally from start till finish?

      no

      How to reproduce the issue using our tutorial deployment?

      Create a mysql connector, setting notification.enabled.channels to log. Execute an incremental snapshot.

              Unassigned Unassigned
              lauren@streamkap.com Lauren Mulholland (Inactive)
              Vincenzo Santonastaso (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: