Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6605

Fixed DataCollections for table scan completion notificaiton

XMLWordPrintable

    • False
    • None
    • False
    • Moderate

      Bug report

      What Debezium connector do you use and what version?

      2.3.0-final 

      What is the connector configuration?

              "notification.enabled.channels": "sink",
              "notification.sink.topic.name": "_dbz-notifications",

      What behaviour do you expect?

      To see only the table name for which the table scan has completed.

      What behaviour do you see?

      All tables for which an incremental snapshot was requested. 

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes

       

      Do you have the connector logs, ideally from start till finish?

      The notification produced to Kafka:
       

      {  "additional_data": {   "connector_name": "my_connector",   "total_rows_scanned": "1165157",   "status": "SUCCEEDED",   "data_collections": "table1,table2"   },   "id": "3924b5c4-aed9-410e-868f-3e47757bbfa0",   "type": "TABLE_SCAN_COMPLETED",   "aggregate_type": "Incremental Snapshot"  }

       
       
      The connector offset after this notification is sent

      {
        ...
        "incremental_snapshot_collections": "[{\"incremental_snapshot_collections_id\":\"table2\",\"incremental_snapshot_collections_additional_condition\":null,\"incremental_snapshot_collections_surrogate_key\":null}]", ... }

      Implementation ideas (optional)

      Send only the current data collection instead of the list of all the backlog of tables to snapshot in IncrementalSnapshotNotificationService.

       

              Unassigned Unassigned
              bdbene Balint Bene
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: