Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4013

Incorrect incremental snapshot DDL triggers snapshot that generates unending* inserts against signalling table

    XMLWordPrintable

Details

    Description

      I accidentally used the incorrect DDL syntax to trigger a incremental snapshot (debezium postgres connector):

      INSERT INTO debezium_signals(id, signal_type, message) VALUES('ad-hoc-11', 'execute-snapshot', '{"data-collections": ["transfers"]}');

      instead of properly prefixing the table name in the data-collections attribute:

      INSERT INTO debezium_signals(id, signal_type, message) VALUES('ad-hoc-12', 'execute-snapshot', '{"data-collections": ["public.transfers"]}');

      Now as long as the Debezium Postgres connector is running, it's inserting hundreds of record per second in the debezium_signals table, and doesn't stop. I checked and the connector isn't streaming snapshotted CDC events to the intended topic associated with the public.transfers table.

       

      I've tried deleting the connector and recreating it but the snapshot always picks up where it lefts off - and resumes inserting hundreds of records per second into the signalling table.

       

      I'm not familiar enough with the internals of Debezium to know if this is a bug or if I just need to reset something on my connector. In any event, Debezium should validate snapshot DDL that is incorrectly formatted (like the above example) to preempt this kind of situation.

       

      Attachments

        Issue Links

          Activity

            People

              jpechane Jiri Pechanec
              adelbertkuhl nate kuhl
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                PagerDuty