Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1163

[postgres] differing logic between snapsnot and streams for create record

XMLWordPrintable

    • Hide
      • create a table without a pk (mine used uuid, hstore; same as used in DBZ-1162)
      • insert some data
      • start the connector
      • (if fix for 1162 is in place, or you're using non-hstore columns) - you should see snapshot complete through step 4, but step 5 will not be present. No exceptions should be raised however
      • data from database should not be in kafka at this point; topic should not exist even
      • insert some more data into the table
      • the topic and new data points should show up in kafka now. Snapshotted data should still be missing
      Show
      create a table without a pk (mine used uuid, hstore; same as used in DBZ-1162 ) insert some data start the connector (if fix for 1162 is in place, or you're using non-hstore columns) - you should see snapshot complete through step 4, but step 5 will not be present. No exceptions should be raised however data from database should not be in kafka at this point; topic should not exist even insert some more data into the table the topic and new data points should show up in kafka now. Snapshotted data should still be missing

      streams code, snapshot code

      I think the outcome of this is that - if you have a table which does not contain a PK, the snapshot will (silently) fail to write data to kafka; however, streams will pick up new data. This was the behavior I observed anyway.

      For me, the lack of a PK was a pebak error though (I intended to have a PK on the table), and I did not investigate further. I can say however, that removing the check for key == null in the snapshot producer did result in data getting into kafka

              Unassigned Unassigned
              p5k6@yahoo.com josh stanfield (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: