Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1163

[postgres] differing logic between snapsnot and streams for create record

    XMLWordPrintable

Details

    • Hide
      • create a table without a pk (mine used uuid, hstore; same as used in DBZ-1162)
      • insert some data
      • start the connector
      • (if fix for 1162 is in place, or you're using non-hstore columns) - you should see snapshot complete through step 4, but step 5 will not be present. No exceptions should be raised however
      • data from database should not be in kafka at this point; topic should not exist even
      • insert some more data into the table
      • the topic and new data points should show up in kafka now. Snapshotted data should still be missing
      Show
      create a table without a pk (mine used uuid, hstore; same as used in DBZ-1162 ) insert some data start the connector (if fix for 1162 is in place, or you're using non-hstore columns) - you should see snapshot complete through step 4, but step 5 will not be present. No exceptions should be raised however data from database should not be in kafka at this point; topic should not exist even insert some more data into the table the topic and new data points should show up in kafka now. Snapshotted data should still be missing

    Description

      streams code, snapshot code

      I think the outcome of this is that - if you have a table which does not contain a PK, the snapshot will (silently) fail to write data to kafka; however, streams will pick up new data. This was the behavior I observed anyway.

      For me, the lack of a PK was a pebak error though (I intended to have a PK on the table), and I did not investigate further. I can say however, that removing the check for key == null in the snapshot producer did result in data getting into kafka

      Attachments

        Issue Links

          is related to

          Feature Request - Feature requests from customers and/or users DBZ-1225 Handle tables without PK consistently across all relational connectors

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed

          Activity

            People

              Unassigned Unassigned
              p5k6@yahoo.com josh stanfield (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: