Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1029

Messages from tables without PK and with REPLICA IDENTITY FULL

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Optional Optional
    • 0.9.0.CR1
    • 0.9.0.Beta1
    • postgresql-connector
    • None

      Hi.
      (Bad English ahead, thanks for understanding)

      I have found one small flaw (probably) in the connector - it does not emit events for tables that do not have primary keys. There is only WARN log message for create and delete transactions. Update transaction also produces WARN log, but emitted as event.

      Main problem I see in the RecordsStreamProducer:generateCreateRecord and RecordsStreamProducer:generateDeleteRecord, they both just aborting execution if passed as argument rowData does not have key part. WARN log message is produced in the RecordsStreamProducer:process after checking table schema for key schema.

      All code for that behavior was written back in 2016, i believe it was time of logical replication release for postgres. May be at that point there was no REPLICA IDENTITY FULL but now we have it and all messages from such tables have all needed data to be emitted as events.

      So I have checked code that uses this 'key' part of the message and found only creation of SourceRecord objects where javaDoc clearly says that key may be null.

      I have made a fix by simply removing checks for key nullability and tested it. As a result:

      Without REPLICA IDENTITY FULL:

      • Create event is ignored
      • Update event is emitted as is ('before' part of the message is empty)
      • Delete event is ignored

      With REPLICA IDENTITY FULL:

      • Create event is emitted with 'before' as null
      • Update event is emitted with both parts
      • Delete event is emitted with full 'before' part and null as 'after' part

      This is enough for my needs and looks good for any sort of join-tables (like user_roles). Appending .patch, but i don't have time rn to write tests, sorry for that

            Unassigned Unassigned
            zibranki Ilia Bogdanov (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: