Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1181

The Postgres wal2json streaming and non-streaming decoders do not process empty events

    Details

    • Steps to Reproduce:
      Hide

      1. Turn on debugging so that you can see all events read in from wal2json
      2. Run a PG connector using wal2json (non-streaming, streaming version does not print everything in debug logs)
      3. Make changes like truncate table, DDL, materialized views, etc.
      4. You will see events come in with a format like:

      DEBUG Message arrived for decoding {
    "xid" : 2753761939,
    "timestamp" : "2019-03-06 21:39:01.095414+00",
    "change" : [ ]
}

      5. No debug messages appear for the corresponding heartbeat that should fire for the change to move the WAL forward.

      Show
      1. Turn on debugging so that you can see all events read in from wal2json 2. Run a PG connector using wal2json (non-streaming, streaming version does not print everything in debug logs) 3. Make changes like truncate table, DDL, materialized views, etc. 4. You will see events come in with a format like: DEBUG Message arrived for decoding { "xid" : 2753761939, "timestamp" : "2019-03-06 21:39:01.095414+00" , "change" : [ ] } 5. No debug messages appear for the corresponding heartbeat that should fire for the change to move the WAL forward.

      Description

      According to investigations in this WAL2JSON ticket, the plugin sends empty changes for DDL, truncate table, vacuum, and materialized view actions. Debezium's Postgres WAL2JSON streaming and non-streaming decoders do not process these empty changes. They should still process them so that the heartbeat happens and the WAL moves forward.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                trolison Taylor Rolison
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: