Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1181

The Postgres wal2json streaming and non-streaming decoders do not process empty events

XMLWordPrintable

    • Hide

      1. Turn on debugging so that you can see all events read in from wal2json
      2. Run a PG connector using wal2json (non-streaming, streaming version does not print everything in debug logs)
      3. Make changes like truncate table, DDL, materialized views, etc.
      4. You will see events come in with a format like:

      DEBUG Message arrived for decoding {
    "xid" : 2753761939,
    "timestamp" : "2019-03-06 21:39:01.095414+00",
    "change" : [ ]
}

      5. No debug messages appear for the corresponding heartbeat that should fire for the change to move the WAL forward.

      Show
      1. Turn on debugging so that you can see all events read in from wal2json 2. Run a PG connector using wal2json (non-streaming, streaming version does not print everything in debug logs) 3. Make changes like truncate table, DDL, materialized views, etc. 4. You will see events come in with a format like: DEBUG Message arrived for decoding { "xid" : 2753761939, "timestamp" : "2019-03-06 21:39:01.095414+00" , "change" : [ ] } 5. No debug messages appear for the corresponding heartbeat that should fire for the change to move the WAL forward.

      According to investigations in this WAL2JSON ticket, the plugin sends empty changes for DDL, truncate table, vacuum, and materialized view actions. Debezium's Postgres WAL2JSON streaming and non-streaming decoders do not process these empty changes. They should still process them so that the heartbeat happens and the WAL moves forward.

              Unassigned Unassigned
              trolison Taylor Rolison (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: