Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9362

Better handling of Mariadb 11.4+ binlog_legacy_event_pos

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Duplicate
    • Icon: Minor Minor
    • None
    • None
    • mariadb-connector
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Which use case/requirement will be addressed by the proposed feature?

      The debezium mariadb docs say to set the binlog_legacy_event_pos variable if using Mariadb 11.4+, so that binlog positions for events within transactions will be added back to the events. However, this means that Debezium users have to forever override the default behaviour, rather than have it just work.  

      I propose that these positions be synthetically created and added back in. 

      I recently helped implement such a mechanism in a PR for go-mysql, which is a golang mysql/mariadb replication library. I then successfully used it with mariadb 11.4 in a golang cdc library i was working on.

      But I've decided to just use Debezium instead, as it is vastly more mature. 

      All that is needed is to track the position after each event is parsed, and then when it is missing, just add the event length (which should be available) to the previous position. You also have to set the BINLOG_SEND_ANNOTATE_ROWS_EVENT flag when connecting to the DB. It's all in the PR there. 

      I don't know anything about debezium or Java, so don't think I can really provide a PR. 

      Please let me know if you have any questions!

              Unassigned Unassigned
              nickchomey Nick Chomey
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: