Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2284

Explain LSN conversion algorithm in documentation

    XMLWordPrintable

Details

    • Task
    • Resolution: Unresolved
    • Major
    • Backlog
    • None
    • postgresql-connector
    • None

    Description

      We noticed that Debezium represents LSN as a 64-bit integer.  Indeed the docs say "Internally, an LSN is a 64-bit integer".  But we are not clear on just how that is done and if it is being done consistently both from WAL and from a read snapshot.  I cannot find any pg function that converts '16/B374D848' to a 64-bit integer.

      It appears that you can remove the `/` separator for the two pieces of the log sequence number, do a bitshiftleft(32) and do a hex conversion based on that.  But that leaves unanswered what if there are only 7 digits that show up in the right half of the LSN?  Are you adding the padding before doing the conversion?  Is there some simple internal method postgres exposes to get this?

      Note that we are using pgoutput.  However we get the LSN also with a snapshot so there must be some algorithm for getting it either out of WAL or in snapshot.

      There is a good reason for the question.  We use another service which does a separate hex conversion for each of the two hex numbers.  We are trying to align LSNs between these two systems and want to be confident that Debezium is doing it correctly and consistently.

      So again, it would help if there were some mention in the docs of how Debezium maps LSN like '16/B374D848' to a 64-bit integer.

      Thank you!

      Attachments

        Activity

          People

            Unassigned Unassigned
            jfinzel Jeremy Finzel
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: