Over at Materialize we realized that we cannot deduplicate Debezium events in constant space—this time with the transaction metadata topic. For historical context, see DBZ-2911, which covered the case of O(1) space deduplication for data topics.

The problem is that the transaction ID for PostgreSQL databases is a 32-bit integer that wraps around. That means we cannot simply track the highest transaction ID that we've seen, and discard any new transactions with a smaller transaction ID. 

Instead, we'd need the transactional metadata topic to expose the end LSN of the transaction. The end LSN is a 64-bit integer and so it never wraps around.

Is that something that would be possible to add? 

cc jpechane