-
Task
-
Resolution: Done
-
Major
-
None
-
None
There are generally two ways to do a PostreSQL database upgrade:
- Use pg_dump/restore, but the LSNs will change using this approach.
- Use pg_upgrade. This is full of all kinds of black-box behavior. In short, it does a dump/restore for the schema only; then, it moves the data back in under the new schema.
In either case, the safest approach here is generally to consider the following steps with Debezium:
- Guarantee that all changes have been read from the replication slot, putting the database in effective read-only mode.
- Stop Debezium and the database.
- Perform the upgrade using an approved PG procedure.
- Remove the connector offsets.
- Restart the connector, using snapshot.mode=never.
- Open the database for writes by users.
Since replication slots are not retained as part of the upgrade procedure, it's generally best that offsets are removed and a new schema snapshot is taken but without data, assuming that the user can guarantee that no data changed throughout the upgrade process and that no changes were lost before the upgrade.
If that cannot be guaranteed, then it'll be essential to note that a new snapshot should be preferred using snapshot.mode=initial to ensure that no data loss occurs.
- links to
-
RHEA-2023:120698
Red Hat build of Debezium 2.3.4 release
[DBZ-6046] Add Debezium steps when performing a PostgreSQL database upgrade
Debezium Builder
added a comment - Released 
Jiri Pechanec
added a comment - broldan@redhat.com The safe approach is just to use the new name for the connector. In that case the user does not need to do anything harmful.
And if they want to remove the old offset as cleanup then there can be a link recommendation like you can use a standard Kafka tool to remove the old offset. One example how to do it is...
Jiri Pechanec
added a comment - broldan@redhat.com The safe approach is just to use the new name for the connector. In that case the user does not need to do anything harmful.
And if they want to remove the old offset as cleanup then there can be a link recommendation like you can use a standard Kafka tool to remove the old offset. One example how to do it is... Jiri Pechanec
added a comment - 1. Yes the safest way to to stop writing and then make sure Debezium has consumed everything from the replication slot by checkingng the flushed LSN.
3. I don't think we have a specific method.
4. I'd keep it as an FAQ entry and refer to it. It is still something exceptional and I don't think it should be part of the regular docs.
Jiri Pechanec
added a comment - broldan@redhat.com
1. Yes the safest way to to stop writing and then make sure Debezium has consumed everything from the replication slot by checkingng the flushed LSN.
3. I don't think we have a specific method.
4. I'd keep it as an FAQ entry and refer to it. It is still something exceptional and I don't think it should be part of the regular docs. ccranfor@redhat.com A few questions:
1. Guarantee that all changes have been read from the replication slot, putting the database in effective read-only mode.
Can you interpret this for me? I understand why we want to give the connector a chance to drain the entries in the log, but how do we verify that? Or is the actual step here that we have to close the database to writing, prior to stopping the db in Step 2 to give the connector that chance to finish consuming entries from the slot?
3. I'll point readers to see the PG doc for information about upgrading the database. Is there a method that is better suited for use with Debezium?
4. We have an FAQ entry about removing offsets. AFAIR, at one point I asked whether that procedure should become part of the documentation, but it was suggested to leave it as an FAQ entry only. WDYT?
Since the problem described in this issue should be resolved in a recent advisory, it has been closed.
For information on the advisory (Red Hat build of Debezium 2.3.4 release), and where to find the updated files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHEA-2023:5471