Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1727

Cannot shutdown PostgreSQL if there is an active Debezium connector

XMLWordPrintable

    • Hide

      1. Start Debezium PostgreSQL connector
      2. Insert some new data into PostgreSQL database
      3. Make sure Debezium receives keepalive message at least once
      4. Shutdown PostgreSQL process

      Show
      1. Start Debezium PostgreSQL connector 2. Insert some new data into PostgreSQL database 3. Make sure Debezium receives keepalive message at least once 4. Shutdown PostgreSQL process

      PostgreSQL is unable to shutdown if there is an active Debezium PostgreSQL connector.

      For example, we can try to shutdown PostgreSQL manually (command below is used for debezium/postgres:12 Docker image):

      su - postgres -c '/usr/lib/postgresql/12/bin/pg_ctl stop -D /var/lib/postgresql/data'
      

      But after some time an error will be occured

      pg_ctl: server does not shut down
      

      Server will continue to work, but will disallow any new connection

      By default it will send SIGINT signal to postgres process. From PostgreSQL docs:

      The server disallows new connections and sends all existing server processes SIGTERM, which will cause them to abort their current transactions and exit promptly. It then waits for all server processes to exit and finally shuts down. If the server is in online backup mode, backup mode will be terminated, rendering the backup useless.

      The postgres process will sent Primary keepalive message (B) to replication stream with "The current end of WAL on the server" and wait for stream to sent back Standby status update (F) which will contain "The location of the last WAL byte + 1 applied in the standby". The process will repeat until those locations (lsn) are equal.

      The main problem is when Debezium PostgreSQL connector receives Primary keepalive message (at any time) it will not apply lsn from its message (ReplicationStream#flushLsn will not be invoked)

      So when PostgreSQL during shutdown process will compare end of WAL on the server and connector's last applied WAL location, it will compare server's last keepalive lsn with some debezium's XLogData lsn. Thus Postgres process will stuck in an infinite cycle and will never shutdown

              Unassigned Unassigned
              igabaydulin Igor Gabaydulin (Inactive)
              Votes:
              1 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: