Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1840

Misleading warning message about uncommitted offsets

    Details

      Description

      Without any database activity, I'm seeing this message in the logs:

      connect_1    | 2020-03-02 11:42:30,047 WARN   Postgres|dbserver1|postgres-connector-task  Received 10001 events which were all filtered out, so no offset could be committed. This prevents the replication slot from acknowledging the processed WAL offsets, causing a growing backlog of non-removeable WAL segments on the database server. Consider to either adjust your filter configuration or enable heartbeat events (via the heartbeat.interval.ms option) to avoid this situation.   [io.debezium.connector.postgresql.PostgresStreamingChangeEventSource]
      

      The reason being that readPending() yields null to the consumer if there's no pending change. I.e. null is a regularly expected value here, but it triggers that message.

      Also, and more importantly, this circumvents the throttling implementation, as readPending() doesn't return false in this case.

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                gunnar.morling Gunnar Morling
                Reporter:
                gunnar.morling Gunnar Morling
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: