Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1840

Misleading warning message about uncommitted offsets

    XMLWordPrintable

Details

    Description

      Without any database activity, I'm seeing this message in the logs:

      connect_1    | 2020-03-02 11:42:30,047 WARN   Postgres|dbserver1|postgres-connector-task  Received 10001 events which were all filtered out, so no offset could be committed. This prevents the replication slot from acknowledging the processed WAL offsets, causing a growing backlog of non-removeable WAL segments on the database server. Consider to either adjust your filter configuration or enable heartbeat events (via the heartbeat.interval.ms option) to avoid this situation.   [io.debezium.connector.postgresql.PostgresStreamingChangeEventSource]
      

      The reason being that readPending() yields null to the consumer if there's no pending change. I.e. null is a regularly expected value here, but it triggers that message.

      Also, and more importantly, this circumvents the throttling implementation, as readPending() doesn't return false in this case.

      Attachments

        Activity

          People

            gmorling@redhat.com Gunnar Morling
            gmorling@redhat.com Gunnar Morling
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: