Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6577

Explain failure on existing publication update when switching to `filtered` from `all_tables`

XMLWordPrintable

    • False
    • None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      With the introduction of DBZ-3921 it's now possible to update the publication by changing the filtering in the config. The idea is to have this feature when `publication.autocreate.mode: filtered` is configured. 
      However, in some cases, the users have to create a publication manually and if it is already created for all tables and then the config is changed to `publication.autocreate.mode: filtered` Deebzium tries to update the publication with filters specified in the config and fails.
      Instead, Debezium should emit a warning message saying that to apply filtering the publication should be recreated and proceed further.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      <Your answer>

      What is the connector configuration?

      <Your answer>

      What is the captured database version and mode of depoyment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      <Your answer>

      What behaviour do you expect?

      Warning message is issued and replication continue working.

      What behaviour do you see?

      <Your answer>

      Do you see the same behaviour using the latest relesead Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      <Your answer>

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      <Your answer>

       

      org.apache.kafka.connect.errors.ConnectException: An exception occurred in the change event producer. This connector will be stopped.
      at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:50)
      at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:180)
      at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:41)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.streamEvents(ChangeEventSourceCoordinator.java:174)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:141)
      at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:109)
      at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
      at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
      at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.kafka.connect.errors.ConnectException: Unable to update filtered publication dbz_publication for "<redacted>"
      at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createOrUpdatePublicationModeFilterted(PostgresReplicationConnection.java:214)
      at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initPublication(PostgresReplicationConnection.java:179)
      at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.initConnection(PostgresReplicationConnection.java:358)
      at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.startStreaming(PostgresReplicationConnection.java:319)
      at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:137)
      ... 9 more
Caused by: org.postgresql.util.PSQLException: ERROR: publication "dbz_publication" is defined as FOR ALL TABLES                                                            Detail: Tables cannot be added to or dropped from FOR ALL TABLES publications.
      at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2675)
      at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2365)
      at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:355)
      at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:490)
      at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:408)
      at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:329)
      at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:315)
      at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:291)
      at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:286)  

       

       

      How to reproduce the issue using our tutorial deployment?

      <Your answer>

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

      <Your answer>

              Unassigned Unassigned
              aipopov Anatolii Popov (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: