Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5247

postgresql-connector table.exclude.list not working

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • under-triaging
    • 1.9.2.Final, 1.9.3.Final
    • postgresql-connector
    • None
    • False
    • None
    • False

    Description

      Bug report

      What Debezium connector do you use and what version?

      I'm using debezium-connecor-postgresql with version 1.9.2.Final

      What is the connector configuration?

      plugin.name: pgoutput
      publication.name: dbz_public_all
      slot.name: debezium_strimzi
      database.hostname: postgresql.service.svc
      database.port: 5432
      database.user: kafkaconnect
      database.password: ${secrets:kafka/debezium-secret:postgresql_password}
      database.dbname: malt
      database.server.name: strimzi
      database.include.list: malt
      schema.exclude.list: pg_catalog,information_schema,quartz,pghero
      table.exclude.list: "public\\.pghero"
      snapshot.mode: never
      heartbeat.interval.ms: 1000

      What is the captured database version and mode of depoyment?

      Captured database version : postgresql 14.2 (managed by zalando postgres operator)

      Mode of deployment : Strimzi operator (strimzi/kafka:0.23.0-kafka-2.6.0 with debezium-connector-postgresql 1.9.3.Final installed)

      kafka destination : strimzi/kafka:0.23.0-kafka-2.6.0 (managed by strimzi 0.23.0 operator)

      What behaviour do you expect?

      The `table.exclude.list` config paramater should allow me to exclude some postgresql table from the capture.

      `table.exclude.list` is a list of regex (comma separated)

      What behaviour do you see?

      When I specify any value in the `table.exclude.list` config parameter, the connector exclude all the tables.

      When I remove this config parameter, all the tables are captured and sent to my kafka (but I want to filter some big tables to avoid fill my kafka topics with unnessessary data)

      I've tried with multiple regex, with dot escapping, only specify one table, etc.. Nothing work.

      Do you see the same behaviour using the latest relesead Debezium version?

      Previously I was using the old debezium 1.3 version and it worked.
      I'm trying to upgrade to the latest version, but unfortunattely I can't because of this.

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            thierry.salle@devoteam.com Thierry SALLE (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: