Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1519

Debezium Erroneously Reporting No Tables to Capture

    Details

    • Steps to Reproduce:
      Hide

      We're using the following config. The DB is on RDS and the schema we're whitelisting has a single table we're capturing.

      {
          "name": "{{connector_name}}",
          "config": {
              "column.blacklist": "thumbnail_service.thumbnail.asset_type,thumbnail_service.thumbnail.score",
              "connector.class": "io.debezium.connector.postgresql.PostgresConnector",
              "database.hostname": "{{db_host}}",
              "database.port": "{{db_port}}",
              "database.user": "{{db_user}}",
              "database.password": "{{db_pass}}",
              "database.dbname": "thumbnail",
              "database.server.name": "thumbnail",
              "max.batch.size": "{{max_batch_size}}",
              "max.queue.size": "{{max_queue_size}}",
              "plugin.name": "wal2json_rds",
              "schema.whitelist":"thumbnail_service",
              "tasks.max": "1",
              "transforms": "unwrap,ReplaceField",
              "transforms.unwrap.type": "io.debezium.transforms.UnwrapFromEnvelope",
              "transforms.unwrap.drop.tombstones": "false",
              "transforms.ReplaceField.type": "org.apache.kafka.connect.transforms.ReplaceField$Key",
              "transforms.ReplaceField.renames": "asset_id:media_id,created_date:created,updated_date:last_modified,timestamp:video_offset_timestamp"
          }
      }
      
      Show
      We're using the following config. The DB is on RDS and the schema we're whitelisting has a single table we're capturing. { "name": "{{connector_name}}", "config": { "column.blacklist": "thumbnail_service.thumbnail.asset_type,thumbnail_service.thumbnail.score", "connector.class": "io.debezium.connector.postgresql.PostgresConnector", "database.hostname": "{{db_host}}", "database.port": "{{db_port}}", "database.user": "{{db_user}}", "database.password": "{{db_pass}}", "database.dbname": "thumbnail", "database.server.name": "thumbnail", "max.batch.size": "{{max_batch_size}}", "max.queue.size": "{{max_queue_size}}", "plugin.name": "wal2json_rds", "schema.whitelist":"thumbnail_service", "tasks.max": "1", "transforms": "unwrap,ReplaceField", "transforms.unwrap.type": "io.debezium.transforms.UnwrapFromEnvelope", "transforms.unwrap.drop.tombstones": "false", "transforms.ReplaceField.type": "org.apache.kafka.connect.transforms.ReplaceField$Key", "transforms.ReplaceField.renames": "asset_id:media_id,created_date:created,updated_date:last_modified,timestamp:video_offset_timestamp" } }

      Description

      I've configured a debezium instance to monitor a database that has two schemas (other than public). It should only replicate one of them, and so I use schema.whitelist in my config.

      Setting this value does prevent debezium from monitoring the schema I don't want captured, and debezium is successfully capturing the schema I do want. However, upon booting up kafka-connect, I see the following WARN:

      2019-10-01 14:24:37,117 WARN   Postgres|thumbnail|postgres-connector-task  After applying blacklist/whitelist filters there are no tables to monitor, please check your configuration   [io.debezium.relational.RelationalDatabaseSchema]
      

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dvfeinblum David Feinblum
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: