Details
-
Bug
-
Resolution: Unresolved
-
Major
-
2.5.0.Final
-
None
-
False
-
None
-
False
-
Important
Description
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
What Debezium connector do you use and what version?
I use Debezium Server (Standalone) with postgresql connector - 2.5.0.Final version
What is the connector configuration?
debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector debezium.source.database.hostname=pg debezium.source.database.port=5432 debezium.source.database.user=postgres debezium.source.database.password=password debezium.source.database.dbname=postgres debezium.source.plugin.name=pgoutput debezium.source.topic.prefix=local debezium.source.offset.storage.file.name=offset debezium.sink.type=kafka debezium.sink.kafka.producer.bootstrap.servers=kafka:29092 debezium.sink.kafka.producer.key.serializer=org.apache.kafka.common.serialization.StringSerializer debezium.sink.kafka.producer.value.serializer=org.apache.kafka.common.serialization.StringSerializer
What is the captured database version and mode of deployment?
(E.g. on-premises, with a specific cloud provider, etc.)
Regular Postgres 14.4
What behaviour do you expect?
I expect debezium to have 2 opened connections
What behaviour do you see?
Debezium has 3 opened connections
Do you see the same behaviour using the latest relesead Debezium version?
Debezium takes 2 connections (normal behaviour) until `2.4.2.Final` version. It started taking 3 connections from the `2.5.0.Final` version.
This is the additional query that's idle introduced by debezium `2.5.0.Final`and more recent versions too.
SELECT t.oid AS oid, t.typname AS name, t.typelem AS element, t.typbasetype AS parentoid, t.typtypmod as modifiers, t.typcategory as category, e.values as enum_values FROM pg_catalog.pg_type t JOIN pg_catalog.pg_namespace n ON (t.typnamespace = n.oid) LEFT JOIN (SELECT t.enumtypid as id, array_agg(t.enumlabel) as values FROM p
g_catalog.pg_enum t GROUP BY id) e ON (t.oid = e.id) WHERE n.nspname != 'pg_toast' AND t.oid = $1
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
Logs are normal, nothing out of the ordinary
How to reproduce the issue using our tutorial deployment?
I attached a `docker-compose` {}file that contains:
- Postgresql db
- Kafka + Zookeeper
- Debezium
Here's how to reproduce the issue:
# 1. Start Postgres DB and Kafka and Debezium 2.4.2.Final docker compose -f docker-compose.dbz.yml up -d dbz # 2. Create a table and insert a row docker compose -f docker-compose.dbz.yml exec pg psql -U postgres -c "CREATE TABLE my_table (id SERIAL PRIMARY KEY, name VARCHAR(255)); INSERT INTO my_table (name) VALUES ('Moustapha');" # 3. Check the number of active connections for debezium docker compose -f docker-compose.dbz.yml exec pg psql -U postgres -c "SELECT count(*) FROM pg_stat_activity WHERE application_name like 'Debezium%';" # Output for debezium 2.4.2.Final: # count # ------- # 2 # (1 row) # 4. Comment line 12 and uncomment line 13 in docker-compose.dbz.yml file # 5. Start Debezium 2.5.0.Final docker compose -f docker-compose.dbz.yml up -d dbz # 6. Check the number of active connections for debezium docker compose -f docker-compose.dbz.yml exec pg psql -U postgres -c "SELECT count(*) FROM pg_stat_activity WHERE application_name like 'Debezium%';" # Output for debezium 2.5.0.Final: # count # ------- # 3 # (1 row)
So we noticed the issue when we migrated to the `2.5.0.Final` version and the number of connections we dedicated for debezium weren't enough anymore.
FYI I found this old issue that describes a very similar issue.
