Bug report

What Debezium connector do you use and what version?

Spanner

What is the connector configuration?

{"name": "livestream-connector","config":

{ "connector.class": "io.debezium.connector.spanner.SpannerConnector", "tasks.max": "1", "gcp.spanner.change.stream": "live-table", "gcp.spanner.project.id": "project-stag, "gcp.spanner.instance.id": "development-instance", "gcp.spanner.database.id": "live-db", "gcp.spanner.credentials.path": "/app/services-sa.json", "key.converter.schemas.enable":false, "value.converter.schemas.enable":false }

}

What is the captured database version and mode of deployment?

GCP Managed Spanner DB and Kafka connect deployed on K8s.

What behaviour do you expect?

BufferedPublisher should terminate normally in case of error.

What behaviour do you see?

The thread is in Blocked state due to deadlock and while loop spins continuously. 

The loop which runs endlessly - https://github.com/debezium/debezium-connector-spanner/blob/ed4fe34fe887e9f5304a40d6d74b2bdf2879f06a/src/main/java/io/debezium/connector/spanner/kafka/internal/BufferedPublisher.java#L85

Do you see the same behaviour using the latest relesead Debezium version?

Yes.

Do you have the connector logs, ideally from start till finish?

Yes.

How to reproduce the issue using our tutorial deployment?

When this publish fails https://github.com/debezium/debezium-connector-spanner/blob/ed4fe34fe887e9f5304a40d6d74b2bdf2879f06a/src/main/java/io/debezium/connector/spanner/kafka/internal/TaskSyncPublisher.java#L80 the loop mentioned above endlessly because thread cannot be terminated as it is blocked state forever.

 

Flow which leads to deadlock 
1. This method is called which calls this.

2. This methods fails. (Will open a separate issue as to why it fails).

3. This method is called which calls close method of Buffered publisher. 

4. The thread cannot be terminated because because it is in Blocked state. (waiting for this to return.)

5. The bufferedPublish method cannot return because its waiting on close method. (This leads to deadlock).

6. Causing while loop to spin endlessly causing high CPU usage.

Implementation ideas

Return exception instead of catching it and close the catch the exception like this. And terminate the thread itself rather than calling close.