-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
None
-
None
-
False
-
-
False
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
For bug reports, provide this information, please:
- I'm trying to upgrade from version 1.9.7 but the offset state cannot be used in version 2.x
https://issues.redhat.com/browse/DBZ-6789
- Snapshot was started again, this error happens only when the database has a lot of records.
- debezium sends data pub/sub, and after pub/sub sends to Cloud Storage.
What Debezium connector do you use and what version?
Mysql connector 2.3.2, 2.4.0
What is the connector configuration?
debezium.sink.type=pubsub debezium.source.offset.storage.file.filename=offsetsNew.dat debezium.source.schema.history.internal=io.debezium.storage.file.history.FileSchemaHistory debezium.source.schema.history.internal.file.filename=db-history.dat debezium.source.schema.history.internal.skip.unparseable.ddl=true debezium.transforms=singleTopic debezium.transforms.singleTopic.type=io.debezium.transforms.ByLogicalTableRouter debezium.transforms.singleTopic.topic.regex=([^.]+).* debezium.transforms.singleTopic.topic.replacement=$1 debezium.sink.pubsub.project.id debezium.source.database.server.id=199999 debezium.source.topic.prefix debezium.source.connector.class debezium.source.database.hostname debezium.source.database.port debezium.source.database.user debezium.source.database.password debezium.source.database.server.name debezium.source.database.include.list
What is the captured database version and mode of depoyment?
(E.g. on-premises, with a specific cloud provider, etc.)
Google Cloud SQL MySQL 8.0
Kubernetes cluster statefulset, GKE
What behaviour do you expect?
Snapshot will be made successfully.
What behaviour do you see?
Snapshot was not completed successfully, it will be re-executed upon connector restart
RelationalSnapshotChangeEventSource","level":"ERROR","message":"Error during snapshot","threadName":"debezium-mysqlconnector-debezium_mysql_cards_test-change-event-source-coordinator","threadId":30,"mdc":{"dbz.taskId":"0","dbz.connectorName":"debezium_mysql_cards_test","dbz.connectorType":"MySQL","dbz.connectorContext":"snapshot"},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1,"exception":{"refId":1,"exceptionType":"java.lang.InterruptedException","message":null,"frames":[{"class<span class="code-quote">":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject","method":"reportInterruptAfterWait","line":2056},{"class<span class="code-quote">":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject","method":"await","line":2090},{"class<span class="code-quote">":"java.util.concurrent.LinkedBlockingQueue","method":"take","line":433},{"class<span class="code-quote">":"java.util.concurrent.ExecutorCompletionService","method":"take","line":200},{"class<span class="code-quote">":"io.debezium.relational.RelationalSnapshotChangeEventSource","method":"createDataEvents","line":463},{"class<span class="code-quote">":"io.debezium.relational.RelationalSnapshotChangeEventSource","method":"doExecute","line":165},{"class<span class="code-quote">":"io.debezium.pipeline.source.AbstractSnapshotChangeEventSource","method":"execute","line":92},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"doSnapshot","line":253},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"doSnapshot","line":237},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"executeChangeEventSources","line":189},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"lambda$start$0","line":137},{"class<span class="code-quote">":"java.util.concurrent.Executors$RunnableAdapter","method":"call","line":515},{"class<span class="code-quote">":"java.util.concurrent.FutureTask","method":"run","line":264},{"class<span class="code-quote">":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","line":1128},{"class<span class="code-quote">":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","line":628},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":829}]}}{"timestamp":"2023-11-13T13:20:29.715Z","sequence":401,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.pipeline.source.AbstractSnapshotChangeEventSource","level":"WARN","message":"Snapshot was interrupted before completion","threadName":"debezium-mysqlconnector-debezium_mysql_cards_test-change-event-source-coordinator","threadId":30,"mdc":{"dbz.taskId":"0","dbz.connectorName":"debezium_mysql_cards_test","dbz.connectorType":"MySQL","dbz.connectorContext":"snapshot"},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1}{"timestamp":"2023-11-13T13:20:29.715Z","sequence":402,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.pipeline.source.AbstractSnapshotChangeEventSource","level":"INFO","message":"Snapshot - Final stage","threadName":"debezium-mysqlconnector-debezium_mysql_cards_test-change-event-source-coordinator","threadId":30,"mdc":{"dbz.taskId":"0","dbz.connectorName":"debezium_mysql_cards_test","dbz.connectorType":"MySQL","dbz.connectorContext":"snapshot"},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1}{"timestamp":"2023-11-13T13:20:29.715Z","sequence":403,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.pipeline.source.AbstractSnapshotChangeEventSource","level":"WARN","message":"Snapshot was not completed successfully, it will be re-executed upon connector restart","threadName":"debezium-mysqlconnector-debezium_mysql_cards_test-change-event-source-coordinator","threadId":30,"mdc":{"dbz.taskId":"0","dbz.connectorName":"debezium_mysql_cards_test","dbz.connectorType":"MySQL","dbz.connectorContext":"snapshot"},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1}{"timestamp":"2023-11-13T13:20:29.716Z","sequence":404,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.pipeline.ChangeEventSourceCoordinator","level":"WARN","message":"Change event source executor was interrupted","threadName":"debezium-mysqlconnector-debezium_mysql_cards_test-change-event-source-coordinator","threadId":30,"mdc":{"dbz.taskId":"0","dbz.connectorName":"debezium_mysql_cards_test","dbz.connectorType":"MySQL","dbz.connectorContext":"snapshot"},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1,"exception":{"refId":1,"exceptionType":"java.lang.InterruptedException","message":null,"frames":[{"class<span class="code-quote">":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject","method":"reportInterruptAfterWait","line":2056},{"class<span class="code-quote">":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject","method":"await","line":2090},{"class<span class="code-quote">":"java.util.concurrent.LinkedBlockingQueue","method":"take","line":433},{"class<span class="code-quote">":"java.util.concurrent.ExecutorCompletionService","method":"take","line":200},{"class<span class="code-quote">":"io.debezium.relational.RelationalSnapshotChangeEventSource","method":"createDataEvents","line":463},{"class<span class="code-quote">":"io.debezium.relational.RelationalSnapshotChangeEventSource","method":"doExecute","line":165},{"class<span class="code-quote">":"io.debezium.pipeline.source.AbstractSnapshotChangeEventSource","method":"execute","line":92},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"doSnapshot","line":253},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"doSnapshot","line":237},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"executeChangeEventSources","line":189},{"class<span class="code-quote">":"io.debezium.pipeline.ChangeEventSourceCoordinator","method":"lambda$start$0","line":137},{"class<span class="code-quote">":"java.util.concurrent.Executors$RunnableAdapter","method":"call","line":515},{"class<span class="code-quote">":"java.util.concurrent.FutureTask","method":"run","line":264},{"class<span class="code-quote">":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","line":1128},{"class<span class="code-quote">":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","line":628},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":829}]}}{"timestamp":"2023-11-13T13:20:29.722Z","sequence":405,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.pipeline.signal.SignalProcessor","level":"INFO","message":"SignalProcessor stopped","threadName":"pool-7-thread-1","threadId":22,"mdc":{},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1}{"timestamp":"2023-11-13T13:20:29.729Z","sequence":406,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.jdbc.JdbcConnection","level":"INFO","message":"Connection gracefully closed","threadName":"pool-14-thread-1","threadId":50,"mdc":{},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1}{"timestamp":"2023-11-13T13:20:29.734Z","sequence":407,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"org.apache.kafka.connect.storage.FileOffsetBackingStore","level":"INFO","message":"Stopped FileOffsetBackingStore","threadName":"pool-7-thread-1","threadId":22,"mdc":{},"ndc":"","hostName":"debezium-mysql-cards-test-0","processName":"io.debezium.server.Main","processId":1}{"timestamp":"2023-11-13T13:20:29.737Z","sequence":408,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.server.ConnectorLifecycle","level":"ERROR","message":"Connector completed: success = 'false', message = 'Stopping connector after error in the application's handler method: java.util.concurrent.TimeoutException: Waited 30000 milliseconds (plus 258941 nanoseconds delay) for ListFuture@11eb7b58[status=PENDING, info=[futures=[com.google.api.core.AbstractApiFuture$InternalSettableFuture@46a5bb3f[status=SUCCESS, result=[java.lang.String@78cc03e0]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@42df5d1f[status=SUCCESS, result=[java.lang.String@8a4cfa7]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@6333e1cf[status=SUCCESS, result=[java.lang.String@7b783d4a]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@4377b09f[status=SUCCESS, result=[java.lang.String@6d0086a1]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@11c12f57[status=SUCCESS, result=[java.lang.String@2b69c248]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@55e46cf2[status=SUCCESS, result=[java.lang.String@3cb8cd61]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@249bf636[status=SUCCESS, result=[java.lang.String@41050b6f]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@37706c41[status=SUCCESS, result=[java.lang.String@4dd97c0b]], com.google.api.core.AbstractApiFuture$InternalSettableFuture@360819c0[status=SUCCESS, result=[java.lang.String@5fbf221d]], com.google.api.core.
Do you see the same behaviour using the latest relesead Debezium version?
For 2.4.0, the same behaviour
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide DEBUG/TRACE level log)
How to reproduce the issue using our tutorial deployment?
<Your answer>
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
<Your answer>
Implementation ideas (optional)
<Your answer>