Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8205

Lock wait timeout exceeded during initial snapshot mysql

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Major Major
    • None
    • None
    • mysql-connector
    • None
    • False
    • None
    • False
    • Moderate

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      Mysql/ 2.4

      What is the connector configuration?

       

      Spec: Class: io.debezium.connector.mysql.MySqlConnector Config: database.dbname: xyz database.hostname: sss.ap-southeast-1.rds.amazonaws.com database.password: ${env:password} database.port: 3306 database.server.id: 1002561 database.server.name: xyz database.user: root key.converter: org.apache.kafka.connect.storage.StringConverter key.enforce.uniqueness: false message.key.columns: xyz._temp:identifier plugin.name: bin_log producer.override.interceptor.classes: com.connect.interceptors.CustomHeaderProducerInterceptor schema.history.internal.consumer.sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="${env:KAFKABROKER_USERNAME}" password="${env:KAFKABROKER_PASSWORD}"; schema.history.internal.consumer.sasl.mechanism: PLAIN schema.history.internal.consumer.security.protocol: SASL_SSL schema.history.internal.consumer.ssl.endpoint.identification.algorithm: https schema.history.internal.kafka.bootstrap.servers: aws.confluent.cloud:9092 schema.history.internal.kafka.topic: topic1 schema.history.internal.producer.sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="${env:KAFKABROKER_USERNAME}" password="${env:KAFKABROKER_PASSWORD}"; schema.history.internal.producer.sasl.mechanism: PLAIN schema.history.internal.producer.security.protocol: SASL_SSL schema.history.internal.producer.ssl.endpoint.identification.algorithm: https snapshot.mode: when_needed table.include.list: xyz._temp tombstones.on.delete: false topic.prefix: test Transforms: filterTableEvents,moveFieldsToHeader,addSchemaSubject,AddNamespace,Reroute transforms.AddNamespace.schema.name: com.CdcEvent transforms.AddNamespace.type: org.apache.kafka.connect.transforms.SetSchemaMetadata$Value transforms.Reroute.key.enforce.uniqueness: false transforms.Reroute.topic.regex: (.*)xyz._temp transforms.Reroute.topic.replacement: topic transforms.Reroute.type: io.debezium.transforms.ByLogicalTableRouter transforms.addSchemaSubject.schema.subject.field: schmea transforms.addSchemaSubject.type: com.AddSchemaSubjectTransform transforms.filterTableEvents.schema.change.event.exclude.list: DROP,TRUNCATE,CREATE,ALTER,DATABASE transforms.filterTableEvents.type: io.debezium.transforms.SchemaChangeEventFilter transforms.moveFieldsToHeader.fields: source,op,ts_ms,transaction transforms.moveFieldsToHeader.headers: source,operation,EVENT_TIMESTAMP,transaction transforms.moveFieldsToHeader.operation: copy transforms.moveFieldsToHeader.type: org.apache.kafka.connect.transforms.HeaderFrom$Value value.converter: io.confluent.connect.avro.AvroConverter value.converter.auto.register.schemas: false value.converter.connect.meta.data: false value.converter.schema.registry.url: https://xyz value.converter.schema.subject: schema value.converter.use.latest.version: true value.converter.value.subject.name.strategy: com.SchemaSubjectNamingStrategy

       

      What is the captured database version and mode of deployment?

      (E.g. on-premises, with a specific cloud provider, etc.)

      EKS based self hosted kafka connect cluster .- Have other mysql working for same db with different table 

      What behavior do you expect?

      snapshot should completed and stream data.

      What behavior do you see?

      <Your answer>

      Do you see the same behaviour using the latest released Debezium version?

      (Ideally, also verify with latest Alpha/Beta/CR version)

      <Your answer>

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide DEBUG/TRACE level log)

      org.apache.kafka.connect.errors.RetriableException: An exception occurred in the change event producer. This connector will be restarted. 
        at io.debezium.pipeline.ErrorHandler.setProducerThrowable(ErrorHandler.java:63) 
        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:144) 
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) 
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) 
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) 
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) 
        at java.base/java.lang.Thread.run(Thread.java:840) 
      Caused by: io.debezium.DebeziumException: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Lock wait timeout exceeded; try restarting transaction 
        at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:101) 
        at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:253) 
        at io.debezium.pipeline.ChangeEventSourceCoordinator.doSnapshot(ChangeEventSourceCoordinator.java:237) 
        at io.debezium.pipeline.ChangeEventSourceCoordinator.executeChangeEventSources(ChangeEventSourceCoordinator.java:189) 
        at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:137) 
        ... 5 more 
      Caused by: com.mysql.cj.jdbc.exceptions.MySQLTransactionRollbackException: Lock wait timeout exceeded; try restarting transaction 
        at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:124) 
        at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) 
        at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:763) 
        at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648) 
        at io.debezium.jdbc.JdbcConnection.executeWithoutCommitting(JdbcConnection.java:1449) 
        at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.tableLock(MySqlSnapshotChangeEventSource.java:552) 
        at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.readTableStructure(MySqlSnapshotChangeEventSource.java:341) 
        at io.debezium.connector.mysql.MySqlSnapshotChangeEventSource.readTableStructure(MySqlSnapshotChangeEventSource.java:59) 
        at io.debezium.relational.RelationalSnapshotChangeEventSource.doExecute(RelationalSnapshotChangeEventSource.java:149) 
        at io.debezium.pipeline.source.AbstractSnapshotChangeEventSource.execute(AbstractSnapshotChangeEventSource.java:92) 
        ... 9 more 
      [2024-09-04 13:16:33,865] INFO [hybris-connector|task-0] Awaiting end of restart backoff period after a retriable error (io.debezium.connector.common.BaseSourceTask:256) 
      [2024-09-04 13:16:34,238] INFO [Consumer clientId=connect--offsets, groupId=connect] Node -1 disconnected. (org.apache.kafka.clients.NetworkClient:977) 

      How to reproduce the issue using our tutorial deployment?

      <Your answer>

      Feature request or enhancement

      For feature requests or enhancements, provide this information, please:

      Which use case/requirement will be addressed by the proposed feature?

      <Your answer>

      Implementation ideas (optional)

      <Your answer>

              Unassigned Unassigned
              sai9920@gmail.com Sai Abhinay Bommakanti (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: