Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8770

Failed to Automatically Create debezium_database_history

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Minor Minor
    • None
    • None
    • jdbc-connector
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Moderate

      When using {}debezium-storage-jdbc{} to automatically generate debezium_database_history, an error occurs because the length of the history_data column is too large.

      What Debezium connector do you use and what version?

      io.debezium:debezium-storage-jdbc:3.0.6

      What is the connector configuration?

      ```

      Embedded engine with Spring Boot

      @Bean
      fun debeziumFactory(): Configuration {
      return Configuration.create()

      .with("schema.history.internal", "io.debezium.storage.jdbc.history.JdbcSchemaHistory")

      ...

      ```

      What is the captured database version and mode of deployment?

      mysql:8.0, using utf8mb4

      What behavior do you expect?

      Automatically create debezium_database_history table and start application

      What behavior do you see?

      When running the application, the following error occurs.

      ```

      Column length too big for column 'history_data' (max = 16383); use BLOB or TEXT instead

      ```

      Do you see the same behaviour using the latest released Debezium version?

      not confirmed

      Do you have the connector logs, ideally from start till finish?

      (You might be asked later to provide [DEBUG/TRACE](https://debezium.io/documentation/reference/stable/operations/logging.html) level log)

      ```

      2025-03-04T09:13:48.385+09:00 INFO 36806 — [kafka-sample] [pool-2-thread-1] i.d.s.jdbc.history.JdbcSchemaHistory : Creating table debezium_database_history to store database history 2025-03-04T09:13:48.386+09:00 WARN 36806 — [kafka-sample] [pool-2-thread-1] i.d.s.j.offset.JdbcOffsetBackingStore : Attempt 1 to call 'initialize storage' failed. java.sql.SQLSyntaxErrorException: Column length too big for column 'history_data' (max = 16383); use BLOB or TEXT instead at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:112) ~[mysql-connector-j-9.1.0.jar:9.1.0] at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:114) ~[mysql-connector-j-9.1.0.jar:9.1.0] at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:988) ~[mysql-connector-j-9.1.0.jar:9.1.0] at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:382) ~[mysql-connector-j-9.1.0.jar:9.1.0] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.lambda$initializeStorage$7(JdbcSchemaHistory.java:263) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.RetriableConnection.executeWithRetry(RetriableConnection.java:140) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.RetriableConnection.executeWithRetry(RetriableConnection.java:98) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.initializeStorage(JdbcSchemaHistory.java:255) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.lambda$start$0(JdbcSchemaHistory.java:87) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:99) ~[debezium-core-3.0.5.Final.jar:3.0.5.Final] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.start(JdbcSchemaHistory.java:80) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.relational.HistorizedRelationalDatabaseSchema.<init>(HistorizedRelationalDatabaseSchema.java:51) ~[debezium-core-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.binlog.BinlogDatabaseSchema.<init>(BinlogDatabaseSchema.java:79) ~[debezium-connector-binlog-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.mysql.MySqlDatabaseSchema.<init>(MySqlDatabaseSchema.java:41) ~[debezium-connector-mysql-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:99) ~[debezium-connector-mysql-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:253) ~[debezium-core-3.0.5.Final.jar:3.0.5.Final] at io.debezium.embedded.EmbeddedEngine.startSourceTask(EmbeddedEngine.java:640) ~[debezium-embedded-3.0.4.Final.jar:3.0.4.Final] at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:452) ~[debezium-embedded-3.0.4.Final.jar:3.0.4.Final] at io.debezium.embedded.ConvertingEngineBuilder$1.run(ConvertingEngineBuilder.java:163) ~[debezium-embedded-3.0.4.Final.jar:3.0.4.Final] at com.example.kafka_sample.handler.DebeziumHandler.start$lambda$1(DebeziumHandler.kt:72) ~[main/:na] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na] at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]

      ```

      How to reproduce the issue using our [tutorial](https://github.com/debezium/debezium-examples/tree/main/tutorial) deployment?

      not confirmed, but if tutorial deployment is not embedded engine, the issue will not occur.

      Which use case/requirement will be addressed by the proposed feature?

      when using debezium embedded engine with Spring Boot and managing schema history with mysql, an error occurs when starting the application due to the size of the history_data column.

      Implementation ideas (optional)

      • Change the type of the history_data column from VARCHAR to TEXT

              Unassigned Unassigned
              katsumimiyajima Katsumi Miyajima (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: