-
Bug
-
Resolution: Duplicate
-
Minor
-
None
-
None
-
None
-
False
-
-
False
-
Moderate
When using {}debezium-storage-jdbc{} to automatically generate debezium_database_history, an error occurs because the length of the history_data column is too large.
What Debezium connector do you use and what version?
io.debezium:debezium-storage-jdbc:3.0.6
What is the connector configuration?
```
Embedded engine with Spring Boot
@Bean
fun debeziumFactory(): Configuration {
return Configuration.create()
.with("schema.history.internal", "io.debezium.storage.jdbc.history.JdbcSchemaHistory")
...
```
What is the captured database version and mode of deployment?
mysql:8.0, using utf8mb4
What behavior do you expect?
Automatically create debezium_database_history table and start application
What behavior do you see?
When running the application, the following error occurs.
```
Column length too big for column 'history_data' (max = 16383); use BLOB or TEXT instead
```
Do you see the same behaviour using the latest released Debezium version?
not confirmed
Do you have the connector logs, ideally from start till finish?
(You might be asked later to provide [DEBUG/TRACE](https://debezium.io/documentation/reference/stable/operations/logging.html) level log)
```
2025-03-04T09:13:48.385+09:00 INFO 36806 — [kafka-sample] [pool-2-thread-1] i.d.s.jdbc.history.JdbcSchemaHistory : Creating table debezium_database_history to store database history 2025-03-04T09:13:48.386+09:00 WARN 36806 — [kafka-sample] [pool-2-thread-1] i.d.s.j.offset.JdbcOffsetBackingStore : Attempt 1 to call 'initialize storage' failed. java.sql.SQLSyntaxErrorException: Column length too big for column 'history_data' (max = 16383); use BLOB or TEXT instead at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:112) ~[mysql-connector-j-9.1.0.jar:9.1.0] at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:114) ~[mysql-connector-j-9.1.0.jar:9.1.0] at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:988) ~[mysql-connector-j-9.1.0.jar:9.1.0] at com.mysql.cj.jdbc.ClientPreparedStatement.execute(ClientPreparedStatement.java:382) ~[mysql-connector-j-9.1.0.jar:9.1.0] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.lambda$initializeStorage$7(JdbcSchemaHistory.java:263) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.RetriableConnection.executeWithRetry(RetriableConnection.java:140) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.RetriableConnection.executeWithRetry(RetriableConnection.java:98) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.initializeStorage(JdbcSchemaHistory.java:255) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.lambda$start$0(JdbcSchemaHistory.java:87) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.util.FunctionalReadWriteLock.write(FunctionalReadWriteLock.java:99) ~[debezium-core-3.0.5.Final.jar:3.0.5.Final] at io.debezium.storage.jdbc.history.JdbcSchemaHistory.start(JdbcSchemaHistory.java:80) ~[debezium-storage-jdbc-3.0.6.Final.jar:3.0.6.Final] at io.debezium.relational.HistorizedRelationalDatabaseSchema.<init>(HistorizedRelationalDatabaseSchema.java:51) ~[debezium-core-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.binlog.BinlogDatabaseSchema.<init>(BinlogDatabaseSchema.java:79) ~[debezium-connector-binlog-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.mysql.MySqlDatabaseSchema.<init>(MySqlDatabaseSchema.java:41) ~[debezium-connector-mysql-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.mysql.MySqlConnectorTask.start(MySqlConnectorTask.java:99) ~[debezium-connector-mysql-3.0.5.Final.jar:3.0.5.Final] at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:253) ~[debezium-core-3.0.5.Final.jar:3.0.5.Final] at io.debezium.embedded.EmbeddedEngine.startSourceTask(EmbeddedEngine.java:640) ~[debezium-embedded-3.0.4.Final.jar:3.0.4.Final] at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:452) ~[debezium-embedded-3.0.4.Final.jar:3.0.4.Final] at io.debezium.embedded.ConvertingEngineBuilder$1.run(ConvertingEngineBuilder.java:163) ~[debezium-embedded-3.0.4.Final.jar:3.0.4.Final] at com.example.kafka_sample.handler.DebeziumHandler.start$lambda$1(DebeziumHandler.kt:72) ~[main/:na] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[na:na] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[na:na] at java.base/java.lang.Thread.run(Thread.java:840) ~[na:na]
```
How to reproduce the issue using our [tutorial](https://github.com/debezium/debezium-examples/tree/main/tutorial) deployment?
not confirmed, but if tutorial deployment is not embedded engine, the issue will not occur.
Which use case/requirement will be addressed by the proposed feature?
when using debezium embedded engine with Spring Boot and managing schema history with mysql, an error occurs when starting the application due to the size of the history_data column.
Implementation ideas (optional)
- Change the type of the history_data column from VARCHAR to TEXT