Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1728

io.debezium.data.VariableScaleDecimal (STRUCT) type doesn't have a mapping to the SQL database column type

    Details

      Description

      We are using debezium oracle connector and one of the table have "NUMBER" datatype without scope and it throws below error. Though we have set "decimal.handling.mode":"double" in the connector config, it doesn't help. Please find the attachment for the config file.

      [2020-01-20 05:58:30,251] INFO Using PostgreSql dialect table "server1"."DOCKERDB"."SLM_STUDENT" absent (io.confluent.connect.jdbc.dialect.PostgreSqlDatabaseDialect:531)
      [2020-01-20 05:58:30,252] ERROR WorkerSinkTask{id=kloud-1} Task threw an uncaught and unrecoverable exception. Task is being killed and will not recover until manually restarted. (org.apache.kafka.connect.runtime.WorkerSinkTask:559)
      org.apache.kafka.connect.errors.ConnectException: io.debezium.data.VariableScaleDecimal (STRUCT) type doesn't have a mapping to the SQL database column type
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.getSqlType(GenericDatabaseDialect.java:1740)
              at io.confluent.connect.jdbc.dialect.PostgreSqlDatabaseDialect.getSqlType(PostgreSqlDatabaseDialect.java:219)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.writeColumnSpec(GenericDatabaseDialect.java:1656)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.lambda$writeColumnsSpec$33(GenericDatabaseDialect.java:1645)
              at io.confluent.connect.jdbc.util.ExpressionBuilder.append(ExpressionBuilder.java:558)
              at io.confluent.connect.jdbc.util.ExpressionBuilder$BasicListBuilder.of(ExpressionBuilder.java:597)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.writeColumnsSpec(GenericDatabaseDialect.java:1647)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.buildCreateTableStatement(GenericDatabaseDialect.java:1570)
              at io.confluent.connect.jdbc.sink.DbStructure.create(DbStructure.java:91)
              at io.confluent.connect.jdbc.sink.DbStructure.createOrAmendIfNecessary(DbStructure.java:61)
              at io.confluent.connect.jdbc.sink.BufferedRecords.add(BufferedRecords.java:121)
              at io.confluent.connect.jdbc.sink.JdbcDbWriter.write(JdbcDbWriter.java:66)
              at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:74)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:539)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:322)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
              at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
              at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      [2020-01-20 05:58:30,253] ERROR WorkerSinkTask{id=kloud-1} Task threw an uncaught and unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:179)
      org.apache.kafka.connect.errors.ConnectException: Exiting WorkerSinkTask due to unrecoverable exception.
              at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:561)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.poll(WorkerSinkTask.java:322)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:224)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:192)
              at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:177)
              at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:227)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
              at java.lang.Thread.run(Thread.java:748)
      Caused by: org.apache.kafka.connect.errors.ConnectException: io.debezium.data.VariableScaleDecimal (STRUCT) type doesn't have a mapping to the SQL database column type
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.getSqlType(GenericDatabaseDialect.java:1740)
              at io.confluent.connect.jdbc.dialect.PostgreSqlDatabaseDialect.getSqlType(PostgreSqlDatabaseDialect.java:219)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.writeColumnSpec(GenericDatabaseDialect.java:1656)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.lambda$writeColumnsSpec$33(GenericDatabaseDialect.java:1645)
              at io.confluent.connect.jdbc.util.ExpressionBuilder.append(ExpressionBuilder.java:558)
              at io.confluent.connect.jdbc.util.ExpressionBuilder$BasicListBuilder.of(ExpressionBuilder.java:597)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.writeColumnsSpec(GenericDatabaseDialect.java:1647)
              at io.confluent.connect.jdbc.dialect.GenericDatabaseDialect.buildCreateTableStatement(GenericDatabaseDialect.java:1570)
              at io.confluent.connect.jdbc.sink.DbStructure.create(DbStructure.java:91)
              at io.confluent.connect.jdbc.sink.DbStructure.createOrAmendIfNecessary(DbStructure.java:61)
              at io.confluent.connect.jdbc.sink.BufferedRecords.add(BufferedRecords.java:121)
              at io.confluent.connect.jdbc.sink.JdbcDbWriter.write(JdbcDbWriter.java:66)
              at io.confluent.connect.jdbc.sink.JdbcSinkTask.put(JdbcSinkTask.java:74)
              at org.apache.kafka.connect.runtime.WorkerSinkTask.deliverMessages(WorkerSinkTask.java:539)
              ... 10 more
      [2020-01-20 05:58:30,253] ERROR WorkerSinkTask{id=kloud-1} Task is being killed and will not recover until manually restarted (org.apache.kafka.connect.runtime.WorkerTask:180)
      [2020-01-20 05:58:30,253] INFO Stopping task (io.confluent.connect.jdbc.sink.JdbcSinkTask:105)
      

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  jpechanec Jiri Pechanec
                  Reporter:
                  pshussain Mohamed Pudukulathan
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  2 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved: