Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7562

Numeric default value decimal scale mismatch

XMLWordPrintable

      Hi there.

      Found the following issue for a PostgreSQL 2.5.1.Final Connector but I assume it can affect other Connectors too as it's related to the *DefaultValueConverter:

      org.apache.kafka.connect.errors.DataException: Decimal value has mismatching scale for given Decimal schema. Schema has scale 0, value has scale 1.
    at org.apache.kafka.connect.data.Decimal.fromLogical(Decimal.java:69)
    at io.confluent.connect.avro.AvroData$5.convert(AvroData.java:293)
    at io.confluent.connect.avro.AvroData.toAvroLogical(AvroData.java:1219)
    at io.confluent.connect.avro.AvroData.defaultValueFromConnect(AvroData.java:1248)
    at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:1192)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:976)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:780)
    at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:765)
    at io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:93)
    at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.lambda$convertTransformedRecord$6(AbstractWorkerSourceTask.java:494)
...

      For example, if you have a table with a NUMERIC data type defined as my_column NUMERIC(12,0) NOT NULL DEFAULT 0.0 the Connector will fail with the above exception.

      Altering the default value to reflect the column's type precision and scale is a workaround: ALTER TABLE ... ALTER COLUMN ... SET DEFAULT 0;

      ccranfor@redhat.com identified the cause of the issue:

      so I see the issue, when we get the default value, PostgreSQL provides it as 0.0, when we then convert that to a BigDecimal in the value converter, the BigDecimal has a scale of 1 and a precision of 1, which causes a mismatch with the column's schema as the column defines a scale of 0.

      We have a method withScaleAdjustedIfNeeded that adjusts the scale up to the column's precision but we don't have a check in that method to adjust the value down to the column's precision via truncation.

              Unassigned Unassigned
              jonathono Jonathon Ogden
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: