-
Task
-
Resolution: Done
-
Major
-
2.5.1.Final
-
None
-
False
-
None
-
False
Hi there.
Found the following issue for a PostgreSQL 2.5.1.Final Connector but I assume it can affect other Connectors too as it's related to the *DefaultValueConverter:
org.apache.kafka.connect.errors.DataException: Decimal value has mismatching scale for given Decimal schema. Schema has scale 0, value has scale 1.
at org.apache.kafka.connect.data.Decimal.fromLogical(Decimal.java:69)
at io.confluent.connect.avro.AvroData$5.convert(AvroData.java:293)
at io.confluent.connect.avro.AvroData.toAvroLogical(AvroData.java:1219)
at io.confluent.connect.avro.AvroData.defaultValueFromConnect(AvroData.java:1248)
at io.confluent.connect.avro.AvroData.addAvroRecordField(AvroData.java:1192)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:976)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:780)
at io.confluent.connect.avro.AvroData.fromConnectSchema(AvroData.java:765)
at io.confluent.connect.avro.AvroConverter.fromConnectData(AvroConverter.java:93)
at org.apache.kafka.connect.runtime.AbstractWorkerSourceTask.lambda$convertTransformedRecord$6(AbstractWorkerSourceTask.java:494)
...
For example, if you have a table with a NUMERIC data type defined as my_column NUMERIC(12,0) NOT NULL DEFAULT 0.0 the Connector will fail with the above exception.
Altering the default value to reflect the column's type precision and scale is a workaround: ALTER TABLE ... ALTER COLUMN ... SET DEFAULT 0;
ccranfor@redhat.com identified the cause of the issue:
so I see the issue, when we get the default value, PostgreSQL provides it as 0.0, when we then convert that to a BigDecimal in the value converter, the BigDecimal has a scale of 1 and a precision of 1, which causes a mismatch with the column's schema as the column defines a scale of 0.
We have a method withScaleAdjustedIfNeeded that adjusts the scale up to the column's precision but we don't have a check in that method to adjust the value down to the column's precision via truncation.
- links to
-
RHEA-2024:129636
Red Hat build of Debezium 2.5.4 release
