-
Bug
-
Resolution: Unresolved
-
Major
-
1.9.5.Final
-
None
-
False
-
None
-
False
-
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
What Debezium connector do you use and what version?
Postgres Connector v1.9.5.Final
What is the connector configuration?
connector.class: "io.debezium.connector.postgresql.PostgresConnector"
database.history.file.filename: "/home/appRunner/debezium-publishers/dbhistory-test.dat"
database.dbname: "redacted"
database.user: "redacted"
database.port: "5432"
database.hostname: "redacted"
database.password: "redacted"
converter.schemas.enable: "false"
offset.storage: "org.apache.kafka.connect.storage.FileOffsetBackingStore"
database.server.name: "debezium-test-connector"
plugin.name: "pgoutput"
offset.flush.interval.ms: "1000"
offset.storage.file.filename: "/home/appRunner/debezium-publishers/offsets-test.dat"
name: "debezium-publisher-test"
database.history: "io.debezium.relational.history.FileDatabaseHistory"
snapshot.mode: "never"
publication.name: "redacted"
slot.name: "redacted"
publication.autocreate.mode: "disabled"
decimal.handling.mode: "double"
binary.handling.mode: "hex"
time.precision.mode: "adaptive_time_microseconds"
What is the captured database version and mode of depoyment?
Postgres 10.12 deployed in a cloudsql instance (GCP)
What behaviour do you expect?
The publisher should be able to parse all columns in the row and send the message with the appropriate value
What behaviour do you see?
Occasionally the publisher throws the following error
org.apache.kafka.connect.errors.DataException: Invalid Java object for schema with type ARRAY: class java.lang.String for field: "ipAddresses"
and that particular field is assigned null value in the debezium message (i think)
the column is of type _text
The column value displayed in the logs is just a generic java object
java.lang.Object@3777dc16
This happens when there's a very large value in the text array column (it gets toasted by postgres). For a consequent update of that row, instead of returning the toasted value placeholder, it throws error the above error.
Do you see the same behaviour using the latest released Debezium version?
Same issue in latest stable release (1.9.5.Final), yet to check in non stable releases
Do you have the connector logs, ideally from start till finish?
Attached
How to reproduce the issue using our tutorial deployment?
- Create a table with multiple columns, including a column of type _text
- Insert a row with a very large value for the _text column (it should get toasted)
- Execute an update for that particular row, but for any other column other than the _text column
- Debezium will throw the error, and will place a null value in the debezium message for the _text column, instead of the toasted value placeholder