-
Bug
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
None
-
False
-
Informational
At the very least, the documentation seems to be wrong although potentially there's issues in the codebase with handing the *.adjustment.mode configuration properties.
Documentation shows them as Connector level configuration properties but they only seem to apply if configured as part of the ExtractNewDocumentState SMT.
What Debezium connector do you use and what version?
v2.5.4.Final
What is the connector configuration?
"name": "my_connector", "transforms": "unwrap", "transforms.unwrap.type": "io.debezium.connector.mongodb.transforms.ExtractNewDocumentState", "transforms.unwrap.drop.tombstones": "true", "transforms.unwrap.add.headers": "op", "transforms.unwrap.add.fields": "source.ts_ms,ts_ms", "transforms.unwrap.delete.handling.mode": "rewrite", "field.name.adjustment.mode": "avro", "schema.name.adjustment.mode": "avro", ...
What behavior do you expect?
Collections and fields with invalid characters e.g. whitespace in the name to be cleansed. For example:
"My Collection" should become "My_Collection"
"My badly !named field" should become "My_badly__named_field"
What behavior do you see?
No name change. Not unless the Connector configuration is adapted as follows:
"name": "my_connector", "transforms": "unwrap", "transforms.unwrap.type": "io.debezium.connector.mongodb.transforms.ExtractNewDocumentState", "transforms.unwrap.drop.tombstones": "true", "transforms.unwrap.add.headers": "op", "transforms.unwrap.add.fields": "source.ts_ms,ts_ms", "transforms.unwrap.delete.handling.mode": "rewrite", "transforms.unwrap.field.name.adjustment.mode": "avro", "transforms.unwrap.schema.name.adjustment.mode": "avro", ...
After the configuration is changed, when invalid names are encountered for schema or field, the following warnings appear in the logs:
WARN [debezium-mongodbconnector-myconnector-change-event-source-coordinator] [myconnector|task-0] Topic 'myconnector.Test.Test Collection6' name isn't a valid topic name, replacing it with 'myconnector.Test.Test_Collection6'. (io.debezium.schema.AbstractTopicNamingStrategy) ... WARN [task-thread-myconnector-0] [myconnector|task-0] The Kafka Connect schema name 'bad col name' is not a valid Avro schema name, so replacing with 'bad_col_name' (io.debezium.schema.SchemaNameAdjuster)
Do you have the connector logs, ideally from start till finish?
Yes
