Bug report

We updated our Debezium SQL Server connector from 2.42 to 2.7.1. We are now unable to create new Debezium SQL Server connectors due to a backward incompatibility issue in the AbstractSourceInfo schema. This occurs when the AbstractSourceInfo schema is being registered in our schema registry in order to produce a schema change message to the schema history topic. Our schema registry is configured to only register schemas that are backward compatible. 

The is caused by the new ts_us and ts_ns fields that were added to the AbstractSourceInfo class in the following commit.

https://github.com/debezium/debezium/commit/6862d0498728c6692429f542acb18a15994a4f86

What Debezium connector do you use and what version?

Current version is 2.7.1

Previous version is 2.4.2

What is the captured database version and mode of deployment?

SQL Server 13.0.5108

What behavior do you expect?

The AbstractSourceInfo schema should be registered in the schema registry and the change event recorded in the schema history topic.

What behavior do you see?

The AbstractSourceInfo schema cannot be registered

[2024-08-21 12:09:06,925] ERROR Request Failed with exception  (io.confluent.rest.exceptions.DebuggableExceptionMapper)
io.confluent.kafka.schemaregistry.rest.exceptions.RestIncompatibleSchemaException: Schema being registered is incompatible with an earlier schema for subject "internal-value", details: [Found incompatible change: Difference{fullPath='#/SchemaChangeValue/Source/8', type=FIELD_SCALAR_KIND_CHANGED}, Found incompatible change: Difference{fullPath='#/SchemaChangeValue/Source/9', type=FIELD_SCALAR_KIND_CHANGED}, Found incompatible change: Difference{fullPath='#/SchemaChangeValue/Source/12', type=FIELD_SCALAR_KIND_CHANGED}, Found incompatible change: Difference{fullPath='#/SchemaChangeValue/Source/8', type=FIELD_SCALAR_KIND_CHANGED}, Found incompatible change: Difference{fullPath='#/SchemaChangeValue/Source/9', type=FIELD_SCALAR_KIND_CHANGED}, Found incompatible change: Difference{fullPath='#/SchemaChangeValue/Source/12', type=FIELD_SCALAR_KIND_CHANGED}]

Do you see the same behaviour using the latest released Debezium version?

The change is also in the main branch

How to reproduce the issue using our tutorial deployment?

Configure a schema registry to only register schemas that are backward compatible.

Create a Kafka Connect instance using that schema registry

Create a Debezium SQL Server connector using the 2.4.2 version. Upgrade to Debezium SQL Server connector to 2.7.1. Create a new connector.

Feature request or enhancement

Update the AbstractSourceInfo class to generate a backward compatible schema.

Which use case/requirement will be addressed by the proposed feature?

Allow the update of the Debezium SQL Server connector from 2.4.2 to 2.7.1 and newer vesions.