Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3749

JsonSerde overwrites configuration when used with Spring Cloud Stream

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • 1.9-backlog
    • 1.6.0.Final
    • core-library
    • None
    • False
    • False
    • Undefined

    Description

      When debezium is used with Spring Cloud Stream and Debezium's JsonSerde is used, Serde's configuration get overwritten.

      https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/blob/f7537e795e6c1833405a56e6e7ce86e84612799e/spring-cloud-stream-binder-kafka-streams/src/main/java/org/springframework/cloud/stream/binder/kafka/streams/KeyValueSerdeResolver.java#L405-L426

      Here line 425 calls configure method on already configured Serde with different set of properties.

      valueSerde.configure(this.streamConfigGlobalProperties, false);
      

      https://github.com/debezium/debezium/blob/5b757372d82f79d314f0392b516cf63245fc72a9/debezium-core/src/main/java/io/debezium/serde/json/JsonSerde.java#L54

      JsonSerde on line 54,  discard old config and overwrites it with new config. Ideally it should merge old config with new one while maintaining precedence.

      In my case from.field was removed from config when Spring Cloud Stream adds global configurations to existing Serde.

      I worked around this issue by extending JsonSerde in following way. This may not be perfect but solves my issue for time being.

      public class DebeziumJsonSerde<T> extends JsonSerde<T> {
      
          private final Map<String, Object> configs = new HashMap<>();
      
          public DebeziumJsonSerde(Class<T> type) {
              super(type);
          }
      
          @Override
          public void configure(final Map<String, ?> configs, boolean isKey) {
              // First store config to a local map and then pass it on to the super method.
              // Everytime configure is called keep appending new config to old one and call super method.
              this.configs.putAll(configs);
              super.configure(this.configs, isKey);
          }
      }
      

      Attachments

        Activity

          People

            Unassigned Unassigned
            jigarcaptain Jigar Captain (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: