Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7419

New functionality of ExtractNewRecordState is not fully compatible with deprecated one

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Major Major
    • None
    • 2.5.0.Final
    • core-library
    • None
    • False
    • None
    • False

      In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      SQL Server connector from Debezium 2.5 release

      What is the connector configuration?

          transforms:                                     unwrap-key,unwrap-value,reroute
          ...
          transforms.unwrap-value.type:                   io.debezium.transforms.ExtractNewRecordState
          transforms.unwrap-value.delete.handling.mode:   none
          ...
      

      What is the captured database version and mode of depoyment?

      on-premise, not sure what is the exact version

      What behaviour do you expect?

      I want to switch to delete.tombstone.handling.mode property to avoid deprecation warnings in the logs and avoid any changes in the consumers processing data enriched by a new SMT version.

      What behaviour do you see?

      {
        ...
        "file": "AbstractExtractNewRecordState.java",
        "logger_name": "io.debezium.transforms.AbstractExtractNewRecordState",
        "message": "The deleted record handling configs \"drop.tombstones\" and \"delete.handling.mode\" have been deprecated, please use \"delete.tombstone.handling.mode\" instead.",
        "class": "io.debezium.transforms.AbstractExtractNewRecordState",
      }
      

      And new functionality doesn't allow to leave deletes only without additional __deleted: true field:

      public static final Field HANDLE_TOMBSTONE_DELETES = Field.create("delete.tombstone.handling.mode")
          .withDisplayName("Handle delete records")
          .withEnum(DeleteTombstoneHandling.class)
          .withWidth(ConfigDef.Width.MEDIUM)
          .withImportance(ConfigDef.Importance.MEDIUM)
          .optional()
          .withDescription("How to handle delete records. Options are: "
              + "drop - Remove the delete event and tombstone from the stream."
              + "tombstone (default) - For each delete event, leave only a tombstone in the stream."
              + "rewrite - Remove tombstone from the record, and add a `__deleted` field with the value `true`."
              + "rewrite-with-tombstone - Retain tombstone in record and add a `__deleted` field with the value `true`.");
      

      Do you see the same behaviour using the latest relesead Debezium version?

      Nothing has changed there.

      Do you have the connector logs, ideally from start till finish?

      It's not necessary. We convert the logs to JSON and get something like that:

      {
        "@timestamp": "2024-01-17T15:07:28.452Z",
        "source_host": "cxp-connect-connect-0",
        "file": "AbstractExtractNewRecordState.java",
        "method": "configure",
        "level": "WARN",
        "line_number": "101",
        "thread_name": "StartAndStopExecutor-connect-1-3",
        "@version": 1,
        "logger_name": "io.debezium.transforms.AbstractExtractNewRecordState",
        "message": "The deleted record handling configs \"drop.tombstones\" and \"delete.handling.mode\" have been deprecated, please use \"delete.tombstone.handling.mode\" instead.",
        "class": "io.debezium.transforms.AbstractExtractNewRecordState",
        "mdc": {
          "connector.context": "[market.regional-controller|task-0] "
        }
      }
      

      How to reproduce the issue using our tutorial deployment?

      Just add `io.debezium.transforms.ExtractNewRecordState` transformation with deprecated setting. For example:

      delete.handling.mode:   none
      

              Unassigned Unassigned
              mikekamornikov Mikhail Kamornikau (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: