Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4413

MongoDB ExtractNewDocumentState SMT overwrites existing document ID field

    XMLWordPrintable

Details

    Description

      We encounter an undesired behaviour of the SMT given the following input documents (other props removed for brevity):

      MongoDB original document:

      {
          "_id" : ObjectId("61af15fcac9f5896f6c2d03f"),
          "id" : "CRSRF-a610fbe0-be92-4dbf-b333-e2c76f71f361"
          "vendor_id": "a610fbe0-be92-4dbf-b333-e2c76f71f361",
      }

      Document in Kafka topic:

      {
        "id": "61af15fcac9f5896f6c2d03f",
        "vendor_id": "a610fbe0-be92-4dbf-b333-e2c76f71f361",
      }

      Please note, that the SMT decides to overwrite the `id` property in the resulting document due to here and here.

      Although the schema of the document is not well designed, I would argue, that DBZ should not overwrite document properties by default. I would either assume that this is configurable (e.g. write _id in to doc as "foo") or at least should be behind a toggle.

      jpechane and I had a conversation about this here and we agreed that I hand in a patch that makes the behaviour configurable, keeping the current behaviour as its default for backwards compatibility.

      Attachments

        Activity

          People

            Unassigned Unassigned
            mario@xenji.com Mario Mueller (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: