Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-561

Provide option to flatten structs in MongoDB unwrapping SMT

    XMLWordPrintable

    Details

      Description

      If there's a complex id type or other struct within a MongoDB document, the UnwrapFromMongoDbEnvelope will correctly restore a Struct from that property, but e.g. the JDBC sink connector only can deal with flat structures.

      There should be an option for the SMT for flattening out such structs by concatenating the fields into plain properties, using a (configurable) character. E.g. when receiving a document with an ide like this:

      _id : { comp: 32, cust: NumberLong("1008") }
      

      The SMT should optionally emit the following two fields:

      id_comp : 32, id_cust: 1008
      

      So we'd have two new options for the SMT:

      • flatten_structs: boolean, defaults to false
      • flattened_structs_field_concat_char: string, defaults to "_"

      Note that this option cannot be used to deal with arrays.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              spolavarapu53 Sairam Polavarapu (Inactive)
              Reporter:
              gunnar.morling Gunnar Morling
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: