Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-561

Provide option to flatten structs in MongoDB unwrapping SMT

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Major Major
    • 0.8.0.Beta1
    • None
    • mongodb-connector
    • None
    • 0
    • 0% 0%

      If there's a complex id type or other struct within a MongoDB document, the UnwrapFromMongoDbEnvelope will correctly restore a Struct from that property, but e.g. the JDBC sink connector only can deal with flat structures.

      There should be an option for the SMT for flattening out such structs by concatenating the fields into plain properties, using a (configurable) character. E.g. when receiving a document with an ide like this:

      _id : { comp: 32, cust: NumberLong("1008") }
      

      The SMT should optionally emit the following two fields:

      id_comp : 32, id_cust: 1008
      

      So we'd have two new options for the SMT:

      • flatten_structs: boolean, defaults to false
      • flattened_structs_field_concat_char: string, defaults to "_"

      Note that this option cannot be used to deal with arrays.

            spolavarapu53 Sairam Polavarapu (Inactive)
            gunnar.morling Gunnar Morling
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: