Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1297

Expose outbox event structure at level of Kafka Connect messages

    XMLWordPrintable

Details

    Description

      Currently, the payload structure of outbox events is fully opaque at the Kafka message level, as it's a string-ified JSON field (or, base64-encoded Avro as one user reported to do).

      It'd be useful for consumers, if there was more explicit schema support for that payload structure. I.e. the payload schema should be part of the event schema itself (by "expanding" the single payload field into a properly typed Kafka Connect data structure). Depending on the chosen serializer, the outbox event payload schema then would be part of the JSON schema sent with each message or it would be registered with the schema registry when using Avro.

      An open question is how to convey the schema information from the application writing to the outbox table to the Debezium outbox router (which would do this "expansion" before sending messages to Kafka). Similar to the existing serialization options in Kafka Connect, one could think of passing the JSON schema in a separate column (when using JSON as payload) or a schema id resolvable against the registry (when using Avro).

      Attachments

        Issue Links

          Activity

            People

              ccranfor@redhat.com Chris Cranford
              gunnar.morling Gunnar Morling
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: