Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9328

Add support for tracing.* config options in MongoEventRouter

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Major Major
    • 3.3.0.Beta1
    • 3.1.2.Final
    • mongodb-connector
    • None
    • Important

      When using the MongoDB source connector with the Outbox Event Router SMT, tracing configuration properties appear to be ignored.

      Connector configuration

       
      "connector.class": "io.debezium.connector.mongodb.MongoDbConnector",
      "mongodb.connection.string": "mongodb://mongo:27017/?replicaSet=rs0",
      "topic.prefix": "test",
      "database.include.list": "outbox",
      "collection.include.list": "outbox.loans",
      "mongodb.user": "mongo-cdc",
      "mongodb.password": "password",
      "value.converter": "org.apache.kafka.connect.json.JsonConverter",
      "value.converter.schemas.enable": "false",
      "transforms": "outbox",
      "transforms.outbox.type": "io.debezium.connector.mongodb.transforms.outbox.MongoEventRouter",
      "transforms.outbox.tracing.span.context.field": "propagation",
      "transforms.outbox.tracing.with.context.field.only": "false",
      "transforms.outbox.tracing.operation.name": "debezium-read",
      "transforms.outbox.collection.field.event.key": "aggregateId",
      "transforms.outbox.collection.field.event.id": "aggregateId",
      "transforms.outbox.collection.field.event.payload": "eventPayload",
      "transforms.outbox.collection.expand.json.payload": "true",
      "transforms.outbox.collection.fields.additional.placement": "aggregateType:header:aggregate-type,aggregateId:header:aggregate-id,eventType:header:event-type",
      "transforms.outbox.route.by.field": "topicName",
      "transforms.outbox.route.topic.replacement": "${routedByValue}",
      "errors.tolerance": "all",
      "errors.log.enable": "true",
      "errors.log.include.messages": "true"

      • Debezium Version: 3.1.2
      • MongoDB Version: 7.0.16 Community (Docker Compose)
      • Connector Class: io.debezium.connector.mongodb.MongoDbConnector
      • Transform: io.debezium.connector.mongodb.transforms.outbox.MongoEventRouter
      • Expected Behavior: Tracing configuration is respected and traceparent is propagated correctly from the outbox event.
      • Observed Behavior: A new traceparent is created for each event, and tracing context is not propagated as expected.

       Repro Steps:

      1. Configure the MongoDB source connector with the MongoEventRouter SMT.
      1. Enable OpenTelemetry tracing with the SMT configuration.
      1. Emit an event from the MongoDB outbox collection with a traceparent field.
      1. Observe that the emitted Kafka message has a new traceparent rather than propagating the original.

      Diagnosis and Findings:

      It seems that class:

          MongoEventRouter.java

      in the method: 

          private <T> Map<String, T> convertConfigMap(Map<String, T> oldConfigMap)

      Only retrieves a subset of keys (and doesn't convert the tracing keys):

       
      Map<String, T> newConfigMap = new HashMap<>();
          for (String config : oldConfigMap.keySet()) {
              if (fieldNameConverter.containsKey(config))

      {             newConfigMap.put(fieldNameConverter.get(config), oldConfigMap.get(config));         }

      }
      All other keys are dropped.

      This config is then sent to 
      EventRouterDelegate tha use the map (with missing configs) to setup the ActivateTracingSpan:
       
      public void configure(Map<String, ?> configMap) {
      if (ActivateTracingSpan.isOpenTelemetryAvailable()) {
         tracingSmt.configure(configMap);
         if(!configMap.containsKey(ActivateTracingSpan.TRACING_CONTEXT_FIELD_REQUIRED.name()))

      {     tracingSmt.setRequireContextField(true);ActivateTracingSpan }

      }
      [...]
       
      Basically can't configure custom values for:
       
      tracing.span.context.field
      tracing.with.context.field.only
      tracing.operation.name

              Unassigned Unassigned
              eljeko Stefano Linguerri (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: