Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4846

MongoConnector's field exclusion configuration does not work with fields with the same name but from different collections

XMLWordPrintable

    • False
    • None
    • False
    • Hide

      Set up a mongo connector field.exclude.list config with two paths having the same field name but coming from two (or more) different collections. Example:

      field.exclude.list: *.collection_1.password, *.collection_2.password  

      Notice, we are have the same field with the name password. 

      Show
      Set up a mongo connector field.exclude.list config with two paths having the same field name but coming from two (or more) different collections. Example: field.exclude.list: *.collection_1.password, *.collection_2.password   Notice, we are have the same field with the name password. 

      Using DBZ version 1.8.0.Final and when we have two fields with the same name ("description" in this case) but belonging to different collections, the connector fails with the following stacktrace:

      Caused by: java.lang.IllegalStateException: Duplicate key description (attempted merging values description and description)

 at java.base/java.util.stream.Collectors.duplicateKeyException(Collectors.java:133)

 at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)

 at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)

 at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)

 at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)

 at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)

 at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)

 at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)

 at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)

 at io.debezium.connector.mongodb.FieldSelector.fieldFilterFor(FieldSelector.java:198)

 at io.debezium.connector.mongodb.Filters.fieldFilterFor(Filters.java:96)

 at io.debezium.connector.mongodb.MongoDbSchema.lambda$schemaFor$0(MongoDbSchema.java:75)

 at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)

 at io.debezium.connector.mongodb.MongoDbSchema.schemaFor(MongoDbSchema.java:74)

 at io.debezium.connector.mongodb.MongoDbSchema.schemaFor(MongoDbSchema.java:30)

 at io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:204)

 ... 14 more

            Unassigned Unassigned
            hzitoun Hamed ZITOUN (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: