Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7485

Builtin database name filter is incorrectly applied only to collections instead of databases in snapshot

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 2.7.0.Alpha1
    • 2.5.2.Final
    • mongodb-connector
    • None
    • False
    • None
    • False

      Filters.java

      private Predicate<String> createDatabaseFilter() {
        var dbIncludeList = config.getDbIncludeList();
        var dbExcludeList = config.getDbExcludeList();

        return Optional.<Predicate<String>> empty()
                .or(() -> dbIncludeList.map(value -> includes(value, s -> s)))
                .or(() -> dbExcludeList.map(value -> excludes(value, s -> s)))
                .orElse((db) -> true);
    }

private Predicate<CollectionId> createCollectionFilter() {
        var collectionIncludeList = config.getCollectionIncludeList();
        var collectionExcludeList = config.getCollectionExcludeList();

        final Predicate<CollectionId> collectionFilter = Optional.<Predicate<CollectionId>> empty()
                .or(() -> collectionIncludeList.map(list -> includes(list, CollectionId::namespace)))
                .or(() -> collectionExcludeList.map(list -> excludes(list, CollectionId::namespace)))
                .orElse((id) -> true)
                .and(this::isNotBuiltIn);

        // Create signal collection filter if specified and not included
        Optional<Predicate<CollectionId>> signalCollectionFilter = config.getSignalDataCollection()
                .map(CollectionId::parse)
                .filter(id -> !collectionFilter.test(id))
                .map(id -> Predicates.includes(id.namespace(), CollectionId::namespace));

        // Combine signal filter and collection filter
        return signalCollectionFilter.map(collectionFilter::or).orElse(collectionFilter);
    }

      While this doesn't break anything it requires the elevated permissions on built-in mongo database (list collections).
      No unexpected collections are read as they are always filter out by the collection filter.

            [DBZ-7485] Builtin database name filter is incorrectly applied only to collections instead of databases in snapshot

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory (Red Hat build of Debezium 2.7.3 release), and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHEA-2024:7859

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory (Red Hat build of Debezium 2.7.3 release), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHEA-2024:7859

            Debezium Builder added a comment -

            Released

            Debezium Builder added a comment - Released

            Jakub Čecháček (Inactive) added a comment -

            anmohant To also cover a situation like this

            mongodb.database.include.list=dbA,dbB
mongodb.collection.include.list=dbC.col1

            No database should be allowed (to snippet above would allow dbC). This is likely a fixed version 

            private Predicate<String> createDatabaseFilter() {
        var dbIncludeList = config.getDbIncludeList();
        var dbExcludeList = config.getDbExcludeList();
        var collectionIncludeList = config.getCollectionIncludeList();

        var dbPredicate = Optional.<Predicate<String>> empty()
                .or(() -> dbIncludeList.map(value -> includes(value, s -> s)))
                .or(() -> dbExcludeList.map(value -> excludes(value, s -> s)))
                .orElse((db) -> true);

        return  collectionIncludeList
                        .map(CollectionId::parse)
                        .map(CollectionId::dbName)
                        .map(value -> includes(value, (String s) -> s))
                .map(dbPredicate::and)
                .orElse(dbPredicate);
    }

            Jakub Čecháček (Inactive) added a comment - anmohant To also cover a situation like this mongodb.database.include.list=dbA,dbB mongodb.collection.include.list=dbC.col1 No database should be allowed (to snippet above would allow dbC). This is likely a fixed version private Predicate< String > createDatabaseFilter() { var dbIncludeList = config.getDbIncludeList(); var dbExcludeList = config.getDbExcludeList(); var collectionIncludeList = config.getCollectionIncludeList(); var dbPredicate = Optional.<Predicate< String >> empty() .or(() -> dbIncludeList.map(value -> includes(value, s -> s))) .or(() -> dbExcludeList.map(value -> excludes(value, s -> s))) .orElse((db) -> true ); return collectionIncludeList .map(CollectionId::parse) .map(CollectionId::dbName) .map(value -> includes(value, ( String s) -> s)) .map(dbPredicate::and) .orElse(dbPredicate); }

            Jakub Čecháček (Inactive) added a comment -

            Probably the fix can look like this in Filters.java

                private Predicate<String> createDatabaseFilter() {
        var dbIncludeList = config.getDbIncludeList();
        var dbExcludeList = config.getDbExcludeList();
        var collectionIncludeList = config.getCollectionIncludeList();

        return Optional.<Predicate<String>> empty()
                .or(() -> collectionIncludeList
                        .map(CollectionId::parse)
                        .map(CollectionId::dbName)
                        .map(value -> includes(value, s -> s)))
                .or(() -> dbIncludeList.map(value -> includes(value, s -> s)))
                .or(() -> dbExcludeList.map(value -> excludes(value, s -> s)))
                .orElse((db) -> true);
    }

            Jakub Čecháček (Inactive) added a comment - Probably the fix can look like this in Filters.java private Predicate< String > createDatabaseFilter() { var dbIncludeList = config.getDbIncludeList(); var dbExcludeList = config.getDbExcludeList(); var collectionIncludeList = config.getCollectionIncludeList(); return Optional.<Predicate< String >> empty() .or(() -> collectionIncludeList .map(CollectionId::parse) .map(CollectionId::dbName) .map(value -> includes(value, s -> s))) .or(() -> dbIncludeList.map(value -> includes(value, s -> s))) .or(() -> dbExcludeList.map(value -> excludes(value, s -> s))) .orElse((db) -> true ); }

            Jakub Čecháček (Inactive) added a comment -

            In MongoDbSnapshotChangeEventStource line 208

                    final List<CollectionId> collections = determineDataCollectionsToBeSnapshotted(mongo.collections(), dataCollectionPattern)

            if you follow the mongo.collections() code, you will see that we list collections on all databases – which in case when mongodb.collection.include.list is specifies also means other databases then those of these collections

            This might pose a problem if the user doesn't have permissions to interact (list collections) on other databases then those containing the allowed collection

            Jakub Čecháček (Inactive) added a comment - In MongoDbSnapshotChangeEventStource line 208 final List<CollectionId> collections = determineDataCollectionsToBeSnapshotted(mongo.collections(), dataCollectionPattern) if you follow the mongo.collections() code, you will see that we list collections on all databases – which in case when mongodb.collection.include.list is specifies also means other databases then those of these collections This might pose a problem if the user doesn't have permissions to interact (list collections) on other databases then those containing the allowed collection

            Jiri Pechanec added a comment -

            Well, there is plan for 2.5.2 as the last released version is 2.5.1 .
            Planned for last week of January.

            Jiri Pechanec added a comment - Well, there is plan for 2.5.2 as the last released version is 2.5.1 . Planned for last week of January.

            Jakub Čecháček (Inactive) added a comment -

            jpechane Is ther a plan for 2.5.3? This should be fixed and backported

            Jakub Čecháček (Inactive) added a comment - jpechane Is ther a plan for 2.5.3? This should be fixed and backported

              anmohant Anisha Mohanty
              jcechace@redhat.com Jakub Čecháček (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: