Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4547

MySQL history topic creation needs DESCRIBE_CONFIGS at the Cluster level

    XMLWordPrintable

Details

    Description

      Hello,

      I'm using debezium with a SaaS Kafka instance where I've got limited permissions usually scoped at my namespace.

      It  means Kafka-ACLS are set to topic level only.

      For MySQL replication, Debezium needs to create an history topic to log schema changes.

      When the history admin client tries to create the topic, it tries to get the broker default replication factor and fails as it does not have the permission.

      https://github.com/debezium/debezium/blob/main/debezium-core/src/main/java/io/debezium/relational/history/KafkaDatabaseHistory.java#L498

      https://github.com/debezium/debezium/blob/main/debezium-core/src/main/java/io/debezium/relational/history/KafkaDatabaseHistory.java#L516

       

      There is no parameter at the connector level to override this.

      Also a simpler method to resolve this is just to use the constructor with optional replication factor :

      public NewTopic​(String name, Optional<Integer> numPartitions, Optional<Short> replicationFactor)
      
      A new topic that optionally defaults numPartitions and replicationFactor to the broker configurations
      for num.partitions and default.replication.factor respectively. 

      https://kafka.apache.org/30/javadoc/org/apache/kafka/clients/admin/NewTopic.html#%3Cinit%3E(java.lang.String,java.util.Optional,java.util.Optional)
       
      Thank you for your help.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              fuyar UYAR Farid
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: