I was using the test-jar for embedding a Kafka cluster in my own application (within tests) and found this (quite annoying) bug with KafkaCluster.

Problem : you can pass whatever property within withKafkaConfiguration(), they won't appear in the Kafka config.

My diagnosis :
When using "withKafkaConfiguration", the KafkaCluster creates "new Properties(kafkaConfig)", which means, an empty Properties object, with default values pointing to KafkaConfig.

This makes absolute sense, but if you pay attention here the KafkaConfig constructor which is called is "KafkaConfig(Map)". So the properties are used by Kafka as a Map. Kafka then iterates over the map keys (see org.apache.kafka.common.config.AbstractConfig), and by doing so, ignores the "defaults" which were set by withKafkaConfiguration.

Hope this helps.