-
Bug
-
Resolution: Done
-
Undefined
-
2.4.0.GA
-
None
-
False
-
None
-
False
-
-
-
- [DOC] log.cleaner.enable doesn't work for log.cleanup.policy=delete topics
- The following description is misleading, as if log.cleaner.enable=false would not delete segments that exceed retention for topics with log.cleanup.policy=delete. However, log.cleaner.enable=false doesn't work for any log.cleanup.policy=delete topics. As far as I know, there is no configuration that stops the segment deletion of topics with log.cleanup.policy=delete that exceed retention or bytes, except for maximizing their values.
If cleanup policy is set to delete logs, older segments are deleted based on log retention limits. Otherwise, if the log cleaner is not enabled, and there are no log retention limits, the log will continue to grow.
- The following description is misleading, as if log.cleaner.enable=false would not delete segments that exceed retention for topics with log.cleanup.policy=delete. However, log.cleaner.enable=false doesn't work for any log.cleanup.policy=delete topics. As far as I know, there is no configuration that stops the segment deletion of topics with log.cleanup.policy=delete that exceed retention or bytes, except for maximizing their values.
-
-
- In addition, log.cleanup.policy should not be changed because of the cluster-wide configuration. Generally, delete policy and compact policy are completely different use cases, generally log.cleanup.policy=delete is kept (for topics to use retention time and retention bytes), and cleanup.policy, which is topic-level configuration, is changed only for the topics that need compact policy.
- log.cleaner.enable(1) does work only for log.cleanup.policy=compact topics(, and log.cleanup.policy=delete,compact topics). In a narrow sense, log cleaner is only for compaction, not delete policy.
-
- (1) log.cleaner.enable: https://kafka.apache.org/documentation.html#brokerconfigs_log.cleaner.enable
Enable the log cleaner process to run on the server. Should be enabled if using any topics with a cleanup.policy=compact including the internal offsets topic. If disabled those topics will not be compacted and continually grow in size.
- relates to
-
-
- Closed
-