Details
-
Feature Request
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
None
-
False
-
0
-
0%
Description
Some logging, can be particularly strong perf hit, for example all of the ones that have to be logged for every row.
E.g. In my case I have a column that for historical reason has a lot of 0000-00-00 00:00:00, which is an invalid value for most DB, debezium included:
if (year == 0 || month == 0 || day == 0) { LOGGER.warn("Invalid value '{}' stored in column '{}' of table '{}' converted to empty value", timestampString, column.name(), table.id()); return true; }
In my case, this gets logged up to 50-100 times per MILLISECOND, and tens of thousands per second, consuming basically all cycles
A solution such as:
https://github.com/Swrve/rate-limited-logger#sample-output
that ratelimited even at levels such 50-300/s, would already be a great improvement.
if (year == 0 || month == 0 || day == 0) { CONVERSION_LOGGER_RL.warn("Invalid value '{}' stored in column '{}' of table '{}' converted to empty value", timestampString, column.name(), table.id()); return true; }
Would you agree in adding this?