-
Enhancement
-
Resolution: Done
-
Major
-
0.1
Most Kafka clusters run with a max message size of 1MB - 2MB. I'm not sure if/how Debezium/Kafka Connect handle this now, but there should be a relief valve for messages that are larger than the limit. Common pitfalls to this are tables that have large TEXT or BLOB columns.
Some ideas:
- Add the ability to project out certain columns on a per-table basis. Users could then filter out the offending columns.
- Add the ability to drop the messages if they're too large.
- Add the ability to truncate text/blob columns.
I'm partial to have options both (1) and (3). (2) is not something that I would use.