Details
-
Enhancement
-
Resolution: Unresolved
-
Minor
-
1.7.0.CR1
-
None
-
False
-
False
-
Undefined
Description
There are a number of strategies to solve problems with data pipelines using Debezium that can be found by searching through gitter or through various tickets. I would suggest we add a special section to the documentation called something like "Common Debezium Problem Solving Notes".
I would put a few things under this category:
- General strategy to snapshot records with the MySQL connector, since it isn't natively supported. This would explain the idea of creating a corollary snapshot connector with various changes in order to accomplish this.
- Same as above for individual or repeated snapshots for the Postgres connector.
- How to re-order events based on transaction LSN or some other fields on the consumer side, especially for use cases where multiple Kafka partitions may be used. There was some useful discussion on this in gitter as well, but nothing that appeared to be deeply well-informed by a MySQL expert who also knows the caveats of Debezium
There are other similar things we have come across where it would be useful to document common solutions. Perhaps this could simply go under the FAQ. Thoughts?
I would be willing to write some of this myself but would want direction as to where it should go and format.