Currently the MySQL and MongoDB connector modules each have quite a few integration tests that use a simple test framework to verify the behavior of a connector under various configurations. This is necessary when writing a new connector, but as the connector becomes functional we instead want to test many more combinations of source configurations and connector configurations. Using the existing test framework would require hand-coding each of these tests to verify the expected output and behavior. I'm proposing adding another test framework that makes it easy to create new tests that are largely used for coverage and regression.

This new framework can still use the embedded connector engine, but it should define the connector configurations and all of the expected values as data. Adding a new test should be as easy as adding a new directory with a connector configuration file and a file containing the expected change events (perhaps in JSON format as an array of change event JSON documents), and then adding a small test method in a test class. (It'd be great if we can data drive this entirely and not require JUnit class changes, but then it becomes difficult to execute/debug individual test cases during development.)

There still are a lot of things to be worked out, but I'd like us to capture in this issue any requirements, goals, ideas, concerns, etc. All input is welcome.