Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-396

MySQL connector "initial_only" snapshot mode results in CPU spike from ConnectorTask polling

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 0.6.2
    • 0.6
    • mysql-connector
    • None

      I have been testing out the "initial_only" snapshot mode for the MySQL Connector. This mode is supposed to complete the initial snapshot and then stop. It completed the snapshot successfully, and the binglog reader was disabled as I would expect.

      However, the CPU usage spiked indefinitely, even though it should be doing nothing once complete.
      The only resolution was updating the connector configuration to the default "initial", which enables the binlog reader. The the CPU returned to normal. (Screenshot attached from JConsole)

      I briefly looked at the code for the MySQLConnectorTask, and I believe the problem is with the poll() implementation. Kafka Connect is in an infinite loop poll()-ing for records, but since the binlog reader is not running we are just immediately returning 'null', with no delay or pause. This leads to the unnecessary CPU spike.

              pgoransson@cainc.com Peter Goransson (Inactive)
              pgoransson@cainc.com Peter Goransson (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: