Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-396

MySQL connector "initial_only" snapshot mode results in CPU spike from ConnectorTask polling

    XMLWordPrintable

    Details

      Description

      I have been testing out the "initial_only" snapshot mode for the MySQL Connector. This mode is supposed to complete the initial snapshot and then stop. It completed the snapshot successfully, and the binglog reader was disabled as I would expect.

      However, the CPU usage spiked indefinitely, even though it should be doing nothing once complete.
      The only resolution was updating the connector configuration to the default "initial", which enables the binlog reader. The the CPU returned to normal. (Screenshot attached from JConsole)

      I briefly looked at the code for the MySQLConnectorTask, and I believe the problem is with the poll() implementation. Kafka Connect is in an infinite loop poll()-ing for records, but since the binlog reader is not running we are just immediately returning 'null', with no delay or pause. This leads to the unnecessary CPU spike.

        Attachments

          Activity

            People

            Assignee:
            pgoransson Peter Goransson
            Reporter:
            pgoransson Peter Goransson
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: