Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-232

MySqlConnector Table and Database recommenders cause timeouts on large instances

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Major
    • 0.5.1
    • 0.5
    • mysql-connector
    • None

    Description

      I am getting unavoidable timeouts caused by the TableRecommender and DatabaseRecommender during config validation. Due to the size of my MySql instances the queries these classes run during validation are running into the Kafka Connect hardcoded 90 second timeout when trying to enumerate every database and every table.

      I'm currently using a patched build of debezium that removes these recommenders. Maybe these recommenders could be changed to just a short selection database and table names instead of everything?

      For my specific case I am dealing with MySql instances that contain hundreds of databases each with hundreds of tables. I'm happy to help try to write this code but I wanted to get some feedback on this. Unfortunately changing the size or configuration of the MySql instances is not a viable solution as it is outside the scope of my responsibilities.

      Attachments

        Activity

          People

            rhauch Randall Hauch (Inactive)
            tholmes_jira Thomas Holmes (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: