Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1854

mysql history database topic growing too large causing reboot debezium take too much time.

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Minor Minor
    • None
    • 1.0.1.Final
    • None
    • None

      We are using debzium mysql source to reading binlog.
      But just a month later, the history topic is growing so big that we have to increase the database.history.kafka.recovery.poll.interval.ms, now up to 6000 ms.

      The situation is we using table filter for only some of the binlog data.But when we are asked to adding new table , we have to re-config and restart the connector, which causing too much time for waiting it to startup .

      I was wondering can we add some tag or config to skip the history database topic checking by using the current version table schema as hole. And we can do the table version comparation manually on the consuming side as well.

      Thanks very much.

            Unassigned Unassigned
            daxingyu xingyu da (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: