Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8570

Reduce frequency table exists and column metadata queries

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Major Major
    • 3.1.0.CR1
    • None
    • jdbc-connector
    • None

      Presently, the connector will always check the JDBC driver metadata if a table exists and what its column details are, even when schema.evolution is disabled. When the user sends many rows in a small window, this can create an excessive CPU load on the target database.

      See https://debezium.zulipchat.com/#narrow/channel/302529-community-general/topic/How.20to.20Reduce.20Metadata.20Queries.20in.20Debezium.20JDBC.20Sink.20Connect/near/493251086

      We should consider caching TableDescriptor objects so that we reduce the number of calls to Dialect#tableExists and Dialect#readTable and avoid doing this on each buffer flush, which can be frequent.

              Unassigned Unassigned
              ccranfor@redhat.com Chris Cranford
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: