Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3250

Add snapshot.select.statement.override options to Oracle documentation

    XMLWordPrintable

Details

    Description

      Hello!

      I've configured a connector successfully, the main problem is, I have a database of 7 terabytes to pull, so it's a lot. I do not need all of this information in my database so i have configured column whitelist and filtering so I don't have to keep all of the 7tb.

      I'm testing on a development environment with only 1tb. But i'm already experiencing speed problems, the messages are coming to my topics but not as fast as i expect.

      I took a print from an admin (CMAK) detailing the topic metrics.

      How can i improve my ingestion speed? I tried the following:

      1 - The Database that i'm taking the data from is working normally
      2 - The machine that is installed the Kafka Connect is only using 48% of CPU and 70% of memory
      3 - The HD (I'm using EBS, seems fine too)
      4 - The machine instance network limit is 10 Gbps (m5.xlarge 10 GB per second right?!)

      I'm not seeing where is the speed problem? Is there a configuration to increase the speed of the connector?

      Regards!

      Attachments

        Activity

          People

            ccranfor@redhat.com Chris Cranford
            thiago-ananias thiago ananias silva (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: