-
Task
-
Resolution: Done
-
Minor
-
1.5.0.Beta1
-
None
-
False
-
False
-
Undefined
-
Hello!
I've configured a connector successfully, the main problem is, I have a database of 7 terabytes to pull, so it's a lot. I do not need all of this information in my database so i have configured column whitelist and filtering so I don't have to keep all of the 7tb.
I'm testing on a development environment with only 1tb. But i'm already experiencing speed problems, the messages are coming to my topics but not as fast as i expect.
I took a print from an admin (CMAK) detailing the topic metrics.
How can i improve my ingestion speed? I tried the following:
1 - The Database that i'm taking the data from is working normally
2 - The machine that is installed the Kafka Connect is only using 48% of CPU and 70% of memory
3 - The HD (I'm using EBS, seems fine too)
4 - The machine instance network limit is 10 Gbps (m5.xlarge 10 GB per second right?!)
I'm not seeing where is the speed problem? Is there a configuration to increase the speed of the connector?
Regards!