Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4277

[Debezium Server] Event Hubs exporter slow/Event data was too large

    XMLWordPrintable

Details

    • False
    • False

    Description

      Hello, I am using Debezium Server for pushing events to eventhubs. I notices that eventhubs is a lot slower than using kafka and it doesn't seem to be related to networking.

       

      I have to lower my max.batch.size to 300 to get it to not throw an error. I want to increase it as much as possible because that usually helps speed things up. but if the value it set too high, i get an error. Shouldn't logic be to send whatever it can, it shouldn't fail if it can't send the max.batch.size number of events.

       

      Also please let me know if there are any other fields to help increase the speed of transfer other than max.batch.size and poll.interval.ms. queue.size and snaphot.fetch.size seems to have no major effects.

       

      Also:

      For kafka (eventhubs kafka api), I get around around 16k/s, but using eventhubs (amqp), i get 6k/s. Perhaps there is something I am missing?

      Attachments

        Activity

          People

            Unassigned Unassigned
            bubbbajoe Joe Williams (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: