Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9098

Reduce calls to getEndOffset during kafka schema history recovery

XMLWordPrintable

    • Icon: Enhancement Enhancement
    • Resolution: Done
    • Icon: Major Major
    • 3.2.0.Beta2
    • None
    • mysql-connector
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      The recover function in KafkaSchemaHistory does the below in loop.

      1. checks the end offset, compares with last seen end offset and prints a warn log in case these are different
      2. polls topic for data and processes that data

      Steps 1 and 2, both are executed in every iteration. Step 2, is the core recovery process, however step1 is an attempt to detect cases where the history topic is being shared amongst multiple connectors. On checking CPU profile in one of the cases where history recovery was in progress for a long history topic, i saw that CPU spent in step1 was more than step2.

      To fix this and save some CPU cycles, we should not do step1 in every iteration. The CPU profile observed by me are attached.


        1. image-2025-05-30-20-24-12-504.png
          272 kB
          Animesh Kumar
        2. image-2025-05-30-20-24-24-435.png
          231 kB
          Animesh Kumar

              Unassigned Unassigned
              akanimesh7 Animesh Kumar
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: