Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4664

Offsets are not flushed periodically

    XMLWordPrintable

Details

    • False
    • False
    • Hide

      Set application.properties setting:
      ```
      debezium.source.offset.flush.interval.ms=60000
      ```
      Launch Debezium server container with sample Postgres container.
      Debezium server syncs the initial data within a few seconds
      No `data/offsets.dat` file is created after 60 seconds.
       

      Show
      Set application.properties setting: ``` debezium.source.offset.flush.interval.ms=60000 ``` Launch Debezium server container with sample Postgres container. Debezium server syncs the initial data within a few seconds No `data/offsets.dat` file is created after 60 seconds.  

    Description

      If there are no change records received, and the `debezium.source.offset.flush.interval.ms` has already passed, it seems there is no background thread that flushes the offsets.

      It appears as though the offset flush interval is only used to debounce repeated calls (in `maybeFlush` method of `RecordCommitter` in the EmbeddedEngine). I could not see a mechanism to periodically flush. As a result, if I set the flush interval, for example, to 60s - and the initial sync takes a few seconds, no offset will be written.

       

      Attachments

        Issue Links

          is related to

          Bug - A problem that impairs or prevents the functions of the product. DBZ-4647 snapshot_completed set to false after schema_only_recovery is run

          • Minor - To be worked after urgent, high, and medium priorities are resolved. This term is chosen when the severity of the issue is low, or the complexity or effort to correct it may be higher, relatively speaking. For low priority issues, known workarounds exist or are not needed due to the trivial effort needed to address the issue.
          • Open

          Activity

            People

              Unassigned Unassigned
              zalmane Oren Elias
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: