Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7382

Records from snapshot delivered out of order

XMLWordPrintable

      RecordsSnapshotProducerIT#shouldGenerateSnapshotAndContinueStreaming fails randomly as the records from the snapshot are not always ordered. It doesn't seem to be any issue with the test itself, but it's still not clear to me if this is an issue in out test infrastructure (embedded enegine or test utils) or in snapshot (core).

      Slightly modified test, which doesn't assert the records, but prints whole snapshot is bellow. The last records from the first table is delivered after the first record from the second table.

      2024-01-23 16:38:58,571 INFO   Postgres|test_server|0|streaming|postgres  Searching for WAL resume position   [io.debezium.connector.postgresql.PostgresStreamingChangeEventSource]
      2024-01-23 16:38:58,678 WARN   RecordsSnapshotParallelProducerIT|||test| counterVal: 0   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,678 WARN   RecordsSnapshotParallelProducerIT|||test| expectedPK: 1   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| actual: 1   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| counterVal: 1   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| expectedPK: 2   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| actual: 2   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| counterVal: 2   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| expectedPK: 3   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| actual: 1   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| counterVal: 3   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| expectedPK: 1   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| actual: 3   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| counterVal: 4   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| expectedPK: 2   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| actual: 2   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| counterVal: 5   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| expectedPK: 3   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,679 WARN   RecordsSnapshotParallelProducerIT|||test| actual: 3   [io.debezium.connector.postgresql.RecordsSnapshotParallelProducerIT]
      2024-01-23 16:38:58,793 INFO   ||||  Connection gracefully closed   [io.debezium.jdbc.JdbcConnection]
      

              vjuranek@redhat.com Vojtech Juranek
              vjuranek@redhat.com Vojtech Juranek
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: