Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-6107

When using LOB support, an UPDATE against multiple rows can lead to inconsistent event data

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 2.1.3.Final, 2.2.0.Alpha2
    • 1.9.7.Final, 2.0.1.Final, 2.1.2.Final, 2.2.0.Alpha1
    • oracle-connector
    • None

      When lob.enabled is set to true, this enables the TransactionCommitConsumer to consolidate multiple events together by batching like operations from the LogMiner view into a single logical event. This is critical to combine INSERT, SEL_LOB_LOCATOR, and UPDATE events that manipulate the same row from a single logical SQL operation.

      If we were to perform the following operation:

      for (int i = 1; i <= 10; i++) {
          connection.execute("INSERT INTO dbz_test (a,b) values (" + i + ",'t" + i + "')");
      }
      connection.execute("UPDATE dbz_test SET a=12 WHERE a=1 OR a=2");
      

      The expected outcome is that we would see 12 events emitted, which is indeed the case when not setting the lob.enabled configuration option to true. However, when lob.enabled is set to true, the outcome is exactly 11 events.

      Looking at the differences more closely, the last two events when lob.enabled isn't set, is the following:

      value=Struct{before=Struct{A=1,B=t1},after=Struct{A=12,B=t1},...}
      value=Struct{before=Struct{A=2,B=t2},after=Struct{A=12,B=t2},...}
      

      But when setting lob.enabled to true, these events get consolidated into a single event and emitted as:

      value=Struct{before=Struct{A=1,B=t1},after=Struct{A=12,B=t2},...}
      

      Unfortunately, these two events should not be consolidated as they're manipulating different logical rows, and therefore the TransactionCommitConsumer should be predicated on ROW_ID when both the previous and next event in the stream are both UPDATE s.

            ccranfor@redhat.com Chris Cranford
            ccranfor@redhat.com Chris Cranford
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: