Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-5351

Log a warning when an unsupported LogMiner operation is detected for a captured table

XMLWordPrintable

      Using online mode, not RAC.

      1. Snapshot completed and streaming started.
      2. For a few days everything was streaming/syncing along well.
      3. Then after 3 days my independent rowcount rec process created an alert to show that my oracle source db had 5 more rows than what kafka received for 1 specific table. But debezium never logged such issue!
      4. After digging into what the scn on oracle side was for couple of the missing rows and then looking at v$logmnr_contents for that scn i saw some rows like this : 
      OPERATION OPERATION_CODE ROLLBACK SEG_OWNER SEG_NAME TABLE_NAME SEG_TYPE SEG_TYPE_NAME TABLE_SPACE ROW_ID USERNAME OS_USERNAME MACHINE_NAME AUDIT_SESSIONID SESSION# SERIAL# SESSION_INFO THREAD# SEQUENCE# RBASQN RBABLK RBABYTE UBAFIL UBABLK UBAREC UBASQN ABS_FILE# REL_FILE# DATA_BLK# DATA_OBJ# DATA_OBJV# DATA_OBJD# SQL_REDO SQL_UNDO RS_ID SSN CSF INFO STATUS REDO_VALUE UNDO_VALUE SAFE_RESUME_SCN CSCN OBJECT_ID EDITION_NAME CLIENT_ID SRC_CON_DBID SRC_CON_GUID CON_ID
      UNSUPPORTED 255 0 ABC REDACT REDACT 2 TABLE REDACT AALGiNAArAAD/T5AAJ UNKNOWN UNKNOWN UNKNOWN 0 0 0 UNKNOWN 1 3 8005 188433 488 33 2522166 40 29101 51 43 1045753 289128 0 2910349 Unsupported Unsupported  0x001f45.0002e011.01e8  0 0 Object or Data type Unsupported 2 823020 823021 0 5.24722E+11     UNKNOWN 0   1
      UNSUPPORTED 255 0 ABC REDACT REDACT 2 TABLE REDACT AALGiNAAAAAAAAAAAA UNKNOWN UNKNOWN UNKNOWN 0 0 0 UNKNOWN 1 4 8005 188436 56 33 2522166 40 29101 33 43 1045753 289128 0 2910349 Unsupported Unsupported  0x001f45.0002e014.0038  0 0 Object or Data type Unsupported 2 823022 823023 0 5.24722E+11     UNKNOWN 0   1

       

      Some actions I can think of:

      1. Make debezium throw an exception whenever it encounters an 'unsupported' row during mining, currently there was not even log to say it encountered this
      2. Make debezium docs refer to https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-logminer-utility.html#GUID-8A4F98EC-C233-4471-BFF9-9FB35EF5AD0D  (23.13.2
        Unsupported Data Types and Table Storage Attributes) and part about tables/cols > 30 characters long
      1. But the curious thing is I don't think I am using any of these types of things like BFILE..etc
      2. And to make things even stranger, looking at a secondtable, it has some rows as supported and some unsupported!? so the restrictions are not just table/column definition level but also row dml level? which makes me think there is oracle side bug or some more hidden restrictions oracle is not telling us...
      TABLE_NAME STATUS INFO COUNT
      secondtable 2 Object or Data type Unsupported 1
      secondtable 0   4

       

      Bonus idea:

      a) make debezium automatically detect unsupported tables before snapshot starts, at least according to the criteria in the docs...which may just be partial criteria

              ccranfor@redhat.com Chris Cranford
              tooptoop toop toop (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: