Details
-
Bug
-
Resolution: Done
-
Critical
-
1.9.4.Final
-
None
-
False
-
None
-
False
Description
Using online mode, not RAC.
- Snapshot completed and streaming started.
- For a few days everything was streaming/syncing along well.
- Then after 3 days my independent rowcount rec process created an alert to show that my oracle source db had 5 more rows than what kafka received for 1 specific table. But debezium never logged such issue!
- After digging into what the scn on oracle side was for couple of the missing rows and then looking at v$logmnr_contents for that scn i saw some rows like this :
| OPERATION | OPERATION_CODE | ROLLBACK | SEG_OWNER | SEG_NAME | TABLE_NAME | SEG_TYPE | SEG_TYPE_NAME | TABLE_SPACE | ROW_ID | USERNAME | OS_USERNAME | MACHINE_NAME | AUDIT_SESSIONID | SESSION# | SERIAL# | SESSION_INFO | THREAD# | SEQUENCE# | RBASQN | RBABLK | RBABYTE | UBAFIL | UBABLK | UBAREC | UBASQN | ABS_FILE# | REL_FILE# | DATA_BLK# | DATA_OBJ# | DATA_OBJV# | DATA_OBJD# | SQL_REDO | SQL_UNDO | RS_ID | SSN | CSF | INFO | STATUS | REDO_VALUE | UNDO_VALUE | SAFE_RESUME_SCN | CSCN | OBJECT_ID | EDITION_NAME | CLIENT_ID | SRC_CON_DBID | SRC_CON_GUID | CON_ID |
| UNSUPPORTED | 255 | 0 | ABC | REDACT | REDACT | 2 | TABLE | REDACT | AALGiNAArAAD/T5AAJ | UNKNOWN | UNKNOWN | UNKNOWN | 0 | 0 | 0 | UNKNOWN | 1 | 3 | 8005 | 188433 | 488 | 33 | 2522166 | 40 | 29101 | 51 | 43 | 1045753 | 289128 | 0 | 2910349 | Unsupported | Unsupported | 0x001f45.0002e011.01e8 | 0 | 0 | Object or Data type Unsupported | 2 | 823020 | 823021 | 0 | 5.24722E+11 | UNKNOWN | 0 | 1 | |||
| UNSUPPORTED | 255 | 0 | ABC | REDACT | REDACT | 2 | TABLE | REDACT | AALGiNAAAAAAAAAAAA | UNKNOWN | UNKNOWN | UNKNOWN | 0 | 0 | 0 | UNKNOWN | 1 | 4 | 8005 | 188436 | 56 | 33 | 2522166 | 40 | 29101 | 33 | 43 | 1045753 | 289128 | 0 | 2910349 | Unsupported | Unsupported | 0x001f45.0002e014.0038 | 0 | 0 | Object or Data type Unsupported | 2 | 823022 | 823023 | 0 | 5.24722E+11 | UNKNOWN | 0 | 1 |
Some actions I can think of:
- Make debezium throw an exception whenever it encounters an 'unsupported' row during mining, currently there was not even log to say it encountered this
- Make debezium docs refer to https://docs.oracle.com/en/database/oracle/oracle-database/19/sutil/oracle-logminer-utility.html#GUID-8A4F98EC-C233-4471-BFF9-9FB35EF5AD0D (23.13.2
Unsupported Data Types and Table Storage Attributes) and part about tables/cols > 30 characters long
- But the curious thing is I don't think I am using any of these types of things like BFILE..etc
- And to make things even stranger, looking at a secondtable, it has some rows as supported and some unsupported!? so the restrictions are not just table/column definition level but also row dml level? which makes me think there is oracle side bug or some more hidden restrictions oracle is not telling us...
| TABLE_NAME | STATUS | INFO | COUNT |
| secondtable | 2 | Object or Data type Unsupported | 1 |
| secondtable | 0 | 4 |
Bonus idea:
a) make debezium automatically detect unsupported tables before snapshot starts, at least according to the criteria in the docs...which may just be partial criteria 
