-
Task
-
Resolution: Done
-
Major
-
None
-
None
-
False
-
None
-
False
What Debezium connector do you use and what version?
Oracle - 2.3.1 Final
What is the connector configuration?
name=enquesta-cdc-connector-1
connector.class=io.debezium.connector.oracle.OracleConnector
tasks.max=1
database.server.name=sdsenq
database.hostname=redacted
database.port=1521
database.user=redacted
database.password=redacted
database.dbname=sdsenq
table.include.list=DBZUSER.DEBEZIUM_SIGNAL,ENQUESTACIS.CI911DISTRICT,ENQUESTACIS.CIACCOUNTACTIVECODE,ENQUESTACIS.CIACCOUNTMETERSTATUS,ENQUESTACIS.CIACCOUNTSTATUS,ENQUESTACIS.CIAGE,ENQUESTACIS.CIAMITYPE,ENQUESTACIS.CIAPPLICATION,ENQUESTACIS.CIAPPTAXDISTRICT,ENQUESTACIS.CIASSETTAXDISTRICT,ENQUESTACIS.CIBALANCEBREAKDOWN,ENQUESTACIS.CIBILLINGCYCLE,ENQUESTACIS.CICLASSTYPE,ENQUESTACIS.CICONSTRUCTIONTYPE,ENQUESTACIS.CICOUNTYCODE,ENQUESTACIS.CIDEVICECONFIG,ENQUESTACIS.CIDEVICELOCATION,ENQUESTACIS.CIDEVICEMODEL,ENQUESTACIS.CIDEVICEPARAMETER,ENQUESTACIS.CIDEVICESIZE,ENQUESTACIS.CIDEVICETYPE,ENQUESTACIS.CIDISCOUNTCODE,ENQUESTACIS.CIEMAIL,ENQUESTACIS.CIITEMCODE,ENQUESTACIS.CIMASTERREPORTCODE,ENQUESTACIS.CINAMEPREFIX,ENQUESTACIS.CINEIGHBORHOODCODE,ENQUESTACIS.CIPHONE,ENQUESTACIS.CIPOLICEDISTRICT,ENQUESTACIS.CIPROPERTYCLASS,ENQUESTACIS.CIREPORTCODEDESC,ENQUESTACIS.CISECONDARYPARTY,ENQUESTACIS.CISERVICETYPE,ENQUESTACIS.CISTATE,ENQUESTACIS.CISUBDIVISION,ENQUESTACIS.CITAXCODE,ENQUESTACIS.CITAXTYPE,ENQUESTACIS.CITOWN,CIS.FCIADDR,CIS.FCIAPREM,CIS.FCIBAL,CIS.FCICIS,CIS.FCICMTR,CIS.FCIDEVICE,CIS.FCIINSTAL,CIS.FCIITEM,CIS.FCIPREM,CIS.FCIREGISTER,CIS.FCISERVI,CIS.FDGCMAS,CIS.FRFCMAS,CIS.FRTCMAS,CIS.FSWCMAS,CIS.FTXCMAS,CIS.FUTCMAS,CIS.FWACMAS
database.history.kafka.bootstrap.servers=redacted:9092
database.history.kafka.topic=schema-changes-cdc.inventory
schema.history.internal.kafka.topic=schema-history-cdc
schema.history.internal.kafka.bootstrap.servers=redacted:9092
topic.prefix=sds_
database.history.store.only.captured.tables.ddl=true
internal.log.mining.query.logs.for.snapshot.offset=true
signal.data.collection=SDSENQ.DBZUSER.DEBEZIUM_SIGNAL
value.converter.schemas.enable=false
key.converter.schemas.enable=false
log.mining.archive.destination.name=log_archive_dest_2
log.mining.strategy=online_catalog
log.mining.query.filter.mode=in
log.mining.buffer.type=infinispan_embedded
log.mining.buffer.infinispan.cache.transactions=<local-cache name="transactions"><persistence passivation="false"><file-store read-only="false" preload="true" shared="false"><data path="/data02/cdc/cache/data"/><index path="/data02/cdc/cache/index"/></file-store></persistence></local-cache>
What is the captured database version and mode of depoyment?
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.20.0.0.0
Oracle Cloud Infrastructure VM
What behaviour do you expect?
After experiencing an OutOfMemory exception after a large transaction and researching fixes, I found ccranfor@redhat.com 's article on Debezium Oracle Connector performance where the option of using off-heap event buffering is described. I would like to utilize the Embedded Infinispan option, but although the Infinispan XML settings for log.mining.buffer.infinispan.cache.transactions are described in the documentation (and seem to parse ok in the config properties file), examples of the XML configuration values for the other required properties
- log.mining.buffer.infinispan.cache.events,
- log.mining.buffer.infinispan.cache.processed_transactions,
- log.mining.buffer.infinispan.cache.schema_changes
are not provided and cause an exception on startup.
java.util.concurrent.ExecutionException: org.apache.kafka.connect.runtime.rest.errors.BadRequestException: Connector configuration is invalid and contains the following 3 error(s): The 'log.mining.buffer.infinispan.cache.events' value is invalid: A value is required The 'log.mining.buffer.infinispan.cache.processed_transactions' value is invalid: A value is required The 'log.mining.buffer.infinispan.cache.schema_changes' value is invalid: A value is required
What behaviour do you see?
The exception above
Do you see the same behaviour using the latest relesead Debezium version?
Yes
Do you have the connector logs, ideally from start till finish?
I can provide them if necessary
How to reproduce the issue using our tutorial deployment?
Replicate the provided configuration in a test environment
Feature request or enhancement
For feature requests or enhancements, provide this information, please:
Which use case/requirement will be addressed by the proposed feature?
Improve/Expand the documentation describing off-heap event buffering