Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7886

Oracle Debezium Connector cannot startup due to failing incremental snapshot

XMLWordPrintable

    • False
    • None
    • False
    • Critical

      Bug report

      For bug description see the following discussion in the Debezium Google group:
      https://groups.google.com/g/debezium/c/J5aaNgiwkwo

      What Debezium connector do you use and what version?

      Oracle Debezium Connector version 2.5.3.Final

      What is the connector configuration?

      {
          "snapshot.locking.mode": "none",
          "connector.class": "io.debezium.connector.oracle.OracleConnector",
          "message.key.columns": "EHDA.A_ARCREDITPROF:CUSTNOBILL,ORGUNITGRP,COMPNO; EHDA.A_COSHIPTO:CONO,COSHIPTOSEQNO; EHDA.A_CSCUBUSINESS:CUSTNO,BUUNIT,COMPNO; EHDA.A_CUSTPARTGRP:CUSTPARTGRP,PARTNO; EHDA.A_CZARSOLVHEAD:COMPNO,COMPID; EHDA.A_HICOHEAD:CONO; EHDA.A_HICOROW:CONO,ROWPOS,ROWSUBPOS,ROWSEQ; EHDA.A_ISINVSTATHOLD:PARTNO,STOCKNO,INVSTATCD,INVHOLDSEQNO; EHDA.A_LOCALPART:COMPNO,PARTNO; EHDA.A_PARTPRICE:PARTNO,PRICELIST,VALID_FROM; EHDA.A_POPARTVENDOR:COMPNO,PARTNO,VENDNO,FROMDATE; EHDA.A_POSHIPTO:PONO,POSHIPTOSEQNO; EHDA.A_POVECLASS5:VENDNO,CLASS5,COMPNO; EHDA.A_PRICELIST:PRICELIST; EHDA.A_PSBASE:PARTNO,STOCKNO; EHDA.A_PSEXTPART:EXTPARTNO,PSPRODCODEQUAL; EHDA.A_PSINVCTRL:PARTNO,STOCKNO; EHDA.A_PSPARTPACK:PARTNO,PARTPACK; EHDA.A_PSPARTSTOPDATES:PARTNO,STOPTYPE; EHDA.A_PSPRODRPLCMNT:STOCKNO,PARTNO,REPLCUIDTYPE,REPLCUID,SEQNO; EHDA.A_PSPRODSUBST:PARTNO,STOCKNO,SUBSTPARTNO; EHDA.A_PSRESERV:PARTNO,STOCKNO; EHDA.A_PSSTLOC:PARTNO,STOCKNO; EHDA.A_SYCOMPORGUNIT:COMPNO,ORGUNIT; EHDA.A_SYEVENTTRIGGER:ESSEVENT,ESSID,ESSIDQUAL,EVENTSEQNO; EHDA.A_WWSHOPPART:WWSHOPID,PARTNO,UNIT; EHDA.COAGREEHEAD:CUSTNO,AGREETYPE,COCLASS,BUUNIT,COMPNO,CURRENCY,COOBJ,CUSTDISC,SEQNO; EHDA.COAGREELINE:CUSTNO,AGREETYPE,COCLASS,BUUNIT,COMPNO,CURRENCY,COOBJ,CUSTDISC,SEQNO,LINENO; EHDA.COHEAD:CONO; EHDA.COROW:CONO,ROWPOS,ROWSUBPOS,ROWSEQ; EHDA.CUSTBILL:CUSTNOBILL; EHDA.CUSTBUY:CUSTNO; EHDA.CUSTSHIP:CUSTNO,SHIPTONO; EHDA.CZACCGROUPHEAD:COMPNO,CZCUSTNOGROUP,CUSTNO,SHIPTONO; EHDA.CZACCGROUPLINE:COMPNO,CZCUSTNOGROUP,CLASS1,CLASS2,CLASS3,CLASS4,CLASS5,CZCLASS8; EHDA.CZAGRPRDATE:CUSTNO,AGREETYPE,COCLASS,BUUNIT,COMPNO,CURRENCY,COOBJ,CUSTDISC,SEQNO,LINENO; EHDA.CZCLASS8GEN:COMPNO,PARTNO,VALID_FROM; EHDA.CZDISCTEMPLATE:CZDISCTEMPL,COMPNO; EHDA.CZDISCTEMPLATEROW:CZDISCTEMPL,COMPNO,CUSTDISC; EHDA.CZCUSTCOM:CZCUSTNOCOM; EHDA.CZCUSTCOMBREAK:CZCUSTNOCOM,CZCUSTCOMBR; EHDA.CZEDIORDERERRLOG:EDISEQNO,CODATE,CUSTNO,CONO,ROWPOS,PARTNO,CZEDIERRORTEXT,METHOFREC,CZEDIERRORNO; EHDA.CZHICLASS8:COMPNO,PARTNO,VALID_TO; EHDA.CZMULTIPLEAGR:CUSTNO,BUUNIT,COMPNO,CUSTDISC; EHDA.CZMULTIPLEKINGRP:CUSTNO,CZCUSTNOKIN,COMPNO; EHDA.CZPARTPROPERTYART:COMPNO,PARTNO,CZSUBPROPERTY2; EHDA.CZPRFACTCLASS8:COMPNO,CZCLASS8; EHDA.CZPRFACTGEN:COMPNO,VALID_FROM,PARTNO,CZCLASS8; EHDA.CZREGIONORGSTRUCT:COMPNO,REGION,DISTRICT,CUSTCLASS,BRANCH; EHDA.CZVENDCOM:CZVENDNOCOM; EHDA.CZVENDCOMBREAK:CZVENDNOCOM,CZVENDCOMBR; EHDA.PART:PARTNO; EHDA.PARTKIT:PARTNO,PARTPOS; EHDA.PARTSALESDESCR:PARTNO,LANGUAGE; EHDA.PARTSEARCH:PARTSEARCHNAME,PARTNO,CZDUBBLETTYN; EHDA.PERS:SY_USER; EHDA.POHEAD:PONO; EHDA.POROW:PONO,POROWPOS,POROWSEQ; EHDA.SFSYCODEVALUE:CODEID,CODEKEY,COMPNO; EHDA.SFSYMENUROW:MENU,ROWNO; EHDA.SFSYTEXTLANG:CMPTNAME,CMPTTYPE,CMPTID1,CMPTID2,LANGCODE; EHDA.STOCK:STOCKNO; EHDA.STOCKLOC:STOCKNO,PICKST,STOCKLOC; EHDA.STOCKPICK:STOCKNO,PICKST; EHDA.STOCKTAKE:STOCKNO,PARTNO,PICKST,STOCKLOC,STMISSNO,INDNO; EHDA.VENDBILL:VENDNO; EHDA.VENDBUY:VENDNO;",
          "topic.creation.default.allow.auto.create.topics": "true",
          "topic.creation.default.partitions": "6",
          "schema.history.internal.consumer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username='${file:/opt/kafka/external-configuration/dbz-ccloud-credentials/cred:username}' password='${file:/opt/kafka/external-configuration/dbz-ccloud-credentials/cred:password}';",
          "config.providers": "file",
          "signal.enabled.channels": "source",
          "topic.creation.default.auto.create.topics.enable": "true",
          "value.converter.basic.auth.credentials.source": "USER_INFO",
          "topic.creation.default.delete.retention.ms": "604800000",
          "schema.history.internal.producer.security.protocol": "SASL_SSL",
          "poll.interval.ms": "500",
          "signal.data.collection": "VIVP.CDC_DBZ.DBZ_SIGNAL",
          "topic.creation.default.replication.factor": "3",
          "config.providers.file.class": "org.apache.kafka.common.config.provider.FileConfigProvider",
          "key.converter": "io.confluent.connect.avro.AvroConverter",
          "schema.history.internal.producer.sasl.mechanism": "PLAIN",
          "schema.history.internal.producer.allow.auto.create.topics": "true",
          "database.dbname": "VIVP",
          "database.user": "${file:/opt/kafka/external-configuration/dbz-oracle-credentials/oracle:user}",
          "topic.creation.default.compression.type": "lz4",
          "schema.history.internal.kafka.bootstrap.servers": "pkc-lq8gm.westeurope.azure.confluent.cloud:9092",
          "key.converter.schemas.enable": "false",
          "value.converter.schema.registry.url": "https://psrc-qrk9d.westeurope.azure.confluent.cloud",
          "database.password": "${file:/opt/kafka/external-configuration/dbz-oracle-credentials/oracle:password}",
          "topic.creation.default.min.compaction.lag.ms": "604800000",
          "value.converter.schemas.enable": "false",
          "name": "vivaldi-cdc-connector",
          "schema.history.internal.consumer.sasl.mechanism": "PLAIN",
          "schema.history.internal.producer.sasl.jaas.config": "org.apache.kafka.common.security.plain.PlainLoginModule required username='${file:/opt/kafka/external-configuration/dbz-ccloud-credentials/cred:username}' password='${file:/opt/kafka/external-configuration/dbz-ccloud-credentials/cred:password}';",
          "key.converter.schema.registry.basic.auth.user.info": "${file:/opt/kafka/external-configuration/dbz-ccloud-sr-credentials/cred:username}:${file:/opt/kafka/external-configuration/dbz-ccloud-sr-credentials/cred:password}",
          "schema.history.internal.producer.auto.create.topics.enable": "true",
          "max.batch.size": "10000",
          "key.converter.basic.auth.credentials.source": "USER_INFO",
          "snapshot.mode": "initial",
          "schema.history.internal.consumer.security.protocol": "SASL_SSL",
          "schema.history.internal.consumer.allow.auto.create.topics": "true",
          "max.queue.size": "40000",
          "incremental.snapshot.chunk.size": "1024",
          "tasks.max": "1",
          "value.converter.schema.registry.basic.auth.user.info": "${file:/opt/kafka/external-configuration/dbz-ccloud-sr-credentials/cred:username}:${file:/opt/kafka/external-configuration/dbz-ccloud-sr-credentials/cred:password}",
          "log.mining.strategy": "online_catalog",
          "schema.history.internal.consumer.auto.create.topics.enable": "true",
          "schema.history.internal.store.only.captured.databases.ddl": "false",
          "schema.history.internal.store.only.captured.tables.ddl": "false",
          "value.converter.enhanced.avro.schema.support": "true",
          "topic.prefix": "dbzvivaldi",
          "decimal.handling.mode": "double",
          "schema.history.internal.kafka.topic": "dbzvivaldi.schema-changes.vivaldi-cdc-connector",
          "topic.creation.default.key.converter.schemas.enable": "false",
          "value.converter": "io.confluent.connect.avro.AvroConverter",
          "topic.creation.default.cleanup.policy": "compact",
          "topic.creation.default.max.compaction.lag.ms": "1209600000",
          "database.url": "<<REDACTED>>",
          "topic.creation.default.value.converter.schemas.enable": "false",
          "snapshot.max.threads": "67",
          "log.mining.query.filter.mode": "in",
          "table.include.list": "CDC_DBZ.DBZ_SIGNAL, EHDA.A_ARCREDITPROF, EHDA.A_COSHIPTO, EHDA.A_CSCUBUSINESS, EHDA.A_CUSTPARTGRP, EHDA.A_CZARSOLVHEAD, EHDA.A_HICOHEAD, EHDA.A_HICOROW, EHDA.A_ISINVSTATHOLD, EHDA.A_LOCALPART, EHDA.A_PARTPRICE, EHDA.A_POPARTVENDOR, EHDA.A_POSHIPTO, EHDA.A_POVECLASS5, EHDA.A_PRICELIST, EHDA.A_PSBASE, EHDA.A_PSEXTPART, EHDA.A_PSINVCTRL, EHDA.A_PSPARTPACK, EHDA.A_PSPARTSTOPDATES, EHDA.A_PSPRODRPLCMNT, EHDA.A_PSPRODSUBST, EHDA.A_PSRESERV, EHDA.A_PSSTLOC, EHDA.A_SYCOMPORGUNIT, EHDA.A_SYEVENTTRIGGER, EHDA.A_WWSHOPPART, EHDA.COAGREEHEAD, EHDA.COAGREELINE, EHDA.COHEAD, EHDA.COROW, EHDA.CUSTBILL, EHDA.CUSTBUY, EHDA.CUSTSHIP, EHDA.CZACCGROUPHEAD, EHDA.CZACCGROUPLINE, EHDA.CZAGRPRDATE, EHDA.CZCLASS8GEN, EHDA.CZDISCTEMPLATE, EHDA.CZDISCTEMPLATEROW, EHDA.CZCUSTCOM, EHDA.CZCUSTCOMBREAK, EHDA.CZEDIORDERERRLOG, EHDA.CZHICLASS8, EHDA.CZMULTIPLEAGR, EHDA.CZMULTIPLEKINGRP, EHDA.CZPARTPROPERTYART, EHDA.CZPRFACTCLASS8, EHDA.CZPRFACTGEN, EHDA.CZREGIONORGSTRUCT, EHDA.CZVENDCOM, EHDA.CZVENDCOMBREAK, EHDA.PART, EHDA.PARTKIT, EHDA.PARTSALESDESCR, EHDA.PARTSEARCH, EHDA.PERS, EHDA.POHEAD, EHDA.POROW, EHDA.SFSYCODEVALUE, EHDA.SFSYMENUROW, EHDA.SFSYTEXTLANG, EHDA.STOCK, EHDA.STOCKLOC, EHDA.STOCKPICK, EHDA.STOCKTAKE, EHDA.VENDBILL, EHDA.VENDBUY",
          "key.converter.schema.registry.url": "https://psrc-qrk9d.westeurope.azure.confluent.cloud"
      }
      

      What is the captured database version and mode of depoyment?

      On premises Oracle 19C

      What behaviour do you expect?

      I expect the connector to start up without any issues.

      What behaviour do you see?

      Connector crashes with ORA-17041

      Do you see the same behaviour using the latest relesead Debezium version?

      Do not know. Could not upgrade to 2.6.1.Final. Have submitted a separate ticket for that.

      Do you have the connector logs, ideally from start till finish?

       

            rh-ee-mvitale Mario Fiore Vitale
            per.dahlen@ahlsell.se Per Dahlén
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: