Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-7059

MariaDB/ MySQL Debezium x RabbitMQ ERROR "[15:53] Martin Visser "Error configuring an instance of KafkaSchemaHistory""

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Major Major
    • None
    • 2.4.0.Final
    • debezium-server
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Bug report

      For bug reports, provide this information, please:

      What Debezium connector do you use and what version?

      MariaDB and MySQL 

      What is the connector configuration?

       

      
      debezium.sink.type=rabbitmq
      debezium.sink.rabbitmq.connection.host=rabbitmq
      debezium.sink.rabbitmq.connection.port=5672
      debezium.sink.rabbitmq.connection.username=guest
      debezium.sink.rabbitmq.connection.password=guest
      debezium.sink.rabbitmq.connection.virtual.host=/
      debezium.source.connector.class=io.debezium.connector.mysql.MySqlConnector
      debezium.source.offset.storage.file.filename=data/offsets.dat
      debezium.source.offset.flush.interval.ms=0
      debezium.source.database.hostname=mysql
      debezium.source.database.port=3306
      debezium.source.database.user=root
      debezium.source.database.password=mysql
      debezium.source.database.server.id=184054
      debezium.source.database.dbname=test
      debezium.source.database.allowPublicKeyRetrieval=true
      debezium.source.topic.prefix=cdc
      debezium.source.schema.whitelist=test
      debezium.source.database.history=io.debezium.relational.history.MemoryDatabaseHistory 

       

       

      What is the captured database version and mode of depoyment?

      MySQL and MariaDB on docker container

      What behaviour do you expect?

      CDC to work fine (as with PostgreSQL for example)

      What behaviour do you see?

       

      {"timestamp":"2023-10-20T13:51:01.201Z","sequence":128,"loggerClassName":"org.slf4j.impl.Slf4jLogger","loggerName":"io.debezium.server.ConnectorLifecycle","level":"ERROR","message":"Connector completed: success = 'false', message = 'Unable to initialize and start connector's task class 'io.debezium.connector.mysql.MySqlConnectorTask' with config: {connector.class=io.debezium.connector.mysql.MySqlConnector, offset.storage.rabbitmq.connection.username=guest, offset.storage.rabbitmq.connection.port=5672, schema.history.internal.rabbitmq.connection.port=5672, offset.storage.rabbitmq.connection.virtual.host=/, topic.prefix=cdc, offset.storage.file.filename=data/offsets.dat, errors.retry.delay.initial.ms=300, value.converter=org.apache.kafka.connect.json.JsonConverter, key.converter=org.apache.kafka.connect.json.JsonConverter, schema.history.internal.rabbitmq.connection.password=********, database.allowPublicKeyRetrieval=true, schema.history.internal.rabbitmq.connection.virtual.host=/, database.user=root, database.dbname=test, database.server.id=184054, offset.storage=org.apache.kafka.connect.storage.FileOffsetBackingStore, offset.storage.rabbitmq.connection.password=********, offset.flush.timeout.ms=5000, errors.retry.delay.max.ms=10000, database.port=3306, schema.whitelist=test, schema.history.internal.rabbitmq.connection.host=rabbitmq, offset.flush.interval.ms=0, offset.storage.rabbitmq.connection.host=rabbitmq, errors.max.retries=-1, database.hostname=mysql, database.password=********, name=rabbitmq, schema.history.internal.rabbitmq.connection.username=guest, database.history=io.debezium.relational.history.MemoryDatabaseHistory}', error = 'org.apache.kafka.connect.errors.ConnectException: Error configuring an instance of KafkaSchemaHistory; check the logs for details'","threadName":"pool-7-thread-1","threadId":25,"mdc":{},"ndc":"","hostName":"b8b8f3e08c96","processName":"io.debezium.server.Main","processId":1,"exception":{"refId":1,"exceptionType":"org.apache.kafka.connect.errors.ConnectException","message":"Error configuring an instance of KafkaSchemaHistory; check the logs for details","frames":[{"class<span class="code-quote">":"io.debezium.storage.kafka.history.KafkaSchemaHistory","method":"configure","line":208},{"class<span class="code-quote">":"io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig","method":"getSchemaHistory","line":137},{"class<span class="code-quote">":"io.debezium.relational.HistorizedRelationalDatabaseSchema","method":"<init>","line":49},{"class<span class="code-quote">":"io.debezium.connector.mysql.MySqlDatabaseSchema","method":"<init>","line":82},{"class<span class="code-quote">":"io.debezium.connector.mysql.MySqlConnectorTask","method":"start","line":98},{"class<span class="code-quote">":"io.debezium.connector.common.BaseSourceTask","method":"start","line":142},{"class<span class="code-quote">":"io.debezium.embedded.EmbeddedEngine","method":"startSourceTask","line":918},{"class<span class="code-quote">":"io.debezium.embedded.EmbeddedEngine","method":"run","line":737},{"class<span class="code-quote">":"io.debezium.embedded.ConvertingEngineBuilder$2","method":"run","line":229},{"class<span class="code-quote">":"io.debezium.server.DebeziumServer","method":"lambda$start$1","line":170},{"class<span class="code-quote">":"java.util.concurrent.ThreadPoolExecutor","method":"runWorker","line":1128},{"class<span class="code-quote">":"java.util.concurrent.ThreadPoolExecutor$Worker","method":"run","line":628},{"class<span class="code-quote">":"java.lang.Thread","method":"run","line":829}]}} 

       

       

      Do you see the same behaviour using the latest relesead Debezium version?

      Yes

      How to reproduce the issue using our tutorial deployment?

      docker-compose.yml

       

       

      version: "3.9"
      services:
        mysql:
          image: mysql:latest
          restart: always
          container_name: mysql
          ports:
            - 3306:3306
          environment:
            - MYSQL_USER=mysql
            - MYSQL_PASSWORD=mysql
            - MYSQL_ROOT_PASSWORD=mysql
          healthcheck:
            test: mysqladmin ping -h localhost -P 3306 -u root --password=mysql
            interval: 2s
            timeout: 20s
            retries: 10
        mysqlsetup:
          image: mysql:latest
          container_name: mysqlsetup
          depends_on:
            mysql:
              condition: service_healthy
          restart: "no"
          environment:
            BINLOG_FORMAT: ROW
          entrypoint: [ "bash", "-c" , "sleep 20 && mysql --host mysql --port=3306 --user=root --password=mysql -e 'SET GLOBAL binlog_format=ROW;'; mysql --host=mysql --port=3306 --user=root --password=mysql -e 'create database test;'; mysql --host=mysql --port=3306 --user=root --password=mysql --database=test -e 'create table customers(id int, name varchar(100))'"]
        rabbitmq:
          image: rabbitmq:latest
          container_name: rabbitmq
          ports:
            - 15672:15672
          healthcheck:
            test: rabbitmq-diagnostics -q ping
            interval: 30s
            timeout: 30s
            retries: 3
        debezium-server:
          image: quay.io/debezium/server:2.4
          container_name: debezium-server
          ports:
            - 8080:8080
          volumes:
            - ./conf:/debezium/conf
          depends_on:
            mysql:
              condition: service_healthy
            rabbitmq:
              condition: service_healthy
       
      

       

       

              Unassigned Unassigned
              ahmed-34 Ahmed rachid (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: