Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
1.9.2.Final
-
None
-
False
-
None
-
False
Description
During the MySQL connector creation, there might be an error in the log (console), and, if error has occurred, it repeatedly shown for up to 2-3 minutes:
2022-05-17 13:15:09,061 ERROR || Error forwarding REST request [org.apache.kafka.connect.runtime.rest.RestClient] org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Internal request missing required signature at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:133) at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:65)
In most cases, a connector will be created after that (see attached log), but sometimes the error “Internal Server Error” might occur. In such case the connector won’t be created.
The debezium-connector’s docker image contains the Groovy scripts. The Dockerfile content is:
FROM debezium/connect:1.9.2.Final COPY ./groovy/lib/groovy-3.0.9.jar /kafka/connect/debezium-connector-mysql/groovy-3.0.9.jar COPY ./groovy/lib/groovy-json-3.0.9.jar /kafka/connect/debezium-connector-mysql/groovy-json-3.0.9.jar COPY ./groovy/lib/groovy-jsr223-3.0.9.jar /kafka/connect/debezium-connector-mysql/groovy-jsr223-3.0.9.jar
Debezium service does work in the Docker Swarm, which consists from 2 nodes. Here is debezium service configuration, as it’s defined in compose.yml:
debezium-connector:
image: debezium-connector-docker:latest
networks:
- confluent_network
ports:
- target: 8083
protocol: tcp
mode: ingress
depends_on:
- confluent-kafka1
- confluent-kafka2
environment:
- BOOTSTRAP_SERVERS=confluent-kafka1:49092,confluent-kafka2:59092
- GROUP_ID=debezium-connector
- ADVERTISED_HOST_NAME=debezium-connector
- ADVERTISED_PORT=8083
- REST_HOST_NAME=0.0.0.0
- CONFIG_STORAGE_TOPIC=connect_configs
- OFFSET_STORAGE_TOPIC=connect_offsets
- STATUS_STORAGE_TOPIC=connect_statuses
- ENABLE_DEBEZIUM_SCRIPTING=true
deploy:
replicas: 2
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
Connector's config JSON:
{ "name": "TUN_TUN1_SB01_1652793308_123", "config": { "connector.class": "io.debezium.connector.mysql.MySqlConnector", "snapshot.locking.mode": "none", "tasks.max": "1", "database.history.kafka.topic": "schema-changes.TUN_TUN1_SB01_1652793308_123", "transforms": "filter,Reroute", "transforms.Reroute.topic.replacement": "TUN_TUN1_SB01_1652793308_123", "value.converter": "org.apache.kafka.connect.json.JsonConverter", "key.converter": "org.apache.kafka.connect.json.JsonConverter", "database.user": ****, "transforms.Reroute.type": "io.debezium.transforms.ByLogicalTableRouter", "database.server.id": "123", "database.history.kafka.bootstrap.servers": "confluent-kafka1:9092,confluent-kafka2:59092", "database.server.name": "TUN_TUN1_SB01_1652793308_123", "internal.database.history.ddl.filter": "# Dummy.*", "transforms.filter.type": "io.debezium.transforms.Filter", "transforms.filter.language": "jsr223.groovy", "transforms.Reroute.topic.regex": "(.*)", "database.port": ****, "key.converter.schemas.enable": "false", "transforms.filter.condition": "value.schema().field('op')!=null && (value.op == 'r' || value.op == 'c') && value.schema().field('after')!=null && value.after.schema().field('Date_Time')!=null && value.after.Date_Time > 1652709074423", "database.hostname": ***********, "database.connectionTimeZone": "Europe/Berlin", "database.password": ****, "value.converter.schemas.enable": "false", "table.include.list": "TUN.TUN1_SB01", "database.include.list": "TUN" } }