Details
-
Bug
-
Resolution: Not a Bug
-
Major
-
None
-
1.9.2.Final
-
None
-
False
-
None
-
False
Description
During the MySQL connector creation, there might be an error in the log (console), and, if error has occurred, it repeatedly shown for up to 2-3 minutes:
2022-05-17 13:15:09,061 ERROR || Error forwarding REST request [org.apache.kafka.connect.runtime.rest.RestClient] org.apache.kafka.connect.runtime.rest.errors.ConnectRestException: Internal request missing required signature at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:133) at org.apache.kafka.connect.runtime.rest.RestClient.httpRequest(RestClient.java:65)
In most cases, a connector will be created after that (see attached log), but sometimes the error “Internal Server Error” might occur. In such case the connector won’t be created.
The debezium-connector’s docker image contains the Groovy scripts. The Dockerfile content is:
FROM debezium/connect:1.9.2.Final COPY ./groovy/lib/groovy-3.0.9.jar /kafka/connect/debezium-connector-mysql/groovy-3.0.9.jar COPY ./groovy/lib/groovy-json-3.0.9.jar /kafka/connect/debezium-connector-mysql/groovy-json-3.0.9.jar COPY ./groovy/lib/groovy-jsr223-3.0.9.jar /kafka/connect/debezium-connector-mysql/groovy-jsr223-3.0.9.jar
Debezium service does work in the Docker Swarm, which consists from 2 nodes. Here is debezium service configuration, as it’s defined in compose.yml:
debezium-connector:
image: debezium-connector-docker:latest
networks:
- confluent_network
ports:
- target: 8083
protocol: tcp
mode: ingress
depends_on:
- confluent-kafka1
- confluent-kafka2
environment:
- BOOTSTRAP_SERVERS=confluent-kafka1:49092,confluent-kafka2:59092
- GROUP_ID=debezium-connector
- ADVERTISED_HOST_NAME=debezium-connector
- ADVERTISED_PORT=8083
- REST_HOST_NAME=0.0.0.0
- CONFIG_STORAGE_TOPIC=connect_configs
- OFFSET_STORAGE_TOPIC=connect_offsets
- STATUS_STORAGE_TOPIC=connect_statuses
- ENABLE_DEBEZIUM_SCRIPTING=true
deploy:
replicas: 2
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
Connector's config JSON:
{
"name": "TUN_TUN1_SB01_1652793308_123",
"config": {
"connector.class": "io.debezium.connector.mysql.MySqlConnector",
"snapshot.locking.mode": "none",
"tasks.max": "1",
"database.history.kafka.topic": "schema-changes.TUN_TUN1_SB01_1652793308_123",
"transforms": "filter,Reroute",
"transforms.Reroute.topic.replacement": "TUN_TUN1_SB01_1652793308_123",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"key.converter": "org.apache.kafka.connect.json.JsonConverter",
"database.user": ****,
"transforms.Reroute.type": "io.debezium.transforms.ByLogicalTableRouter",
"database.server.id": "123",
"database.history.kafka.bootstrap.servers": "confluent-kafka1:9092,confluent-kafka2:59092",
"database.server.name": "TUN_TUN1_SB01_1652793308_123",
"internal.database.history.ddl.filter": "# Dummy.*",
"transforms.filter.type": "io.debezium.transforms.Filter",
"transforms.filter.language": "jsr223.groovy",
"transforms.Reroute.topic.regex": "(.*)",
"database.port": ****,
"key.converter.schemas.enable": "false",
"transforms.filter.condition": "value.schema().field('op')!=null && (value.op == 'r' || value.op == 'c') && value.schema().field('after')!=null && value.after.schema().field('Date_Time')!=null && value.after.Date_Time > 1652709074423",
"database.hostname": ***********,
"database.connectionTimeZone": "Europe/Berlin",
"database.password": ****,
"value.converter.schemas.enable": "false",
"table.include.list": "TUN.TUN1_SB01",
"database.include.list": "TUN"
}
}
