-
Bug
-
Resolution: Not a Bug
-
Blocker
-
None
-
3.2.0.Alpha1
-
None
-
False
-
-
False
-
Important
In order to make your issue reports as actionable as possible, please provide the following information, depending on the issue type.
Bug report
We are running Debezium server on our Kubernetes cluster. We would like to use Redis for offsets.
We set this value in the config:
offset.storage: io.debezium.storage.redis.offset.RedisOffsetBackingStore
We also provide the nessesary redis configuration
I have checked the redis cluster and no keys are being created..
So I explored further and I noticed no matter what value I set inside the configmap for debezium "application.properties" we see the following:
debezium.source.offset.storage=org.apache.kafka.connect.storage.MemoryOffsetBackingStore
What Debezium connector do you use and what version?
We are using: /debezium/server:3.2.0.Alpha1 (as we need the predicates fix!)
What is the connector configuration?
Here is the full debezium server spec
apiVersion: debezium.io/v1alpha1
kind: DebeziumServer
metadata:
name: {{ .Release.Name }}
spec:
runtime:
templates:
pod:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: nodetype
operator: In
values:
- worker
metrics:
jmxExporter:
enabled: true
image: dotdigital.azurecr.io/debezium/server:3.2.0.Alpha1
quarkus:
config:
log.console.json: true
kubernetes-config.enabled: true
kubernetes-config.secrets: debezium-secrets
source:
class: "io.debezium.connector.sqlserver.SqlServerConnector"
config:
tasks.max:
offset.storage: io.debezium.storage.redis.offset.RedisOffsetBackingStore
offset.storage.redis.address: {{ .Values.debezium_server.redis.address }}
offset.storage.redis.auth.password: ${REDIS_PASSWORD}
offset.storage.redis.ssl.enabled: true
offset.storage.redis.wait.retry.enabled: true
offset.storage.redis.wait.timeout.ms: 1000
offset.storage.redis.key: {{ .Values.debezium_server.redis.key }}
offset.flush.interval.ms: 1000
database.trustServerCertificate: true
database.hostname: {{ .Values.debezium_server.database.hostname }}
database.port: {{ .Values.debezium_server.database.port }}
{{- if and (ne .Values.environment "production") (ne .Values.environment "staging") }}
database.instance: {{ .Values.debezium_server.database.instance }}
{{- end }}
database.user: {{ .Values.debezium_server.database.user }}
database.password: ${DB_PASSWORD}
database.names: {{ .Values.debezium_server.database.names }}
table.include.list: {{ .Values.debezium_server.database.table.list }}
topic.prefix: redacted
snapshot.mode: no_data
snapshot.locking.mode: none
snapshot.isolation.mode: read_uncommitted
schema.history.internal.store.only.captured.tables.ddl: true
poll.interval.ms: 500
max.iteration.transactions: 500
max.batch.size: 2048
sink:
type: eventhubs
config:
connectionstring: ${EHUBS_CONNECTION_STRING}
hubname: redacted
format:
value:
type: json
config:
schemas.enable: false
predicates:
contacts:
type: org.apache.kafka.connect.transforms.predicates.TopicNameMatches
config:
pattern: .redacted
contactEmails:
type: org.apache.kafka.connect.transforms.predicates.TopicNameMatches
config:
pattern: redacted
transactionalDataDocuments:
type: org.apache.kafka.connect.transforms.predicates.TopicNameMatches
config:
pattern: redacted
transforms:
- config:
partition.payload.fields: redacted
partition.topic.num: 10
predicate: redacted
type: io.debezium.transforms.partitions.PartitionRouting
- config:
partition.payload.fields: redacted
partition.topic.num: 10
predicate: redacted
type: io.debezium.transforms.partitions.PartitionRouting
- config:
partition.payload.fields: redacted
partition.topic.num: 10
predicate: redacted
type: io.debezium.transforms.partitions.PartitionRouting
What behavior do you expect?
I expect to see debezium server configured to use redis for offsets
What behavior do you see?
Regardless of what configuration we provide we see the following configured:
debezium.source.offset.storage=org.apache.kafka.connect.storage.MemoryOffsetBackingStore
Do you see the same behaviour using the latest released Debezium version?
We are using: /debezium/server:3.2.0.Alpha1 (as we need the predicates fix!)
Do you have the connector logs, ideally from start till finish?
I can provide any but I can see in the config the value isn't being respected
How to reproduce the issue using our tutorial deployment?
Deploy debezium-server configured to use redis for offsets
Our project is blocked by this, we cannot look to use debezium in production if we do not use redis for offsets 
