-
Bug
-
Resolution: Unresolved
-
Blocker
-
None
-
None
-
False
-
None
-
False
-
Important
Due to the documentation of Debezium connected image https://hub.docker.com/r/debezium/connect message filtering is supported when ENABLE_DEBEZIUM_SCRIPTING env variable is set to True. When I tested filtering on two version of images 2.6 and 2.7.1 final, filter condition couldn't be resolved because of the lack of groovy. After installing groovy manually I got error of missing groovyjarjaropenbeans dependency. I belive that ENABLE_DEBEZIUM_SCRIPTING set to true should be sufficient condition to support Debezium scripting without manual installation of groovy.
Part of my configuration responsible for filtering:
"transforms": "filter", "transforms.filter.type": "io.debezium.transforms.Filter", "transforms.filter.language": "jsr223.groovy", "transforms.filter.condition": "value.op == 'u' && value.before.mywwfm_address == 'abc'",
Result of request curl -s {debezium host}/connectors/{config name}/status | jq
1. Without a groovy, plain image with mentioned flag set to true:
o.debezium.DebeziumException: Failed to parse expression 'value.get('op') == 'u' && value.get('before').get('mywwfm_address') == 'abc''\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:313)\n\tat org.apache.kafka.connect.runtime.Worker$SourceTaskBuilder.doBuild(Worker.java:1841)\n\tat org.apache.kafka.connect.runtime.Worker$TaskBuilder.build(Worker.java:1754)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)\n\tat org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:560)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:2001)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$38(DistributedHerder.java:2018)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: io.debezium.DebeziumException: Failed to parse expression 'value.get('op') == 'u' && value.get('before').get('mywwfm_address') == 'abc''\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:165)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:302)\n\t... 10 more\nCaused by: io.debezium.DebeziumException: Implementation of language 'groovy' not found on the classpath\n\tat io.debezium.transforms.scripting.Jsr223Engine.configure(Jsr223Engine.java:47)\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:162)\n\t... 11 more\n"
2. With groovy, jsr-223 set, and mentioned env:
"trace": "org.apache.kafka.connect.errors.ConnectException: io.debezium.DebeziumException: Failed to parse expression 'value.get.op == 'u' && value.get.before.get.mywwfm_address == 'abc''\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:313)\n\tat org.apache.kafka.connect.runtime.Worker$SourceTaskBuilder.doBuild(Worker.java:1841)\n\tat org.apache.kafka.connect.runtime.Worker$TaskBuilder.build(Worker.java:1754)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)\n\tat org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:560)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:2001)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$38(DistributedHerder.java:2018)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: io.debezium.DebeziumException: Failed to parse expression 'value.get.op == 'u' && value.get.before.get.mywwfm_address == 'abc''\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:165)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:302)\n\t... 10 more\nCaused by: io.debezium.DebeziumException: javax.script.ScriptException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:\nGeneral error during instruction selection: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\njava.lang.RuntimeException: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:972)\n\tat org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692)\n\tat org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666)\n\tat groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:373)\n\tat groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:316)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:298)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:258)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.getScriptClass(GroovyScriptEngineImpl.java:350)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:190)\n\tat io.debezium.transforms.scripting.Jsr223Engine.configure(Jsr223Engine.java:53)\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:162)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:302)\n\tat org.apache.kafka.connect.runtime.Worker$SourceTaskBuilder.doBuild(Worker.java:1841)\n\tat org.apache.kafka.connect.runtime.Worker$TaskBuilder.build(Worker.java:1754)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)\n\tat org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:560)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:2001)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$38(DistributedHerder.java:2018)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\tat org.codehaus.groovy.classgen.Verifier.<clinit>(Verifier.java:159)\n\tat org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:753)\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:938)\n\t... 24 more\nCaused by: java.lang.ClassNotFoundException: groovyjarjaropenbeans.Transient\n\tat java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)\n\tat org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:124)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)\n\t... 27 more\n\n1 error\n\n\tat io.debezium.transforms.scripting.Jsr223Engine.configure(Jsr223Engine.java:56)\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:162)\n\t... 11 more\nCaused by: javax.script.ScriptException: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:\nGeneral error during instruction selection: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\njava.lang.RuntimeException: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:972)\n\tat org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692)\n\tat org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666)\n\tat groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:373)\n\tat groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:316)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:298)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:258)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.getScriptClass(GroovyScriptEngineImpl.java:350)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:190)\n\tat io.debezium.transforms.scripting.Jsr223Engine.configure(Jsr223Engine.java:53)\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:162)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:302)\n\tat org.apache.kafka.connect.runtime.Worker$SourceTaskBuilder.doBuild(Worker.java:1841)\n\tat org.apache.kafka.connect.runtime.Worker$TaskBuilder.build(Worker.java:1754)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)\n\tat org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:560)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:2001)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$38(DistributedHerder.java:2018)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\tat org.codehaus.groovy.classgen.Verifier.<clinit>(Verifier.java:159)\n\tat org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:753)\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:938)\n\t... 24 more\nCaused by: java.lang.ClassNotFoundException: groovyjarjaropenbeans.Transient\n\tat java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)\n\tat org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:124)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)\n\t... 27 more\n\n1 error\n\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:192)\n\tat io.debezium.transforms.scripting.Jsr223Engine.configure(Jsr223Engine.java:53)\n\t... 12 more\nCaused by: org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:\nGeneral error during instruction selection: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\njava.lang.RuntimeException: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:972)\n\tat org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692)\n\tat org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666)\n\tat groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:373)\n\tat groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:316)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:298)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:258)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.getScriptClass(GroovyScriptEngineImpl.java:350)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:190)\n\tat io.debezium.transforms.scripting.Jsr223Engine.configure(Jsr223Engine.java:53)\n\tat io.debezium.transforms.ScriptingTransformation.configure(ScriptingTransformation.java:162)\n\tat org.apache.kafka.connect.runtime.ConnectorConfig.transformationStages(ConnectorConfig.java:302)\n\tat org.apache.kafka.connect.runtime.Worker$SourceTaskBuilder.doBuild(Worker.java:1841)\n\tat org.apache.kafka.connect.runtime.Worker$TaskBuilder.build(Worker.java:1754)\n\tat org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:664)\n\tat org.apache.kafka.connect.runtime.Worker.startSourceTask(Worker.java:560)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:2001)\n\tat org.apache.kafka.connect.runtime.distributed.DistributedHerder.lambda$getTaskStartingCallable$38(DistributedHerder.java:2018)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.base/java.lang.Thread.run(Thread.java:1583)\nCaused by: java.lang.NoClassDefFoundError: groovyjarjaropenbeans/Transient\n\tat org.codehaus.groovy.classgen.Verifier.<clinit>(Verifier.java:159)\n\tat org.codehaus.groovy.control.CompilationUnit$3.call(CompilationUnit.java:753)\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:938)\n\t... 24 more\nCaused by: java.lang.ClassNotFoundException: groovyjarjaropenbeans.Transient\n\tat java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)\n\tat org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:124)\n\tat java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)\n\t... 27 more\n\n1 error\n\n\tat org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:292)\n\tat org.codehaus.groovy.control.ErrorCollector.addException(ErrorCollector.java:140)\n\tat org.codehaus.groovy.control.CompilationUnit$IPrimaryClassNodeOperation.doPhaseOperation(CompilationUnit.java:972)\n\tat org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:692)\n\tat org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:666)\n\tat groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:373)\n\tat groovy.lang.GroovyClassLoader.lambda$parseClass$2(GroovyClassLoader.java:316)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.compute(StampedCommonCache.java:163)\n\tat org.codehaus.groovy.runtime.memoize.StampedCommonCache.getAndPut(StampedCommonCache.java:154)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:314)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:298)\n\tat groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:258)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.getScriptClass(GroovyScriptEngineImpl.java:350)\n\tat org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.compile(GroovyScriptEngineImpl.java:190)\n\t... 13 more\n"
Thanks in advance for any help.
