Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8175

Include Jackson libraries to JDBC connector Docker image distribution

      In the latest 2/3 versions, the connector fails (see discussion) with NoClassDefFoundError.

      The workaround is to mount those Jackson JARs as Docker volumes

            [DBZ-8175] Include Jackson libraries to JDBC connector Docker image distribution

            Released

            Debezium Builder added a comment - Released

            This issue is caused by Apicurio providing only a subset of Jackson libraries that are part of Kafka Connect runtime. Upon further investigation we decided this is not something that is solvable a the build time. The steps for correction

            1) Jackson libraries (and potential others as well) should be handled as a single logical unit. If a single one of them is ovewritten at connector/convertor level then all should be. We need to get in touch with Apicurio guys, explain the situation and provide a PR that will remedy the problem.
            2) In the meantime the Docker build will be modified so download the three missing JARs in the version that is required by Apicurio and put into Apicurio directory. This wasy we will simulate option 1.

            Jiri Pechanec added a comment - This issue is caused by Apicurio providing only a subset of Jackson libraries that are part of Kafka Connect runtime. Upon further investigation we decided this is not something that is solvable a the build time. The steps for correction 1) Jackson libraries (and potential others as well) should be handled as a single logical unit. If a single one of them is ovewritten at connector/convertor level then all should be. We need to get in touch with Apicurio guys, explain the situation and provide a PR that will remedy the problem. 2) In the meantime the Docker build will be modified so download the three missing JARs in the version that is required by Apicurio and put into Apicurio directory. This wasy we will simulate option 1.

            Roman Kudryashov added a comment - - edited

            jpechane 

            Yes, there is no error if I remove ENABLE_APICURIO_CONVERTERS: true env var from Docker Compose file

            Roman Kudryashov added a comment - - edited jpechane   Yes, there is no error if I remove ENABLE_APICURIO_CONVERTERS: true env var from Docker Compose file

            romankudryashov Could you please try it without apicurio? I am afraid this is the root cause.

            ccranfor@redhat.com I suppose Apicurio brings its own jackson/JAXB implementation and taht breaks classloading.

            Jiri Pechanec added a comment - romankudryashov Could you please try it without apicurio? I am afraid this is the root cause. ccranfor@redhat.com I suppose Apicurio brings its own jackson/JAXB implementation and taht breaks classloading.

            1) as far as I remember, I tried the latest version of one of them (maybe `jaxb-annotations`); it didn't work. I tried several lower versions of the library; after finding the working one, I used other JARs of the same version. Perhaps, `2.13.4` is not highest version that might work

             

            Roman Kudryashov added a comment - 1) as far as I remember, I tried the latest version of one of them (maybe `jaxb-annotations`); it didn't work. I tried several lower versions of the library; after finding the working one, I used other JARs of the same version. Perhaps, `2.13.4` is not highest version that might work  

            OK, I beleive there must be something deeper going on.
            1) Why did you chose the versions you are using?
            2) If you look into /libs directory the missing files in question are present in version 2.16.2. So this would imply that the files are missing but classloading in Hibernate or Kafka Connect is broken

            Jiri Pechanec added a comment - OK, I beleive there must be something deeper going on. 1) Why did you chose the versions you are using? 2) If you look into /libs directory the missing files in question are present in version 2.16.2 . So this would imply that the files are missing but classloading in Hibernate or Kafka Connect is broken

            jpechane WDYM by direct and indirect?

            The following is a list of Jackson dependencies (as Docker volumes) that I use locally to make JDBC connector from 3.0.0.Beta1 working:

            - ./kafka-connect/temp/jackson-module-afterburner-2.13.4.jar:/kafka/connect/debezium-connector-jdbc/jackson-module-afterburner-2.13.4.jar
            - ./kafka-connect/temp/jackson-module-jaxb-annotations-2.13.4.jar:/kafka/connect/debezium-connector-jdbc/jackson-module-jaxb-annotations-2.13.4.jar
            - ./kafka-connect/temp/jackson-module-scala_2.13-2.13.4.jar:/kafka/connect/debezium-connector-jdbc/jackson-module-scala_2.13-2.13.4.jar 

            I haven't found none of them in Git repos of Debezium or containers-images. So probably they are indirect. It is difficult to say why and when they were excluded

            Roman Kudryashov added a comment - jpechane WDYM by direct and indirect? The following is a list of Jackson dependencies (as Docker volumes) that I use locally to make JDBC connector from 3.0.0.Beta1 working: - ./kafka-connect/temp/jackson-module-afterburner-2.13.4.jar:/kafka/connect/debezium-connector-jdbc/jackson-module-afterburner-2.13.4.jar - ./kafka-connect/temp/jackson-module-jaxb-annotations-2.13.4.jar:/kafka/connect/debezium-connector-jdbc/jackson-module-jaxb-annotations-2.13.4.jar - ./kafka-connect/temp/jackson-module-scala_2.13-2.13.4.jar:/kafka/connect/debezium-connector-jdbc/jackson-module-scala_2.13-2.13.4.jar I haven't found none of them in Git repos of Debezium or containers-images. So probably they are indirect. It is difficult to say why and when they were excluded

            Are those libraries either direct or indirect dependencies? If they are why are their excluded?

            Depending on the anwer it could be decided if they should go into pom or into container image only.

            Jiri Pechanec added a comment - Are those libraries either direct or indirect dependencies? If they are why are their excluded? Depending on the anwer it could be decided if they should go into pom or into container image only.

              ccranfor@redhat.com Chris Cranford
              romankudryashov Roman Kudryashov
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: