Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3019

Complete support for properties that contain hyphens

    XMLWordPrintable

    Details

    • Steps to Reproduce:
      Hide

      Run Connect docker image via docker-compose with a hyphen-containing variable such as:

      CONNECT_KEY_CONVERTER_APICURIO_REGISTRY_GLOBAL-ID:                io.apicurio.registry.utils.serde.strategy.CachedSchemaIdStrategy
      

       Check file:

      /kafka/config/connect-distributed.properties
      

      And see that the property HAS been added but the value is empty.

      Note also the lack-luster Apicurio logging that occurs, preventing a typical user from discovering what is going on under the hood.

      Now empathize with how infuriating it is to have this happen and spend a day trying 100 different configuration permutations to try to work out what you have done wrong.

      Finally, fix the problem and experience the near-orgasmic joy and relief of having finally conquered this beast...

      Hope this helps. Keep up the good work. Debezium is amazing.

      Show
      Run Connect docker image via docker-compose with a hyphen-containing variable such as: CONNECT_KEY_CONVERTER_APICURIO_REGISTRY_GLOBAL- ID: io.apicurio.registry.utils.serde.strategy.CachedSchemaIdStrategy  Check file: /kafka/config/connect-distributed.properties And see that the property HAS been added but the value is empty. Note also the lack-luster Apicurio logging that occurs, preventing a typical user from discovering what is going on under the hood. Now empathize with how infuriating it is to have this happen and spend a day trying 100 different configuration permutations to try to work out what you have done wrong. Finally, fix the problem and experience the near-orgasmic joy and relief of having finally conquered this beast... Hope this helps. Keep up the good work. Debezium is amazing.

      Description

      Linux has patchy support for hyphen-containing ENV vars. Docker and most of this image does support them.
      Unfortunately in one place it does not and this prevents configuring the image correctly for Apicurio correctly.
      e.g.

       

      CONNECT_KEY_CONVERTER_APICURIO_REGISTRY_GLOBAL-ID: io.apicurio.registry.utils.serde.strategy.CachedSchemaIdStrategy
       
      

      The script in question within the connect-base image: /kafka/bin/docker-entrypoint.sh

      It contains an indirect variable reference using a ENV var as the lookup. This works fine until there is a hyphen in the ENV var name in which case it very helpfully sets the correct variable to 'empty' without informing you.

      An example of the offending in the script:

       

      prop_name=`echo "$VAR" | sed -r "s/^CONNECT_(.*)=.*/\1/g" | tr '[:upper:]' '[:lower:]' | tr _ .`
      
      [...]
      
      echo "$prop_name=${!env_var}" >> $KAFKA_HOME/config/connect-distributed.properties
      

       

       

      The fix I have used for my situation is below.

      It may work in all cases unless there is a requirement I am not aware of that requires that indirect reference:

       

      prop_name=`echo "$VAR" | sed -r "s/^CONNECT_(.*)=.*/\1/g" | tr '[:upper:]' '[:lower:]' | tr _ .`
      prop_value=`echo "$VAR" | sed -r "s/^CONNECT_.*=(.*)/\1/g"`
      
      [...]
      echo "$prop_name=${prop_value}" >> $KAFKA_HOME/config/connect-distributed.properties
      

       

       

       

       

       

       

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            mrshanepaul Shane Paul (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: