Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2957

message.key.columns Regex Validation Time Complexity

XMLWordPrintable

    • False
    • False
    • Undefined
    • Hide

      To reproduce, create a connector with these tables/columns and add this configuration to the connector config. 

      "message.key.columns": "dbo.RbsUsers:integrationRecordId;dbo.RbsUserPrefs:integrationRecordId;dbo.RbsUserCompany:integrationRecordId;dbo.RbsCompanies:integrationRecordId;dbo.RbsCompanyValues:integrationRecordId;dbo.RbsHistory:integrationRecordId;dbo.RbsPassPols:integrationRecordId;dbo.SSOUsers:integrationRecordId;dbo.SSODirectLoginConfig:integrationRecordId;dbo.SSOCoDefaultsHeader:integrationRecordId;"

      The above will cause the connector to never start up and be stuck validating the above input with the validation regex. Removing the ";" at the end of the configuration value will lead to the connector starting properly.

       

      Show
      To reproduce, create a connector with these tables/columns and add this configuration to the connector config.  "message.key.columns" : "dbo.RbsUsers:integrationRecordId;dbo.RbsUserPrefs:integrationRecordId;dbo.RbsUserCompany:integrationRecordId;dbo.RbsCompanies:integrationRecordId;dbo.RbsCompanyValues:integrationRecordId;dbo.RbsHistory:integrationRecordId;dbo.RbsPassPols:integrationRecordId;dbo.SSOUsers:integrationRecordId;dbo.SSODirectLoginConfig:integrationRecordId;dbo.SSOCoDefaultsHeader:integrationRecordId;" The above will cause the connector to never start up and be stuck validating the above input with the validation regex. Removing the ";" at the end of the configuration value will lead to the connector starting properly.  

      The runtime for the `message.key.columns` validation regex blows up under certain conditions and can cause misleading failures. The validation needs to be adjusted such that the connector start up will fail rather than explode the runtime.

      Regex in question:

      private static final Pattern MSG_KEY_COLUMNS_PATTERN = Pattern.compile("^(([^:]+):([^:;\\s]+))+[^;]$");

      https://github.com/debezium/debezium/blob/master/debezium-core/src/main/java/io/debezium/relational/RelationalDatabaseConnectorConfig.java#L46

       

      In the case this regex never completes, the connector will never fully start up and will continue to chew on validating `message.key.columns` for an eternity. The connector will continuously spam this log making it seem like there was some other kind of error causing the connector to crash, but in fact the connector hasn't started up yet and it still validating.

      Couldn't commit processed log positions with the source database due to a concurrent connector shutdown or restart

      A way to make the regex's runtime blow up is to create a list of tables/columns and to add an extra semi-colon at the end. 

              anmohant Anisha Mohanty
              jgormley6 James Gormley (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: