Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1850

postgres oid is too large to cast to integer

    Details

      Description

      when starting debezium for a postgres database, i get the following log messages/errors :

      [2020-03-03 12:26:00,620] TRACE Type OID '-1661332183' not cached, attempting to lookup from database. (io.debezium.connector.postgresql.TypeRegistry:452)
      [2020-03-03 12:26:01,230] ERROR Failed to create job for /debezium/kafka/debezium.properties (org.apache.kafka.connect.cli.ConnectStandalone:108)
      [2020-03-03 12:26:01,230] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:119)
      java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Database connection failed during resolving unknown type
          at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:79)
          at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:66)
          at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:116)
      Caused by: org.apache.kafka.connect.errors.ConnectException: Database connection failed during resolving unknown type
          at io.debezium.connector.postgresql.TypeRegistry.resolveUnknownType(TypeRegistry.java:488)
          at io.debezium.connector.postgresql.TypeRegistry.get(TypeRegistry.java:169)
          at io.debezium.connector.postgresql.PostgresType$Builder.build(PostgresType.java:300)
          at io.debezium.connector.postgresql.TypeRegistry.prime(TypeRegistry.java:379)
          at io.debezium.connector.postgresql.TypeRegistry.<init>(TypeRegistry.java:122)
          at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:72)
          at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:102)
          at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:282)
          at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:188)
          at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:113)
      Caused by: org.postgresql.util.PSQLException: ERROR: OID out of range
      

      looking in the database, i see the following max oid :

      SELECT min(t.oid), max(t.oid) FROM pg_catalog.pg_type t JOIN pg_catalog.pg_namespace n ON (t.typnamespace = n.oid) WHERE n.nspname != 'pg_toast' AND t.typcategory <> 'A';
       min |    max     
      -----+------------
        16 | 2975143508
      

      I see the following in io.debezium.connector.postgresql.TypeRegistry (line 322/362) :

      final int oid = (int) rs.getLong("oid");
      

      i wrote a quick test :

      public class Test {
        public static void main(String[] args) {
          long env1_max = 2132162216L;
          long env2_max = 2975143508L;
          System.out.println("env1 long : " + env1_max);
          System.out.println("env2 long : " + env2_max);
          System.out.println("env1 int  : " + (int)env1_max);
          System.out.println("env2 int  : " + (int)env2_max);
        }
      }
      

      and the output was :

      env1 long : 2132162216
      env2 long : 2975143508
      env1 int  : 2132162216
      env2 int  : -1319823788
      

      some of our oid's are too large to be casted to an int

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                dajerome David Jerome
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: