Details
-
Bug
-
Resolution: Done
-
Critical
-
1.0.1.Final
-
None
Description
when starting debezium for a postgres database, i get the following log messages/errors :
[2020-03-03 12:26:00,620] TRACE Type OID '-1661332183' not cached, attempting to lookup from database. (io.debezium.connector.postgresql.TypeRegistry:452)
[2020-03-03 12:26:01,230] ERROR Failed to create job for /debezium/kafka/debezium.properties (org.apache.kafka.connect.cli.ConnectStandalone:108)
[2020-03-03 12:26:01,230] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:119)
java.util.concurrent.ExecutionException: org.apache.kafka.connect.errors.ConnectException: Database connection failed during resolving unknown type
at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:79)
at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:66)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:116)
Caused by: org.apache.kafka.connect.errors.ConnectException: Database connection failed during resolving unknown type
at io.debezium.connector.postgresql.TypeRegistry.resolveUnknownType(TypeRegistry.java:488)
at io.debezium.connector.postgresql.TypeRegistry.get(TypeRegistry.java:169)
at io.debezium.connector.postgresql.PostgresType$Builder.build(PostgresType.java:300)
at io.debezium.connector.postgresql.TypeRegistry.prime(TypeRegistry.java:379)
at io.debezium.connector.postgresql.TypeRegistry.<init>(TypeRegistry.java:122)
at io.debezium.connector.postgresql.connection.PostgresConnection.<init>(PostgresConnection.java:72)
at io.debezium.connector.postgresql.PostgresConnector.validate(PostgresConnector.java:102)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:282)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:188)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:113)
Caused by: org.postgresql.util.PSQLException: ERROR: OID out of range
looking in the database, i see the following max oid :
SELECT min(t.oid), max(t.oid) FROM pg_catalog.pg_type t JOIN pg_catalog.pg_namespace n ON (t.typnamespace = n.oid) WHERE n.nspname != 'pg_toast' AND t.typcategory <> 'A'; min | max -----+------------ 16 | 2975143508
I see the following in io.debezium.connector.postgresql.TypeRegistry (line 322/362) :
final int oid = (int) rs.getLong("oid");
i wrote a quick test :
public class Test { public static void main(String[] args) { long env1_max = 2132162216L; long env2_max = 2975143508L; System.out.println("env1 long : " + env1_max); System.out.println("env2 long : " + env2_max); System.out.println("env1 int : " + (int)env1_max); System.out.println("env2 int : " + (int)env2_max); } }
and the output was :
env1 long : 2132162216 env2 long : 2975143508 env1 int : 2132162216 env2 int : -1319823788
some of our oid's are too large to be casted to an int
