Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-223

Postgres connector crash on a database managed by Django

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 0.5.1
    • 0.5
    • postgresql-connector
    • None
    • Hide

      1. run migrations in django (manage.py migrate)
      2. start zookeeper:

      docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:0.5

      3. Start kafka:

      docker run -it --rm --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka:0.5

      4. start Kafka Connect:

      docker run -it --rm --name connect -p 8083:8083\
      -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs\
      -e OFFSET_STORAGE_TOPIC=my_connect_offsets --link zookeeper:zookeeper --link kafka:kafka\
      debezium/connect:0.5

      5. Stop postgres (brew services stop postgresql), modify postgresql.conf and pg_hba.conf as in the docs. Start postgres. Run migrations. I simply got the snapshot of /usr/local/var/postgres dir and attach here. You can stop postgres, remove /usr/local/var/postgres, untar at /usr/local/var/postgres, and restart postgres. DB user/password is naci/naci.

      6. Add connector:

      curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d @pg-test-connect-props.json

      7. Insert a record to accounts_user table:

      INSERT INTO "accounts_user" ("password", "last_login", "is_superuser", "stripe_customer_id", "stripe_subscription_data", "subscription_status", "cancel_at_period_end", "cancel_date", "uuid", "created", "modified", "email", "first_name", "last_name", "image", "is_staff", "is_active", "date_joined", "facebook_access_token", "timezone", "phone", "zip_code", "allow_phi_in_emails", "last_opened", "google_auth_flow", "google_api_credentials") VALUES ('', NULL, false, NULL, NULL, 'canceled', false, NULL, '46b82ef3-52c7-4113-8a60-4dbd9f7cbca1'::uuid, '2017-04-17T19:43:54.722483+00:00'::timestamptz, '2017-04-17T19:43:54.722501+00:00'::timestamptz, 'naci20@test.com', 'naci', 'ishak', '', false, true, '2017-04-17T19:43:54.708325+00:00'::timestamptz, NULL, 'America/Los_Angeles', NULL, NULL, false, NULL, NULL, NULL) RETURNING "id";

      Show
      1. run migrations in django (manage.py migrate) 2. start zookeeper: docker run -it --rm --name zookeeper -p 2181:2181 -p 2888:2888 -p 3888:3888 debezium/zookeeper:0.5 3. Start kafka: docker run -it --rm --name kafka -p 9092:9092 --link zookeeper:zookeeper debezium/kafka:0.5 4. start Kafka Connect: docker run -it --rm --name connect -p 8083:8083\ -e GROUP_ID=1 -e CONFIG_STORAGE_TOPIC=my_connect_configs\ -e OFFSET_STORAGE_TOPIC=my_connect_offsets --link zookeeper:zookeeper --link kafka:kafka\ debezium/connect:0.5 5. Stop postgres (brew services stop postgresql), modify postgresql.conf and pg_hba.conf as in the docs. Start postgres. Run migrations. I simply got the snapshot of /usr/local/var/postgres dir and attach here. You can stop postgres, remove /usr/local/var/postgres, untar at /usr/local/var/postgres, and restart postgres. DB user/password is naci/naci. 6. Add connector: curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" localhost:8083/connectors/ -d @pg-test-connect-props.json 7. Insert a record to accounts_user table: INSERT INTO "accounts_user" ("password", "last_login", "is_superuser", "stripe_customer_id", "stripe_subscription_data", "subscription_status", "cancel_at_period_end", "cancel_date", "uuid", "created", "modified", "email", "first_name", "last_name", "image", "is_staff", "is_active", "date_joined", "facebook_access_token", "timezone", "phone", "zip_code", "allow_phi_in_emails", "last_opened", "google_auth_flow", "google_api_credentials") VALUES ('', NULL, false, NULL, NULL, 'canceled', false, NULL, '46b82ef3-52c7-4113-8a60-4dbd9f7cbca1'::uuid, '2017-04-17T19:43:54.722483+00:00'::timestamptz, '2017-04-17T19:43:54.722501+00:00'::timestamptz, 'naci20@test.com', 'naci', 'ishak', '', false, true, '2017-04-17T19:43:54.708325+00:00'::timestamptz, NULL, 'America/Los_Angeles', NULL, NULL, false, NULL, NULL, NULL) RETURNING "id";

      When run on a database managed by Django 1.8.6, after the connector is created with curl POST, an insertion to a table causes connector to crash. Message seen on connector log:

      WARN Postgres|dbsvname|records-stream-producer Closing replication stream due to db connection IO exception... [io.debezium.connector.postgresql.RecordsStreamProducer]

      Message seen on postgresql log:

      LOG: starting logical decoding for slot "debezium"
      DETAIL: streaming transactions committing after 0/2978F88, reading WAL from 0/2978F50
      LOG: logical decoding found consistent point at 0/2978F50
      DETAIL: There are no running transactions.
      LOG: server process (PID 66012) was terminated by signal 11: Segmentation fault
      LOG: terminating any other active server processes
      WARNING: terminating connection because of crash of another server process
      DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
      HINT: In a moment you should be able to reconnect to the database and repeat your command.
      WARNING: terminating connection because of crash of another server process
      DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.
      HINT: In a moment you should be able to reconnect to the database and repeat your command.

      Environment:
      OS: Mac OS 10.12.3
      zookeeper: debezium/zookeeper:0.5 docker image
      kafka: debezium/kafka:0.5 docker image
      connect: debezium/connect:0.5 docker image
      postgres: ver 9.6 installed via homebrew
      django: ver 1.8.6

        1. pg-test-connect-props.json
          0.6 kB
        2. postgres.debug1.log.gz
          2 kB
        3. postgres.debug3.log.gz
          46 kB
        4. usr_local_var_postgres.tar.gz
          10.08 MB

            Unassigned Unassigned
            naci_jira Naci Ishakbeyoglu (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: