Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-1913

Multiline stack traces can be collapsed into a single log event

    Details

    • Type: Feature Request
    • Status: Closed (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 1.1.0.Final
    • Fix Version/s: 1.2.0.Beta1
    • Component/s: core-library
    • Labels:
      None

      Description

      Context

      • Debezium uses Log4j for logging and currently logs stacktraces over multiple lines.
      • As a result some infrastructure components for monitoring can not link the different lines of the stacktrace together and interpret every line as a new log event.
      • Therefore, monitoring alerts filters often miss required context to determine whether an alert should be send.

      Example

      Debezium 1.1.0.Final introduced a retry-mechanism for lost Postgres connections but the following is still logged when a disconnect occurs

       
      2020-03-27 07:52:43,168 ERROR  Postgres|audit|postgres-connector-task  Producer failure   [io.debezium.pipeline.ErrorHandler]
      org.postgresql.util.PSQLException: Database connection failed when writing to copy
      	at org.postgresql.core.v3.QueryExecutorImpl.flushCopy(QueryExecutorImpl.java:1052)
      	at org.postgresql.core.v3.CopyDualImpl.flushCopy(CopyDualImpl.java:23)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.updateStatusInternal(V3PGReplicationStream.java:193)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.timeUpdateStatus(V3PGReplicationStream.java:184)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.readInternal(V3PGReplicationStream.java:126)
      	at org.postgresql.core.v3.replication.V3PGReplicationStream.readPending(V3PGReplicationStream.java:80)
      	at io.debezium.connector.postgresql.connection.PostgresReplicationConnection$1.readPending(PostgresReplicationConnection.java:402)
      	at io.debezium.connector.postgresql.PostgresStreamingChangeEventSource.execute(PostgresStreamingChangeEventSource.java:125)
      	at io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:101)
      	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      	at java.base/java.lang.Thread.run(Thread.java:834)
      Caused by: java.net.SocketException: Broken pipe (Write failed)
      	at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
      	at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
      	at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
      	at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:81)
      	at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.java:142)
      	at org.postgresql.core.PGStream.flush(PGStream.java:554)
      	at org.postgresql.core.v3.QueryExecutorImpl.flushCopy(QueryExecutorImpl.java:1050)
      	... 13 more
      

      Since Debezium will reconnect to Postgres, no alert should be send since Debezium automatically recovers from the issue. However, the SocketException could still be interpreted as an issue to alert on since it misses the context of it occurring for a Postgres reconnect.

      Requested change

      Note

        Gliffy Diagrams

          Attachments

            Activity

              People

              • Assignee:
                jpechanec Jiri Pechanec
                Reporter:
                wouter.bancken.aca Wouter Bancken
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: