Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-4631

"No maximum LSN recorded" log message can be spammed on low-activity databases

XMLWordPrintable

    • False
    • False
    • Hide

      1. Deploy the pipeline using the SQL Server example from the tutorial:

      env DEBEZIUM_VERSION=1.8 docker compose -f docker-compose-sqlserver.yaml up

      2. Create the test database and insert data:

      cat debezium-sqlserver-init/inventory.sql | docker-compose -f docker-compose-sqlserver.yaml exec -T sqlserver bash -c '/opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD'

      3. Configure the Logger:

      curl -s -X PUT -H "Content-Type:application/json" \ 10http://localhost:8083/admin/loggers/io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource \ 
-d '{"level": "WARN"}' | jq '.' 

curl -s -X PUT -H "Content-Type:application/json" \ http://localhost:8083/admin/loggers/root \ 
-d '{"level": "WARN"}' | jq '.'

      4. Start a connector:

      curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http://localhost:8083/connectors/ -d @register-sqlserver.json

      5. Wait for the connector to start and complete snapshotting

      6. Simulate the clean up job deleting all entries from the cdc.lsn_time_mapping table with tran_id <> 0x00

      echo "DELETE FROM testDB.cdc.lsn_time_mapping WHERE tran_id <> 0x00;" | docker-compose -f docker-compose-sqlserver.yaml exec -T sqlserver bash -c '/opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD'

      7. Verify the log message is produced repeatedly like the following:

      connect_1 | 2022-01-27 17:36:54,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 2connect_1 | 2022-01-27 17:36:54,742 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 3connect_1 | 2022-01-27 17:36:55,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 4connect_1 | 2022-01-27 17:36:55,743 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 5connect_1 | 2022-01-27 17:36:56,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 6connect_1 | 2022-01-27 17:36:56,742 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 7connect_1 | 2022-01-27 17:36:57,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 8connect_1 | 2022-01-27 17:36:57,742 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 9connect_1 | 2022-01-27 17:36:58,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running
      Show
      1. Deploy the pipeline using the SQL Server example from the tutorial: env DEBEZIUM_VERSION=1.8 docker compose -f docker-compose-sqlserver.yaml up 2. Create the test database and insert data: cat debezium-sqlserver-init/inventory.sql | docker-compose -f docker-compose-sqlserver.yaml exec -T sqlserver bash -c '/opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD' 3. Configure the Logger: curl -s -X PUT -H "Content-Type:application/json" \ 10http: //localhost:8083/admin/loggers/io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource \ -d '{ "level" : "WARN" }' | jq '.' curl -s -X PUT -H "Content-Type:application/json" \ http: //localhost:8083/admin/loggers/root \ -d '{ "level" : "WARN" }' | jq '.' 4. Start a connector: curl -i -X POST -H "Accept:application/json" -H "Content-Type:application/json" http: //localhost:8083/connectors/ -d @register-sqlserver.json 5. Wait for the connector to start and complete snapshotting 6. Simulate the clean up job deleting all entries from the cdc.lsn_time_mapping table with tran_id <> 0x00 echo "DELETE FROM testDB.cdc.lsn_time_mapping WHERE tran_id <> 0x00;" | docker-compose -f docker-compose-sqlserver.yaml exec -T sqlserver bash -c '/opt/mssql-tools/bin/sqlcmd -U sa -P $SA_PASSWORD' 7. Verify the log message is produced repeatedly like the following: connect_1 | 2022-01-27 17:36:54,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 2connect_1 | 2022-01-27 17:36:54,742 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 3connect_1 | 2022-01-27 17:36:55,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 4connect_1 | 2022-01-27 17:36:55,743 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 5connect_1 | 2022-01-27 17:36:56,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 6connect_1 | 2022-01-27 17:36:56,742 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 7connect_1 | 2022-01-27 17:36:57,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 8connect_1 | 2022-01-27 17:36:57,742 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running [io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource] 9connect_1 | 2022-01-27 17:36:58,242 WARN SQL_Server|server1|streaming No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running

      On low activity databases, the cdc clean up job can delete all entries in the cdc.lsn_time_mapping table where the tran_id <> 0x00. This causes the SQLServerConnection#getNthTransactionFromLast method to return NULL and the resulting log message below to be spammed: 

      No maximum LSN recorded in the database; please ensure that the SQL Server Agent is running

            Unassigned Unassigned
            sugarcrm-jgminder Jacob Gminder (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: