Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-12208

Upgrading ACM causes search-postgres pod to be in CrashLoopBackOff state

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • ACM 2.11.5
    • ACM 2.11.0, ACM 2.9.4, ACM 2.12.0, ACM 2.10.3
    • Search
    • None
    • False
    • None
    • False
    • Search Sprint 2024-22
    • Moderate
    • No

      Description of problem:

      After ACM upgrade from 2.9.4 to 2.10.3 , search-postgres pod is in CrashLoopBackOff state.

      search-postgres pod logs:

      2024-06-07T08:36:41.936006626Z waiting for server to start....2024-06-07 08:36:42.003 UTC [24] LOG:  redirecting log output to logging collector process
2024-06-07T08:36:42.003080166Z 2024-06-07 08:36:42.003 UTC [24] HINT:  Future log output will appear in directory "log".
2024-06-07T08:36:42.338862342Z  stopped waiting
2024-06-07T08:36:42.338918868Z pg_ctl: could not start server
2024-06-07T08:36:42.338918868Z Examine the log output.

      Even in the past, issues with the search-postgres pod were observed which usually got resolved with just restarting the postgres-search pod. But this time, restart of the pod didn't help.

      Error (if applicable):

      2024-06-07 08:12:06.976 UTC [24] LOG:  starting PostgreSQL 13.14 on x86_64-redhat-linux-gnu, compiled by gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2), 64-bit
2024-06-07 08:12:06.976 UTC [24] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2024-06-07 08:12:06.980 UTC [24] LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
2024-06-07 08:12:06.990 UTC [26] LOG:  database system was interrupted while in recovery at 2024-06-07 08:11:12 UTC
2024-06-07 08:12:06.990 UTC [26] HINT:  This probably means that some data is corrupted and you will have to use the last backup for recovery.
2024-06-07 08:12:07.044 UTC [26] LOG:  database system was not properly shut down; automatic recovery in progress
2024-06-07 08:12:07.048 UTC [26] LOG:  redo starts at 9B/CC1905A8
2024-06-07 08:12:07.243 UTC [26] FATAL:  posting list tuple with 6 items cannot be split at offset 18
2024-06-07 08:12:07.243 UTC [26] CONTEXT:  WAL redo at 9B/CEB3ABF0 for Btree/INSERT_POST: off 54
2024-06-07 08:12:07.247 UTC [24] LOG:  startup process (PID 26) exited with exit code 1
2024-06-07 08:12:07.247 UTC [24] LOG:  aborting startup due to startup process failure
2024-06-07 08:12:07.258 UTC [24] LOG:  database system is shut down

        1. temp_files_query.txt
          2 kB

              jpadilla@redhat.com Jorge Padilla
              rhn-support-ribanerj Riya Banerjee
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: