Uploaded image for project: 'Product Technical Learning'
  1. Product Technical Learning
  2. PTL-15803

DO280-4.18: updates-olm: Timeout while installing operator

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • DO280 - O4.x NEXT
    • DO280
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • en-US (English)

      XREF: https://redhat-internal.slack.com/archives/C06E3HWF9K4/p1764604966052829

      I ran a smoke test which consists in executing all the start/grade/finish scripts of the course "from top to bottom" (in the order that are presented in the book).

      I found out that the updates-olm start script has an error

      + lab start updates-olm
      🤖 Running: lab start updates-olm                                                          Starting Verifying cluster state
      Done                                                                                       Starting Install web-terminal operator                                                     Done                                                                                               - An unexpected error ocurred: Timed out.                                                  - Check /tmp/log/labs/updates-olm or use the 'lab logs' command for details.               - Cannot continue lab script                                                                                                                                                  An error has occurred.
                                                                                                 For more details, run: lab logs updates-olm                                                                                                                                           real    5m11.656s
      user    0m13.114s
      sys     0m2.527s
      + echo 1
      

      These are the full logs of the lab script at the DEBUG level

      2025-11-30 23:48:10,389:updates-olm:INFO:lablog.py(229)                #################### updates-olm ####################
      2025-11-30 23:48:10,497:updates-olm:DEBUG:cmd.py(1270)                Popen(['git', 'version'], cwd=/home/student, stdin=None, shell=False, universal_newlines=False)
      2025-11-30 23:48:10,500:updates-olm:DEBUG:cmd.py(1270)                Popen(['git', 'version'], cwd=/home/student, stdin=None, shell=False, universal_newlines=False)
      Starting Verifying cluster state
      2025-11-30 23:48:10,855:updates-olm:DEBUG:health.py(225)                Waiting for API to be online...
      2025-11-30 23:48:10,869:updates-olm:DEBUG:utils.py(50)                Original max_wait: 0:04:00
      2025-11-30 23:48:10,869:updates-olm:DEBUG:utils.py(52)                Adjusted max_wait 0:04:00 (1.0 multiplier)
      2025-11-30 23:48:11,175:updates-olm:DEBUG:connectionpool.py(1049)                Starting new HTTPS connection (1): api.ocp4.example.com:6443
      2025-11-30 23:48:11,216:updates-olm:DEBUG:connectionpool.py(544)                https://api.ocp4.example.com:6443 "GET /version?timeout=10s HTTP/1.1" 200 308
      2025-11-30 23:48:11,218:updates-olm:DEBUG:utils.py(63)                Executed condition, result was True
      2025-11-30 23:48:11,218:updates-olm:DEBUG:health.py(211)                Waiting for router to be up...
      2025-11-30 23:48:11,226:updates-olm:DEBUG:utils.py(50)                Original max_wait: 0:14:00
      2025-11-30 23:48:11,226:updates-olm:DEBUG:utils.py(52)                Adjusted max_wait 0:14:00 (1.0 multiplier)
      2025-11-30 23:48:11,531:updates-olm:DEBUG:connectionpool.py(1049)                Starting new HTTPS connection (1): console-openshift-console.apps.ocp4.example.com:443
      2025-11-30 23:48:11,573:updates-olm:DEBUG:connectionpool.py(544)                https://console-openshift-console.apps.ocp4.example.com:443 "GET / HTTP/1.1" 200 None
      2025-11-30 23:48:11,576:updates-olm:DEBUG:utils.py(63)                Executed condition, result was True
      2025-11-30 23:48:11,576:updates-olm:DEBUG:health.py(194)                Waiting for authentication to be up...
      2025-11-30 23:48:11,588:updates-olm:DEBUG:utils.py(50)                Original max_wait: 0:07:00
      2025-11-30 23:48:11,588:updates-olm:DEBUG:utils.py(52)                Adjusted max_wait 0:07:00 (1.0 multiplier)
      2025-11-30 23:48:11,893:updates-olm:DEBUG:connectionpool.py(1049)                Starting new HTTPS connection (1): oauth-openshift.apps.ocp4.example.com:443
      2025-11-30 23:48:11,936:updates-olm:DEBUG:connectionpool.py(544)                https://oauth-openshift.apps.ocp4.example.com:443 "GET / HTTP/1.1" 403 185
      2025-11-30 23:48:11,938:updates-olm:DEBUG:utils.py(63)                Executed condition, result was True
      2025-11-30 23:48:11,938:updates-olm:DEBUG:health.py(176)                Waiting for operators to stop progressing...
      2025-11-30 23:48:11,945:updates-olm:DEBUG:utils.py(50)                Original max_wait: 0:06:00
      2025-11-30 23:48:11,945:updates-olm:DEBUG:utils.py(52)                Adjusted max_wait 0:06:00 (1.0 multiplier)
      2025-11-30 23:48:12,278:updates-olm:DEBUG:utils.py(63)                Executed condition, result was True
      2025-11-30 23:48:12,278:updates-olm:DEBUG:health.py(157)                Waiting for machine config pools to stop updating...
      2025-11-30 23:48:12,286:updates-olm:DEBUG:utils.py(50)                Original max_wait: 0:02:00
      2025-11-30 23:48:12,286:updates-olm:DEBUG:utils.py(52)                Adjusted max_wait 0:02:00 (1.0 multiplier)
      2025-11-30 23:48:12,599:updates-olm:DEBUG:utils.py(63)                Executed condition, result was True
      2025-11-30 23:48:12,600:updates-olm:DEBUG:health.py(151)                Cluster verification returned: True
      Done
      Starting Install web-terminal operator
      2025-11-30 23:48:13,242:updates-olm:DEBUG:ocp_funcs.py(165)                Waiting for install plan name
      2025-11-30 23:48:13,242:updates-olm:DEBUG:utils.py(18)                Elapsed 0:00:00.000002 - Max wait 0:05:00
      2025-11-30 23:48:13,554:updates-olm:DEBUG:utils.py(23)                Executed condition, result was (MissingModelBranch)
      
      	...	...	...	...	...	...	...	...	...
      
      2025-11-30 23:53:18,080:updates-olm:DEBUG:utils.py(18)                Elapsed 0:05:04.837650 - Max wait 0:05:00
      2025-11-30 23:53:18,080:updates-olm:ERROR:step.py(54)                Step 'Install web-terminal operator' has failed. 
      Traceback (most recent call last):
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/do280/common/ocp/ocp_funcs.py", line 166, in install_operator_step
          installplan_name = utils.wait_until(
              lambda: oc.selector(f"subscription/{name}").object().model.status.installplan.name,
              5,
              datetime.timedelta(minutes=5),
          )
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/do280/common/utils.py", line 20, in wait_until
          raise Exception("Timed out")
      Exception: Timed out
      Done
              - An unexpected error ocurred: Timed out. 
              - Check /tmp/log/labs/updates-olm or use the 'lab logs' command for details.
              - [31mCannot continue lab script
      2025-11-30 23:53:18,083:updates-olm:ERROR:lab.py(61)                Lab script has failed due to an error: Step 'Install web-terminal operator' has failed
      Traceback (most recent call last):
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/do280/common/ocp/ocp_funcs.py", line 166, in install_operator_step
          installplan_name = utils.wait_until(
              lambda: oc.selector(f"subscription/{name}").object().model.status.installplan.name,
              5,
              datetime.timedelta(minutes=5),
          )
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/do280/common/utils.py", line 20, in wait_until
          raise Exception("Timed out")
      Exception: Timed out
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/labs/lab.py", line 349, in start
          grading.prestart()
          ~~~~~~~~~~~~~~~~^^
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/labs/grading.py", line 27, in prestart
          self.start()
          ~~~~~~~~~~^^
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/do280/updates-olm.py", line 34, in start
          ocp_funcs.install_operator_step(
          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
              name="web-terminal",
              ^^^^^^^^^^^^^^^^^^^^
          ...<3 lines>...
              cleanup_function=ocp_funcs.web_terminal_cleanup_step,
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          )
          ^
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/do280/common/ocp/ocp_funcs.py", line 143, in install_operator_step
          with Step(f"Install {name} operator"):
               ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/labs/ui/step.py", line 62, in __exit__
          self.end()
          ~~~~~~~~^^
        File "/home/student/.cache/uv/archive-v0/XpTYhXkLCudLDBUqRY_A_/lib/python3.13/site-packages/labs/ui/step.py", line 143, in end
          raise StepFatalError(self)
      labs.ui.step.StepFatalError: Step 'Install web-terminal operator' has failed
      [31m
      An error has occurred.
      
      For more details, run: lab logs updates-olm
      

              gls-curriculum-ocp-core@redhat.com PTL - OCP Platform Team
              rht-anhernan Andres Hernandez
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: