-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
None
The plugin artifacts-collector started too earlier while waiting not enough time to chained plugin initialization (blocked plugin).
The environment was a hosted cluster installed with Hypershift using 4.19.0-ec.1 release x86_64 with three nodes. Looks like the images takes more time to login and download the required e2e image from MGR cluster.
mtulio@localhost opct-hosted]$ oc logs pod/sonobuoy-99-openshift-artifacts-collector-job-448ea82e36f34e82 -c plugin -n opct 2025-02-04T19:48:55+00:00 | [collector] | entrypoint-collector:17> Starting plugin... time="2025-02-04T19:48:55Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04T19:48:55Z" level=info msg="[ERR] POST http://127.0.0.1:8099/progress request failed: Post \"http://127.0.0.1:8099/progress\": dial tcp 127.0.0.1:8099: connect: connection refused" time="2025-02-04T19:48:55Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress: retrying in 1s (5 left)" 2025-02-04T19:48:56+00:00 | [collector] | entrypoint-collector:38> logging to the cluster... 2025-02-04T19:48:56+00:00 | [collector] | entrypoint-collector:39> [executor] Checking if credentials are present... 2025-02-04T19:48:56+00:00 | [collector] | entrypoint-collector:43> [login] Login to OpenShift cluster [https://kubernetes.default.svc:443] Logged into "https://kubernetes.default.svc:443" as "system:serviceaccount:opct:sonobuoy-serviceaccount" using the token provided. You have access to 62 projects, the list has been suppressed. You can list all projects with 'oc projects' Using project "default". 2025-02-04T19:48:57+00:00 | [collector] | entrypoint-collector:53> [watch_dependency] Starting dependency check... 2025-02-04T19:48:57+00:00 | [collector] | entrypoint-collector:55> waiting for plugin [80-openshift-tests-replay] time="2025-02-04 19:48:57" level=info msg="Initializing dependency waiter for plugin[openshift-artifacts-collector] blocked by[openshift-tests-replay]..." time="2025-02-04 19:48:57" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:48:57" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:48:57" level=info msg="[58.001µs/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:48:57" level=info msg="[58.001µs/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:48:57Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:48:57" level=info msg="[58.001µs/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:02" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:02" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:02" level=info msg="[5.021590738s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:02" level=info msg="[5.021590738s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:02Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:02" level=info msg="[5.021590738s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:07" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:07" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:07" level=info msg="[10.044590084s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:07" level=info msg="[10.044590084s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:07Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:07" level=info msg="[10.044590084s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:12" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:12" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:12" level=info msg="[15.069586547s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:12" level=info msg="[15.069586547s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:12Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:12" level=info msg="[15.069586547s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:17" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:17" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:17" level=info msg="[20.101592295s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:17" level=info msg="[20.101592295s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:17Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:17" level=info msg="[20.101592295s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:22" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:22" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:22" level=info msg="[25.126647561s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:22" level=info msg="[25.126647561s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:22Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:22" level=info msg="[25.126647561s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:27" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:27" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:27" level=info msg="[30.150657207s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:27" level=info msg="[30.150657207s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:27Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:27" level=info msg="[30.150657207s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:32" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:32" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:32" level=info msg="[35.174680279s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:32" level=info msg="[35.174680279s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:32Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:32" level=info msg="[35.174680279s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:37" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:37" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:37" level=info msg="[40.289680278s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:37" level=info msg="[40.289680278s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:37Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:37" level=info msg="[40.289680278s/6h0m0s] reconciling: pod[openshift-tests-replay] is in failed state or returned unexpected value (Phase==[NotReady]). Starting timeout..." time="2025-02-04 19:49:42" level=info msg="Reconciling blocker plugin waiter: plugin=openshift-artifacts-collector blocked by=openshift-tests-replay" time="2025-02-04 19:49:42" level=info msg="Getting pod with labels: {map[component:sonobuoy sonobuoy-plugin:80-openshift-tests-replay] []}\n" time="2025-02-04 19:49:42" level=info msg="[45.331687777s/6h0m0s] reconciling: blocker info: status=running podPhase=NotReady" time="2025-02-04 19:49:42" level=info msg="[45.331687777s/6h0m0s] reconciling: sending message=waiting-for" time="2025-02-04T19:49:42Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" time="2025-02-04 19:49:42" level=error msg="Pod[openshift-tests-replay] is in failed state or returned unxpected value (Phase==[NotReady]). Stop waiter and continue..." time="2025-02-04 19:49:42" level=info msg="Plugin blocker waiter is unlocked." time="2025-02-04 19:49:42" level=info msg="exec wait-updater completed!" time="2025-02-04 19:49:42" level=info msg="Plugin done controller activated." 2025-02-04T19:49:42+00:00 | [collector] | entrypoint-collector:64> [plugin dependencies] Finished! time="2025-02-04T19:49:42Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" 2025-02-04T19:49:42+00:00 | [collector] | entrypoint-collector:71> starting executor... 2025-02-04T19:49:42+00:00 | [collector] | collector:153> [executor][PluginID#99] Starting Artifacts Collector /tmp/sonobuoy/results /plugin time="2025-02-04T19:49:42Z" level=info msg="[DEBUG] POST http://127.0.0.1:8099/progress" 2025-02-04T19:49:42+00:00 | [collector] | collector:32> [executor][PluginID#99] Collecting must-gather [must-gather ] OUT 2025-02-04T19:49:43.150131241Z Using must-gather plug-in image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:6581d8737b7503fa9baf2063b6df1bdfd8f63c49a810bd96e9aaa4ece139f6f6 When opening a support case, bugzilla, or issue please include the following summary data along with any other requested information: