-
Bug
-
Resolution: Done
-
Normal
-
None
-
4.13.z
-
None
-
Quality / Stability / Reliability
-
False
-
-
5
-
Important
-
No
-
None
-
None
-
OSDOCS Sprint 242
-
1
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
Api-checks-pod crash as getting a null pod value on a aws-ipi-ovn-hypershift-mce-mgmt cluster: $ oc logs pod/hypershift-cisk57aw88gry-api-checks-pod --all-containers … Fetching URI: '/api/v1/namespaces/clusters-79136a1bdb84b3c13217/pods?labelSelector=app%3Dkube-controller-manager' FATAL:Error fetching resources: couldn't filter '{"kind":"PodList","apiVersion":"v1","metadata":{"resourceVersion":"188114"},"items":[]} ': cannot iterate over: null Error from server (BadRequest): container "log-collector" in pod "hypershift-cisk57aw88gry-api-checks-pod" is waiting to start: PodInitializing
Version-Release number of selected component (if applicable):
registry.ci.openshift.org/ocp/release:4.14.0-0.nightly-2023-07-11-092038
Version-Release number of selected component (if applicable):
4.13.5
How reproducible:
Always
Steps to Reproduce:
1. Install compliance operator 2. Create a tailoredprofile: $ oc get hostedcluster -A NAMESPACE NAME VERSION KUBECONFIG PROGRESS AVAILABLE PROGRESSING MESSAGE local-cluster 79136a1bdb84b3c13217 4.13.5 79136a1bdb84b3c13217-admin-kubeconfig Completed True False The hosted control plane is available $ cat tp.json { "kind": "List", "apiVersion": "v1", "metadata": {}, "items": [ { "apiVersion": "compliance.openshift.io/v1alpha1", "kind": "TailoredProfile", "metadata": { "name": "hypershift-cisk57aw88gry", "namespace": "openshift-compliance" }, "spec": { "description": "This profile test required rules", "extends": "ocp4-cis", "setValues": [ { "name": "ocp4-hypershift-cluster", "rationale": "This value is used for HyperShift version detection", "value": "79136a1bdb84b3c13217" } ], "title": "My little profile" } } ] } $ oc apply -f /tmp/e2e-test-compliance-b5fpn5pl8x-mbrtr-ut2ihk3kktisc-config.json tailoredprofile.compliance.openshift.io/hypershift-cisk57aw88gry created $ oc get tp NAME STATE hypershift-cisk57aw88gry READY 3. Create a ssb: $ oc compliance bind -N test -S default tailoredprofile/hypershift-cisk57aw88gry Creating ScanSettingBinding test Actual result: The pod hypershift-cisk57aw88gry-api-checks-pod crash.The compliancesuite will stuck at Running status. $ oc get pod NAME READY STATUS RESTARTS AGE compliance-operator-68577cd558-wj6wg 1/1 Running 1 (104m ago) 104m hypershift-cisk57aw88gry-api-checks-pod 0/2 Init:CrashLoopBackOff 10 (42s ago) 28m hypershift-cisk57aw88gry-rs-7866c6b8c9-5qfbk 1/1 Running 0 28m ocp4-openshift-compliance-pp-5f5f567b85-hz5fx 1/1 Running 0 104m rhcos4-openshift-compliance-pp-5c445fd5f5-xrhxx 1/1 Running 0 104m
Actual results:
The pod hypershift-cisk57aw88gry-api-checks-pod crash.The compliancesuite will stuck at Running status.
$ oc get pod
NAME READY STATUS RESTARTS AGE
compliance-operator-68577cd558-wj6wg 1/1 Running 1 (104m ago) 104m
hypershift-cisk57aw88gry-api-checks-pod 0/2 Init:CrashLoopBackOff 10 (42s ago) 28m
hypershift-cisk57aw88gry-rs-7866c6b8c9-5qfbk 1/1 Running 0 28m
ocp4-openshift-compliance-pp-5f5f567b85-hz5fx 1/1 Running 0 104m
rhcos4-openshift-compliance-pp-5c445fd5f5-xrhxx 1/1 Running 0 104m
Expected results:
The pod hypershift-cisk57aw88gry-api-checks-pod should not crash.The compliancesuite should return COMPLIANT/NON-COMPLIANT soon.
Additional info:
$ oc get ns -l hypershift.openshift.io/hosted-control-plane=true NAME STATUS AGE local-cluster-79136a1bdb84b3c13217 Active 4h30m $ oc get ns local-cluster-79136a1bdb84b3c13217 --show-labels NAME STATUS AGE LABELS local-cluster-79136a1bdb84b3c13217 Active 4h27m hypershift.openshift.io/hosted-control-plane=true,hypershift.openshift.io/monitoring=true,kubernetes.io/metadata.name=local-cluster-79136a1bdb84b3c13217,pod-security.kubernetes.io/audit-version=v1.24,pod-security.kubernetes.io/audit=privileged,pod-security.kubernetes.io/warn-version=v1.24,pod-security.kubernetes.io/warn=privileged $ oc get pod -n local-cluster-79136a1bdb84b3c13217 -l app=kube-controller-manager NAME READY STATUS RESTARTS AGE kube-controller-manager-668b6d8549-rzln9 2/2 Running 0 4h19m
- links to