Description of problem:
After compliance-operator upgrade from v1.5.1 to v1.6.0, create a ssb with ocp4-stig, ocp4-stig-node, rhcos4-stig profiles. The compliancesuite will stuck at Running phase. This is because one scan pod will always stuck at “NotReady” due to “Failed to upload results to server”.

% cat ~/func/ssb.yaml 

apiVersion: compliance.openshift.io/v1alpha1

kind: ScanSettingBinding

metadata:

  name: stig-compliance

  namespace: openshift-compliance

profiles:

  - name: ocp4-stig-node

    kind: Profile

    apiGroup: compliance.openshift.io/v1alpha1

  - name: ocp4-stig

    kind: Profile

    apiGroup: compliance.openshift.io/v1alpha1

  - name: rhcos4-stig

    kind: Profile

    apiGroup: compliance.openshift.io/v1alpha1

settingsRef:

  name: default

  kind: ScanSetting

  apiGroup: compliance.openshift.io/v1alpha1

% oc apply -f ~/func/ssb.yaml 

scansettingbinding.compliance.openshift.io/stig-compliance created

% oc get pod

NAME                                                          READY   STATUS      RESTARTS      AGE

compliance-operator-f57dbddc-xjp5m                            1/1     Running     1 (89m ago)   89m

ocp4-openshift-compliance-pp-6c747c677c-lltz5                 1/1     Running     0             89m

ocp4-stig-node-worker-rs-7cc68b796c-9pc2g                     1/1     Running     0             9m17s

ocp4-stig-node-worker-xiyuan-415-a-gxw64-worker-a-chlng-pod   1/2     NotReady    0             9m17s

ocp4-stig-node-worker-xiyuan-415-a-gxw64-worker-b-w7c22-pod   0/2     Completed   0             9m17s

ocp4-stig-node-worker-xiyuan-415-a-gxw64-worker-c-v7t96-pod   0/2     Completed   0             9m17s

rhcos4-openshift-compliance-pp-6df8c85f75-svw68               1/1     Running     0             89m

rhcos4-stig-worker-rs-67d776c4d9-wxkr5                        1/1     Running     0             9m56s

rhcos4-stig-worker-xiyuan-415-a-gxw64-worker-a-chlng-pod      1/2     NotReady    0             9m56s

rhcos4-stig-worker-xiyuan-415-a-gxw64-worker-b-w7c22-pod      0/2     Completed   0             9m56s

rhcos4-stig-worker-xiyuan-415-a-gxw64-worker-c-v7t96-pod      0/2     Completed   0             9m56s

 

% oc logs pod/rhcos4-stig-worker-xiyuan-415-a-gxw64-worker-a-chlng-pod -c log-collector

{"level":"info","ts":"2024-09-25T07:43:18Z","logger":"cmd","msg":"Results file found, will upload it.","resuts-file":"/reports/exit_code"} {"level":"info","ts":"2024-09-25T07:43:18Z","logger":"cmd","msg":"Got exit-code from file","exit-code":"2"} {"level":"info","ts":"2024-09-25T07:43:18Z","logger":"cmd","msg":"Results file found, will upload it.","resuts-file":"/reports/report-arf.xml"} {"level":"info","ts":"2024-09-25T07:43:39Z","logger":"cmd","msg":"File needs compression","results-file":"/reports/report-arf.xml"} {"level":"info","ts":"2024-09-25T07:43:39Z","logger":"cmd","msg":"Compressed results"} {"level":"info","ts":"2024-09-25T07:43:39Z","logger":"cmd","msg":"Results file found, will upload it.","resuts-file":"/reports/report.xml"} {"level":"info","ts":"2024-09-25T07:43:39Z","logger":"cmd","msg":"Trying to upload results ConfigMap"} {"level":"info","ts":"2024-09-25T07:43:39Z","logger":"cmd","msg":"Trying to upload to resultserver","url":"https://rhcos4-stig-worker-rs:8443/"} {"level":"info","ts":"2024-09-25T07:43:40Z","logger":"cmd","msg":"Uploaded ConfigMap"} {"level":"error","ts":"2024-09-25T07:45:57Z","logger":"cmd","msg":"Failed to upload results to server","error":"Post \"https://rhcos4-stig-worker-rs:8443/\": dial tcp 172.30.75.29:8443: connect: connection timed out","stacktrace":"github.com/ComplianceAsCode/compliance-operator/cmd/manager.uploadToResultServer.func1\n\tgithub.com/ComplianceAsCode/compliance-operator/cmd/manager/resultcollector.go:316\ngithub.com/cenkalti/backoff/v4.RetryNotifyWithTimer.Operation.withEmptyData.func1\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:18\ngithub.com/cenkalti/backoff/v4.doRetryNotify[...]\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:88\ngithub.com/cenkalti/backoff/v4.RetryNotifyWithTimer\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:61\ngithub.com/cenkalti/backoff/v4.RetryNotify\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:49\ngithub.com/cenkalti/backoff/v4.Retry\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:38\ngithub.com/ComplianceAsCode/compliance-operator/cmd/manager.uploadToResultServer\n\tgithub.com/ComplianceAsCode/compliance-operator/cmd/manager/resultcollector.go:299\ngithub.com/ComplianceAsCode/compliance-operator/cmd/manager.handleCompleteSCAPResults.func1\n\tgithub.com/ComplianceAsCode/compliance-operator/cmd/manager/resultcollector.go:390"} {"level":"info","ts":"2024-09-25T07:45:57Z","logger":"cmd","msg":"Trying to upload to resultserver","url":"https://rhcos4-stig-worker-rs:8443/"} {"level":"error","ts":"2024-09-25T07:48:08Z","logger":"cmd","msg":"Failed to upload results to server","error":"Post \"https://rhcos4-stig-worker-rs:8443/\": dial tcp 172.30.75.29:8443: connect: connection timed out","stacktrace":"github.com/ComplianceAsCode/compliance-operator/cmd/manager.uploadToResultServer.func1\n\tgithub.com/ComplianceAsCode/compliance-operator/cmd/manager/resultcollector.go:316\ngithub.com/cenkalti/backoff/v4.RetryNotifyWithTimer.Operation.withEmptyData.func1\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:18\ngithub.com/cenkalti/backoff/v4.doRetryNotify[...]\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:88\ngithub.com/cenkalti/backoff/v4.RetryNotifyWithTimer\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:61\ngithub.com/cenkalti/backoff/v4.RetryNotify\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:49\ngithub.com/cenkalti/backoff/v4.Retry\n\tgithub.com/cenkalti/backoff/v4@v4.3.0/retry.go:38\ngithub.com/ComplianceAsCode/compliance-operator/cmd/manager.uploadToResultServer\n\tgithub.com/ComplianceAsCode/compliance-operator/cmd/manager/resultcollector.go:299\ngithub.com/ComplianceAsCode/compliance-operator/cmd/manager.handleCompleteSCAPResults.func1\n\tgithub.com/ComplianceAsCode/compliance-operator/cmd/manager/resultcollector.go:390"} {"level":"info","ts":"2024-09-25T07:48:09Z","logger":"cmd","msg":"Trying to upload to resultserver","url":"[https://rhcos4-stig-worker-rs:8443/]"}

Version-Release number of selected component (if applicable):

How reproducible:

Steps to Reproduce:

Actual results:

Expected results:

Additional info: