Description of problem:

unable to deploy Unable to deploy RHODS Operator as

DataScienceCluster resource reconciled with component errors

Prerequisites (if any, like setup, operators/versions):

Steps to Reproduce

1. Create a osd cluster
2. Deploy rhods operator 

Actual results:

Failed to create pod because of permission issue 

DataScienceCluster resource reconciled with component errors: 6 errors occurred:\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\n"

Expected results:

all the expected pod should be up

Reproducibility (Always/Intermittent/Only Once):

2/2

Build Details:

brew.registry.redhat.io/rh-osbs/iib:611348
 
Workaround:
NA

Additional info:

conditions:

• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:31:24Z'
  message: "DataScienceCluster resource reconciled with component errors: 6 errors occurred:\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\n"
  reason: ReconcileCompletedWithComponentErrors
  status: 'True'
  type: ReconcileComplete
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:31:24Z'
  message: "DataScienceCluster resource reconciled with component errors: 6 errors occurred:\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\n"
  reason: ReconcileCompletedWithComponentErrors
  status: 'True'
  type: Available
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:31:24Z'
  message: "DataScienceCluster resource reconciled with component errors: 6 errors occurred:\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\n"
  reason: ReconcileCompletedWithComponentErrors
  status: 'False'
  type: Progressing
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:31:15Z'
  message: "DataScienceCluster resource reconciled with component errors: 6 errors occurred:\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\n"
  reason: ReconcileCompletedWithComponentErrors
  status: 'False'
  type: Degraded
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:31:24Z'
  message: "DataScienceCluster resource reconciled with component errors: 6 errors occurred:\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* RoleBinding.rbac.authorization.k8s.io \"redhat-ods-applications\" not found\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\t* open /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no such file or directory\n\n"
  reason: ReconcileCompletedWithComponentErrors
  status: 'True'
  type: Upgradeable
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:32:29Z'
  message: >-
  Component reconciliation failed: RoleBinding.rbac.authorization.k8s.io
  "redhat-ods-applications" not found
  reason: ReconcileFailed
  status: 'False'
  type: dashboardReady
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:32:29Z'
  message: >-
  Component reconciliation failed: RoleBinding.rbac.authorization.k8s.io
  "redhat-ods-applications" not found
  reason: ReconcileFailed
  status: 'False'
  type: workbenchesReady
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:32:29Z'
  message: >-
  Component reconciliation failed: RoleBinding.rbac.authorization.k8s.io
  "redhat-ods-applications" not found
  reason: ReconcileFailed
  status: 'False'
  type: model-meshReady
• lastHeartbeatTime: '2023-10-31T05:32:30Z'
  lastTransitionTime: '2023-10-31T05:32:30Z'
  message: >-
  Component reconciliation failed: open
  /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no
  such file or directory
  reason: ReconcileFailed
  status: 'False'
  type: data-science-pipelines-operatorReady
• lastHeartbeatTime: '2023-10-31T05:32:32Z'
  lastTransitionTime: '2023-10-31T05:32:32Z'
  message: >-
  Component removal failed: open
  /opt/manifests/monitoring/prometheus/apps/prometheus-configs.yaml: no
  such file or directory
  reason: ReconcileFailed
  status: 'False'
  type: codeflareReady
• lastHeartbeatTime: '2023-10-31T05:32:33Z'
  lastTransitionTime: '2023-10-31T05:32:33Z'
  message: Component is disabled
  reason: ReconcileInit
  status: Unknown
  type: rayReady
• lastHeartbeatTime: '2023-10-31T05:32:29Z'
  lastTransitionTime: '2023-10-31T05:32:29Z'
  message: Component reconciled successfully
  reason: ReconcileCompleted
  status: 'True'
  type: trustyaiReady