Loading...

Type: Bug
Resolution: Done-Errata
Priority: Blocker
Fix Version/s: OADP 1.5.0
Affects Version/s: OADP 1.5.0
Component/s: cloudstorage
Labels:

Activity Type:
Quality / Stability / Reliability
Story Points:
3
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Ready:
False
Fixed in Build:
oadp-operator-bundle-container-1.5.0-94
QEStatus:
ToDo
Intelligence Requested:
Market:

Risk Probability:
Very Likely
Risk Score:
0

Workstream:

None

Root Cause:
Unset
Failure Category:
Unknown

Regression:
Yes

SFDC Cases Links:
SFDC Cases Open:
SFDC Cases Counter:

Description of problem:

OADP operator is failing to create bucket on top of aws. OADP operator is installed with standerdized sts configuration. Attached logs below

2025-04-24T11:35:41Z ERROR unable to determine if bucket exists. {"controller": "cloudstorage", "controllerGroup": "oadp.openshift.io", "controllerKind": "CloudStorage", "CloudStorage": {"name":"test-oadp","namespace":"openshift-adp"}, "namespace": "openshift-adp", "name": "test-oadp", "reconcileID": "9628ea56-bdad-45e1-93d1-f3bdbca583c3", "bucket": {"name":"test-oadp","namespace":"openshift-adp"}, "error": "open /tmp/aws-shared-credentials1211864681: read-only file system"} github.com/openshift/oadp-operator/internal/controller.CloudStorageReconciler.Reconcile /remote-source/internal/controller/cloudstorage_controller.go:154 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Reconcile /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:116 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).reconcileHandler /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:303 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).processNextWorkItem /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:263 sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller[...]).Start.func2.2 /remote-source/deps/gomod/pkg/mod/sigs.k8s.io/controller-runtime@v0.19.3/pkg/internal/controller/controller.go:224 2025-04-24T11:35:41Z DEBUG events unable to find bucket: open /tmp/aws-shared-credentials1211864681: read-only file system {"type": "Warning", "object": {"kind":"CloudStorage","namespace":"openshift-adp","name":"test-oadp","uid":"173c28e3-6f5c-4b78-9eac-dd92508de954","apiVersion":"oadp.openshift.io/v1alpha1","resourceVersion":"277007"}, "reason": "BucketNotFound"}

Version-Release number of selected component (if applicable):
ROSA 4.19
OADP 1.5.0

How reproducible:

Always

Steps to Reproduce:

1. Install OADP with standardized configuration

2. Create cloudstorage CR

oc get cloudstorage -o yaml test-oadp
apiVersion: oadp.openshift.io/v1alpha1
kind: CloudStorage
metadata:
  creationTimestamp: "2025-04-24T11:15:41Z"
  finalizers:
  - oadp.openshift.io/bucket-protection
  generation: 1
  name: test-oadp
  namespace: openshift-adp
  resourceVersion: "277007"
  uid: 173c28e3-6f5c-4b78-9eac-dd92508de954
spec:
  creationSecret:
    key: credentials
    name: cloud-credentials
  enableSharedConfig: true
  name: oadprosa2461h4rgs
  provider: aws
  region: us-east-1

Actual results:

openshift-adp-controller pod has an error related to bucket not found.

Secret content

[default]
sts_regional_endpoints = regional
role_arn = arn:aws:iam::160792166956:role/oadprosa2461h4rgs
web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token

Expected results:

There should be no error in CR

Additional info:

Tested the same thing with OADP 1.4.4 (GA) using standardized sts configuration, It works fine.

Created cloudstorage CR

$ oc get cloudstorage -o yaml
apiVersion: v1
items:
- apiVersion: oadp.openshift.io/v1alpha1
  kind: CloudStorage
  metadata:
    creationTimestamp: "2025-04-24T08:08:00Z"
    finalizers:
    - oadp.openshift.io/bucket-protection
    generation: 1
    name: test-oadp
    namespace: openshift-adp
    resourceVersion: "184411"
    uid: f60f3499-4812-4149-8093-bac925271846
  spec:
    creationSecret:
      key: credentials
      name: cloud-credentials
    enableSharedConfig: true
    name: oadprosa2461h4rgs
    provider: aws
    region: us-east-1
  status:
    lastSyncTimestamp: "2025-04-24T08:08:00Z"
    name: oadprosa2461h4rgs
kind: List
metadata:
  resourceVersion: ""

$ oc get secret cloud-credentials -o yaml
[default]
sts_regional_endpoints = regional
role_arn = arn:aws:iam::160792166956:role/oadprosa2461h4rgs
web_identity_token_file = /var/run/secrets/openshift/serviceaccount/token