Uploaded image for project: 'OpenShift Storage'
  1. OpenShift Storage
  2. STOR-2039

[secrets-store-csi-driver-provider-gcp] PermissionDenied: unable to obtain auth for mount: unable to obtain workload identity auth

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • Proposed

      Description of problem:

      PermissionDenied desc = unable to obtain auth for mount: unable to obtain workload identity auth: unexpected end of JSON input    

      Version-Release number of selected component (if applicable):

      4.17.0-0.nightly-2024-08-09-031511

      How reproducible:

      Always    

      Steps to Reproduce:

      1. Create GCP cluster from below mentioned flexy template, it helps to create workload identity pool + provider using ccoctl tool
         ipi-on-gcp/versioned-installer-sts-ci
      2. Install secret store operator, driver, gcp provider and check the pods are up and running. 
      GCP Provider: https://github.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp/blob/main/deploy/provider-gcp-plugin.yaml 
      
      oc get pods -n openshift-cluster-csi-drivers | grep "secret"
      csi-secrets-store-provider-gcp-55x5m                 1/1     Running   0               115m
      csi-secrets-store-provider-gcp-7bnzj                 1/1     Running   0               115m
      csi-secrets-store-provider-gcp-bp4gf                 1/1     Running   0               115m
      secrets-store-csi-driver-node-bhj4h                  3/3     Running   0               116m
      secrets-store-csi-driver-node-h4fgz                  3/3     Running   0               116m
      secrets-store-csi-driver-node-j7tzd                  3/3     Running   0               116m
      secrets-store-csi-driver-node-jxkg4                  3/3     Running   0               116m
      secrets-store-csi-driver-node-lmrr4                  3/3     Running   0               116m
      secrets-store-csi-driver-node-wjq7p                  3/3     Running   0               116m
      secrets-store-csi-driver-operator-56ff8bcdbb-ldmbl   1/1     Running   0               117m
      3. Do precondition setup as per the doc: https://github.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp/blob/main/README.md 
      4. Create secret in the GCP secretManager and the permissions
      5. Create secretproviderclass and pod as mentioned in doc and check status: https://github.com/GoogleCloudPlatform/secrets-store-csi-driver-provider-gcp/tree/main/examples 

      Actual results:

      NAME    READY   STATUS              RESTARTS   AGE
      mypod   0/1     ContainerCreating   0          13m
      
        Warning  FailedMount  5s (x4 over 12s)  kubelet            MountVolume.SetUp failed for volume "mysecret" : rpc error: code = PermissionDenied desc = failed to mount secrets store objects for pod default/mypod, err: rpc error: code = PermissionDenied desc = unable to obtain auth for mount: unable to obtain workload identity auth: unexpected end of JSON input    

      Expected results:

      The pod should be up and running and able to mount the secret successfully

      Additional info:

      Discussion: https://redhat-internal.slack.com/archives/CJED3290A/p1723178628185569 

              jdobson@redhat.com Jonathan Dobson
              ropatil@redhat.com Rohit Patil
              Rohit Patil Rohit Patil
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: