Uploaded image for project: 'OpenShift Storage'
  1. OpenShift Storage
  2. STOR-1439

Automatically restart `alibaba-disk-csi-driver-controller` pods when the secret `alibaba-disk-csi-driver-controller-metrics-serving-cert` is updated

    • Icon: Story Story
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • BU Product Work
    • 1
    • False
    • None
    • False

      The pod `alibaba-disk-csi-driver-controller` mounts the secret:

      $ cat assets/controller.yaml
      ...
            containers:
              - name: provisioner-kube-rbac-proxy
      
                volumeMounts:
                - mountPath: /etc/tls/private
                  name: metrics-serving-cert
      
            volumes:
              - name: metrics-serving-cert
                secret:
                  secretName: alibaba-disk-csi-driver-controller-metrics-serving-cert
      

      Hence, if the secret is updated (e.g. as a result of CA cert update), the Pod must be restarted

            [STOR-1439] Automatically restart `alibaba-disk-csi-driver-controller` pods when the secret `alibaba-disk-csi-driver-controller-metrics-serving-cert` is updated

            Test passed with the pre merged build `4.14.0-0.ci.test-2023-09-01-085811-ci-ln-rxpny9b-latest` (build with openshift/cluster-cloud-controller-manager-operator#276 which solved the install failed blocker issue)

            # After the metrics-serving-cert secret changed, driver controller restarted
             
             wangpenghao@MacBook-Pro  ~  oc delete secret alibaba-disk-csi-driver-controller-metrics-serving-cert
            secret "alibaba-disk-csi-driver-controller-metrics-serving-cert" deleted
            
             wangpenghao@MacBook-Pro  ~  oc get po -l app=alibaba-disk-csi-driver-controller -w
            NAME                                                  READY   STATUS    RESTARTS        AGE
            alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   10/10   Running   0               8h
            alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   10/10   Running   4 (7h58m ago)   8h
            alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   10/10   Terminating   4 (7h58m ago)   8h
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Pending       0               0s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Pending       0               0s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    ContainerCreating   0               0s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               1s
            alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    Pending             0               0s
            alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    Pending             0               0s
            alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   0/10    Terminating         4 (7h58m ago)   8h
            alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   0/10    Terminating         4 (7h58m ago)   8h
            alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   0/10    Terminating         4 (7h58m ago)   8h
            alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    Pending             0               1s
            alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    ContainerCreating   0               1s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   10/10   Terminating         0               3s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               3s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               4s
            alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               4s
            alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    10/10   Running             0               5s
            alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   10/10   Terminating         0               8h
            alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    0/10    Pending             0               0s
            alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    0/10    Pending             0               0s
            alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    0/10    ContainerCreating   0               0s
            alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   0/10    Terminating         0               8h
            alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   0/10    Terminating         0               8h
            alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   0/10    Terminating         0               8h
            alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    10/10   Running             0               3s
            ^C%                                                                                                                                                                       ✘ wangpenghao@MacBook-Pro  ~  oc get po -l app=alibaba-disk-csi-driver-controller
            NAME                                                 READY   STATUS    RESTARTS   AGE
            alibaba-disk-csi-driver-controller-c4df967f9-ksz6k   10/10   Running   0          13s
            alibaba-disk-csi-driver-controller-c4df967f9-rrlw2   10/10   Running   0          18s

             

            Penghao Wang added a comment - Test passed with the pre merged build `4.14.0-0.ci.test-2023-09-01-085811-ci-ln-rxpny9b-latest` (build with openshift/cluster-cloud-controller-manager-operator#276 which solved the install failed blocker issue) # After the metrics-serving-cert secret changed, driver controller restarted    wangpenghao@MacBook-Pro  ~  oc delete secret alibaba-disk-csi-driver-controller-metrics-serving-cert secret "alibaba-disk-csi-driver-controller-metrics-serving-cert" deleted  wangpenghao@MacBook-Pro  ~  oc get po -l app=alibaba-disk-csi-driver-controller -w NAME                                                  READY   STATUS    RESTARTS        AGE alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   10/10   Running   0               8h alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   10/10   Running   4 (7h58m ago)   8h alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   10/10   Terminating   4 (7h58m ago)   8h alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Pending       0               0s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Pending       0               0s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    ContainerCreating   0               0s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               1s alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    Pending             0               0s alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    Pending             0               0s alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   0/10    Terminating         4 (7h58m ago)   8h alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   0/10    Terminating         4 (7h58m ago)   8h alibaba-disk-csi-driver-controller-599b69c69c-j6sbg   0/10    Terminating         4 (7h58m ago)   8h alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    Pending             0               1s alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    0/10    ContainerCreating   0               1s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   10/10   Terminating         0               3s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               3s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               4s alibaba-disk-csi-driver-controller-5dfb99d58d-jpdb8   0/10    Terminating         0               4s alibaba-disk-csi-driver-controller-c4df967f9-rrlw2    10/10   Running             0               5s alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   10/10   Terminating         0               8h alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    0/10    Pending             0               0s alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    0/10    Pending             0               0s alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    0/10    ContainerCreating   0               0s alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   0/10    Terminating         0               8h alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   0/10    Terminating         0               8h alibaba-disk-csi-driver-controller-599b69c69c-4fdbv   0/10    Terminating         0               8h alibaba-disk-csi-driver-controller-c4df967f9-ksz6k    10/10   Running             0               3s ^C%                                                                                                                                                                       ✘ wangpenghao@MacBook-Pro  ~  oc get po -l app=alibaba-disk-csi-driver-controller NAME                                                 READY   STATUS    RESTARTS   AGE alibaba-disk-csi-driver-controller-c4df967f9-ksz6k   10/10   Running   0          13s alibaba-disk-csi-driver-controller-c4df967f9-rrlw2   10/10   Running   0          18s  

            Since the blocker bug https://issues.redhat.com/browse/OCPBUGS-17652 , we couldn't install alibaba cloud clusters on 4.14, currently I haven't done the pre/post merge test about this user story.

            Penghao Wang added a comment - Since the blocker bug https://issues.redhat.com/browse/OCPBUGS-17652 , we couldn't install alibaba cloud clusters on 4.14, currently I haven't done the pre/post merge test about this user story.

              rh-ee-mpatlaso Maxim Patlasov
              rh-ee-mpatlaso Maxim Patlasov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: