Uploaded image for project: 'OpenShift Hosted Control Plane'
  1. OpenShift Hosted Control Plane
  2. HOSTEDCP-669

csi-snapshot-controller is in CrashLoopBackOff after endpointAccess is changed from PublicAndPrivate to Private

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Undefined
    • None
    • None
    • None
    • False
    • None
    • False
    • Hypershift Sprint 21, Hypershift Sprint 22
    • 0
    • 0
    • 0

    Description

      Images: the latest hypershift image + 4.13.0-0.nightly-2022-12-03-190258

      Found while test HOSTEDCP-572

      csi-snapshot-controller is in CrashLoopBackOff after endpointAccess is changed from PublicAndPrivate to Private:
      jiezhao-mac:hypershift jiezhao$ oc get pods -n clusters-jz-test
      NAME                                                  READY   STATUS             RESTARTS      AGE
      capi-provider-846fddcf97-kvl5m                        2/2     Running            0             31m
      catalog-operator-f574499cb-ljftx                      2/2     Running            0             29m
      certified-operators-catalog-757876ddcc-62w7k          1/1     Running            0             29m
      cloud-network-config-controller-bc648776d-gdg9r       3/3     Running            0             27m
      cluster-api-b4f9c9f6c-zztkj                           1/1     Running            0             31m
      cluster-autoscaler-775f8b8ccb-mfr9z                   1/1     Running            0             30m
      cluster-image-registry-operator-56c55bd655-8cb76      3/3     Running            0             29m
      cluster-network-operator-56868b8bbb-578l7             1/1     Running            0             29m
      cluster-node-tuning-operator-598796cf75-tqwkr         1/1     Running            0             29m
      cluster-policy-controller-96fd5cf77-4xw5j             1/1     Running            0             29m
      cluster-policy-controller-96fd5cf77-8qvpv             1/1     Running            0             29m
      cluster-policy-controller-96fd5cf77-tsbrt             1/1     Running            0             29m
      cluster-version-operator-7c47fcb885-g5ksd             1/1     Running            0             29m
      community-operators-catalog-5fcb676869-l2675          1/1     Running            0             29m
      control-plane-operator-54f7f5c7dc-wjm9g               2/2     Running            0             31m
      csi-snapshot-controller-7b59cd8c88-x9xkk              0/1     CrashLoopBackOff   5 (35s ago)   28m
      csi-snapshot-controller-operator-55cd6679b9-qrr6c     1/1     Running            0             29m
      csi-snapshot-webhook-7f8dc5798c-tsc8x                 1/1     Running            0             28m
      dns-operator-55f4d7f8dc-m7sjk                         1/1     Running            0             29m
      etcd-0                                                1/1     Running            0             30m
      etcd-1                                                1/1     Running            0             30m
      etcd-2                                                1/1     Running            0             30m
      hosted-cluster-config-operator-5fb8bcb5f9-sc4w6       1/1     Running            0             29m
      ignition-server-5b559d5d5d-6tpg7                      1/1     Running            0             30m
      ignition-server-5b559d5d5d-krjdw                      1/1     Running            0             30m
      ignition-server-5b559d5d5d-n52lg                      1/1     Running            0             30m
      ingress-operator-54df5c6879-5t7bn                     3/3     Running            0             29m
      konnectivity-agent-78dd7796f-7z7ms                    1/1     Running            0             30m
      konnectivity-agent-78dd7796f-gbd65                    1/1     Running            0             30m
      konnectivity-agent-78dd7796f-mjm5z                    1/1     Running            0             30m
      konnectivity-server-849646c966-rkzb9                  1/1     Running            0             30m
      kube-apiserver-675cc6dd58-l4hzk                       5/5     Running            0             30m
      kube-apiserver-675cc6dd58-mcbdx                       5/5     Running            0             30m
      kube-apiserver-675cc6dd58-n5dh6                       5/5     Running            0             30m
      kube-controller-manager-5b5fcd8dd7-288xh              2/2     Running            0             19m
      kube-controller-manager-5b5fcd8dd7-8nnlb              2/2     Running            0             20m
      kube-controller-manager-5b5fcd8dd7-md2sj              2/2     Running            0             20m
      kube-scheduler-5664cbfc46-rnxqf                       1/1     Running            0             29m
      kube-scheduler-5664cbfc46-v5kts                       1/1     Running            0             29m
      kube-scheduler-5664cbfc46-wpm8r                       1/1     Running            0             29m
      machine-approver-6b7469d69d-2slkv                     1/1     Running            0             30m
      multus-admission-controller-69d789cc6b-w2sk8          2/2     Running            0             27m
      oauth-openshift-66554dbcc-82pz8                       2/2     Running            0             6m28s
      oauth-openshift-66554dbcc-lv4lp                       2/2     Running            0             5m25s
      oauth-openshift-66554dbcc-wk6g6                       2/2     Running            0             5m57s
      olm-operator-69b94cdc58-f9kd5                         2/2     Running            0             29m
      openshift-apiserver-67467cc956-4mckx                  2/2     Running            0             19m
      openshift-apiserver-67467cc956-h6rjp                  2/2     Running            0             20m
      openshift-apiserver-67467cc956-jpb9k                  2/2     Running            0             20m
      openshift-controller-manager-65fb9fffd4-95gqv         1/1     Running            0             29m
      openshift-controller-manager-65fb9fffd4-dnr4z         1/1     Running            0             29m
      openshift-controller-manager-65fb9fffd4-mh7hk         1/1     Running            0             29m
      openshift-oauth-apiserver-5df64db854-6qzkq            1/1     Running            0             29m
      openshift-oauth-apiserver-5df64db854-77ln2            1/1     Running            0             29m
      openshift-oauth-apiserver-5df64db854-t9bzx            1/1     Running            0             29m
      openshift-route-controller-manager-66bc86f649-q65s6   1/1     Running            0             29m
      openshift-route-controller-manager-66bc86f649-ss5x6   1/1     Running            0             29m
      openshift-route-controller-manager-66bc86f649-tw7p8   1/1     Running            0             29m
      ovnkube-master-0                                      7/7     Running            0             27m
      ovnkube-master-1                                      7/7     Running            0             27m
      ovnkube-master-2                                      7/7     Running            0             27m
      packageserver-676548d859-bp8c5                        2/2     Running            0             29m
      packageserver-676548d859-hxdbz                        2/2     Running            0             29m
      packageserver-676548d859-ths2s                        2/2     Running            0             29m
      redhat-marketplace-catalog-684d85c7-mwhss             1/1     Running            0             29m
      redhat-operators-catalog-8545ddf456-rqqht             1/1     Running            0             29m
      router-86868f8db9-5cmsx                               1/1     Running            0             6m29s
      router-86868f8db9-hdxd8                               1/1     Running            0             5m17s
      router-86868f8db9-p5j4r                               1/1     Running            0             5m54s
      jiezhao-mac:hypershift jiezhao$ oc logs csi-snapshot-controller-7b59cd8c88-x9xkk -n clusters-jz-test
      I1204 01:56:07.394528       1 main.go:125] Version: v4.13.0-202211071433.p0.g7e23256.assembly.stream-0-g883ff30-dirty
      I1204 01:56:07.395670       1 main.go:174] Start NewCSISnapshotController with kubeconfig [/etc/kubernetes/kubeconfig] resyncPeriod [15m0s]
      E1204 01:56:07.399385       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:07.502174       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:07.654688       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:07.881612       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:08.222478       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:08.730398       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:09.492397       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:10.633949       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:12.345304       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:14.911500       1 main.go:86] Failed to list v1 volumesnapshots with error=Get "https://api.jz-test.hypershift.local:443/apis/snapshot.storage.k8s.io/v1/volumesnapshots": dial tcp: lookup api.jz-test.hypershift.local on 172.30.0.10:53: no such hostE1204 01:56:14.911528       1 main.go:193] Exiting due to failure to ensure CRDs exist during startup: timed out waiting for the condition
      jiezhao-mac:hypershift jiezhao$ 
       

      Attachments

        Issue Links

          Activity

            People

              sjenning Seth Jennings
              rhn-support-jiezhao Jie Zhao
              Jie Zhao Jie Zhao
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: