Uploaded image for project: 'Cert Manager support for Red Hat OpenShift'
  1. Cert Manager support for Red Hat OpenShift
  2. CM-735

[Istio-csr] CR object intermittently not proceeded after creation: empty status and missing deployment

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • None
    • OAPE Sprint 278
    • 1

      Sometimes (not always) IstioCSR CR object won't be proceeded after creation, its status remains empty and the underlying deployment won't be created.

      Observations:

      • the IstioCSR object `.status` is empty.
      • the cert-manager-istio-csr deployment won't show-up for a long time
      • in the operator log, it shows as below. Seems the object wasn't be detected correctly so the reconciling "skipped":
      I1008 13:24:53.403180       1 controller.go:302] "Reconciling" logger="operator-manager" controller="cert-manager-istio-csr-controller" controllerGroup="operator.openshift.io" controllerKind="IstioCSR" IstioCSR="istio-system/default" namespace="istio-system" name="default" reconcileID="6bfa7336-ecf5-451e-9859-ebfa2b229691"
I1008 13:24:53.403203       1 controller.go:255] "reconciling" logger="cert-manager-istio-csr-controller" request="istio-system/default"
I1008 13:24:53.403261       1 controller.go:264] "istiocsr.openshift.operator.io object not found, skipping reconciliation" logger="cert-manager-istio-csr-controller" request="istio-system/default"
I1008 13:24:53.403281       1 controller.go:331] "Reconcile successful" logger="operator-manager" controller="cert-manager-istio-csr-controller" controllerGroup="operator.openshift.io" controllerKind="IstioCSR" IstioCSR="istio-system/default" namespace="istio-system" name="default" reconcileID="6bfa7336-ecf5-451e-9859-ebfa2b229691"
      • Tried to manually force delete the operator pod, then everything would be back to normal in seconds. (deployment created and ready, CR object has ready status)

      Example logs

        1. REPRODUCED_RACE_CONDITION_ANALYSIS.md
          31 kB
          Mytreya Kasturi
        2. o3
          1.29 MB
          Mytreya Kasturi
        3. log_changes
          21 kB
          Mytreya Kasturi
        4. app.yaml
          0.5 kB
          Mytreya Kasturi
        5. permissions.yaml
          0.8 kB
          Mytreya Kasturi
        6. test_race.sh
          1 kB
          Mytreya Kasturi
        7. o
          56 kB
          Mytreya Kasturi

              rh-ee-yuewu Yuedong Wu
              rh-ee-yuewu Yuedong Wu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: