Uploaded image for project: 'Cert Manager support for Red Hat OpenShift'
  1. Cert Manager support for Red Hat OpenShift
  2. CM-546

[Istio-csr] IstioCSR object's "type: Ready" condition is missing occasionally

XMLWordPrintable

    • 3
    • False
    • Hide

      None

      Show
      None
    • False
    • OAPE Sprint 271, OAPE Sprint 272, OAPE Sprint 273
    • 3
    • Important

      In IstioCSR feature e2e tests, a timeout error is observed sporadically at: https://github.com/openshift/cert-manager-operator/blob/0431e1878087c139b41051e8add803e06ec72efe/test/e2e/istio_csr_test.go#L118

      One major situation is that the IstioCSR object's "type: Ready" condition is missing, although the deployment was successfully created and became ready.

      $ oc describe istiocsr/default -n istio-system
      Name:         default
      Namespace:    istio-system
      Labels:       <none>
      Annotations:  operator.openshift.io/istio-csr-processed: true
      API Version:  operator.openshift.io/v1alpha1
      Kind:         IstioCSR
      Metadata:
        Creation Timestamp:  2025-03-18T08:59:59Z
        Finalizers:
          istiocsr.openshift.operator.io/cert-manager-istio-csr-controller
        Generation:        2
        Resource Version:  166977
        UID:               0a13c3bb-f3ca-4271-b8f9-a9a88e40dd39
      Spec:
        Istio CSR Config:
          Cert Manager:
            Issuer Ref:
              Group:  cert-manager.io
              Kind:   Issuer
              Name:   istio-ca
          Istio:
            Namespace:  istio-system
            Revisions:
              default
          Istiod TLS Config:
            Certificate Duration:      1h0m0s
            Certificate Renew Before:  30m0s
            Max Certificate Duration:  1h0m0s
            Private Key Size:          2048
            Signature Algorithm:       RSA
            Trust Domain:              cluster.local
          Log Format:                  text
          Log Level:                   1
          Resources:
      Status:
        Cluster Role:          cert-manager-istio-csr-kfqj7
        Cluster Role Binding:  cert-manager-istio-csr-pg65j
        Conditions:
          Last Transition Time:  2025-03-18T09:00:00Z
          Message:               
          Reason:                Ready
          Status:                False
          Type:                  Degraded
        Istio CSRGRPC Endpoint:  cert-manager-istio-csr.istio-system.svc:443
        Istio CSR Image:         registry.redhat.io/cert-manager/cert-manager-istio-csr-rhel9@sha256:9573d74bd2b926ec94af76f813e6358f14c5b2f4e0eedab7c1ff1070b7279a5c
        Service Account:         cert-manager-istio-csr
      Events:
        Type     Reason                 Age    From                               Message
        ----     ------                 ----   ----                               -------
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  service resource istio-system/cert-manager-istio-csr created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  serviceaccount resource istio-system/cert-manager-istio-csr created
        Warning  ResourceAlreadyExists  2m56s  cert-manager-istio-csr-controller  /cert-manager-istio-csr-kfqj7 clusterrole resource already exists, maybe from previous installation
        Warning  ResourceAlreadyExists  2m56s  cert-manager-istio-csr-controller  /cert-manager-istio-csr-pg65j clusterrolebinding resource already exists, maybe from previous installation
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  role resource istio-system/cert-manager-istio-csr created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  rolebinding resource istio-system/cert-manager-istio-csr created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  role resource istio-system/cert-manager-istio-csr-leases created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  rolebinding resource istio-system/cert-manager-istio-csr-leases created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  certificate resource istio-system/istiod created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  configmap resource istio-system/cert-manager-istio-csr-issuer-ca-copy created
        Normal   Reconciled             2m56s  cert-manager-istio-csr-controller  deployment resource istio-system/cert-manager-istio-csr created
      
      $ oc get deploy -n istio-system
      NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
      cert-manager-istio-csr   1/1     1            1           2m23s

      Example test run: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_cert[…]rator-master-e2e-operator-tech-preview/1902710448948514816

      Operator log (level 6): https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshi[…]ager-7d877f5bb8-l6qns_cert-manager-operator.log (When searching for "updated updated current istio-csr", the most recent batch corresponds to the failed attempt. ~15:08:07)

      How to reproduce: https://github.com/openshift/cert-manager-operator/pull/241/commits/658505b2f268a49673aded568353620ee5c9a486

      Slack discussion: https://redhat-internal.slack.com/archives/C045Y7FL3A6/p1742469512499459?thread_ts=1741748686.420949&cid=C045Y7FL3A6

       

              rh-ee-yuewu Yuedong Wu
              rh-ee-yuewu Yuedong Wu
              Manish Pillai, Mytreya Kasturi, Yuedong Wu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: