Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-39241

Increase Event Count of RequiredInstallerResourcesMissing on SNO Upgrades

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • 4.17.0, 4.18.0
    • kube-apiserver
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Critical
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Failed Test: 

      [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets
      

      Description of problem:

      We have noticed an increase in events for missing installer resources, within a span of a few seconds we'll see a high count of events being fired off. This is tripping the test failure [bz-etcd] pathological event should not see excessive RequiredInstallerResourcesMissing secrets

      Ex Event:

      {
        "metadata": {
          "name": "kube-apiserver-operator.17f013a36534933c",
          "namespace": "openshift-kube-apiserver-operator",
          "uid": "7783601f-9e49-42f9-a1ee-d308316469e1",
          "resourceVersion": "23677",
          "creationTimestamp": "2024-08-29T02:57:29Z"
        },
        "reason": "RequiredInstallerResourcesMissing",
        "message": "configmaps: kube-apiserver-audit-policies-8,sa-token-signing-certs-8, secrets: etcd-client-8,localhost-recovery-client-token-8,localhost-recovery-serving-certkey-8",
        "source": {
          "component": "kube-apiserver-operator-installer-controller"
        },
        "firstTimestamp": "2024-08-29T02:57:29Z",
        "lastTimestamp": "2024-08-29T02:57:37Z",
        "count": 40,
        "type": "Warning",
        "eventTime": null,
        "reportingComponent": "kube-apiserver-operator-installer-controller",
        "reportingInstance": "",
        "involvedObject": {
          "kind": "Deployment",
          "namespace": "openshift-kube-apiserver-operator",
          "name": "kube-apiserver-operator",
          "uid": "50a10eb3-a5ea-4993-9322-1fc9d99e1639",
          "apiVersion": "apps/v1",
          "labels": {
            "app": "kube-apiserver-operator"
          },
          "annotations": {
            "exclude.release.openshift.io/internal-openshift-hosted": "true",
            "include.release.openshift.io/self-managed-high-availability": "true",
            "include.release.openshift.io/single-node-developer": "true"
          }
        }
      }
      

      Version-Release number of selected component (if applicable):

          4.18

      How reproducible:

          This will happen on occasion on SNO upgrades

      Additional info:

      Example Job: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.18-upgrade-from-stable-4.17-e2e-aws-upgrade-ovn-single-node/1828978182104027136

      Loki Info:

      {invoker="openshift-internal-ci/periodic-ci-openshift-release-master-nightly-4.18-upgrade-from-stable-4.17-e2e-aws-upgrade-ovn-single-node/1828978182104027136"} | unpack | namespace="openshift-kube-apiserver-operator" |~ "missing required resources:"
      
      {invoker="openshift-internal-ci/periodic-ci-openshift-release-master-nightly-4.18-upgrade-from-stable-4.17-e2e-aws-upgrade-ovn-single-node/1828978182104027136"} | unpack | namespace="openshift-kube-apiserver-operator" and container="event-exporter" | json | reason="RequiredInstallerResourcesMissing"
      

      https://grafana-loki.ci.openshift.org/explore?orgId=1&left=%7B%22datasource%22:%22PCEB727DF2F34084E%22,%22queries%22:%5B%7B%22expr%22:%22%7Binvoker%3D%5C%22openshift-internal-ci%2Fperiodic-ci-openshift-release-master-nightly-4.18-upgrade-from-stable-4.17-e2e-aws-upgrade-ovn-single-node%2F1828978182104027136%5C%22%7D%20%7C%20unpack%20%7C%20namespace%3D%5C%22openshift-kube-apiserver-operator%5C%22%20%7C~%20%5C%22missing%20required%20resources:%5C%22%22,%22refId%22:%22A%22,%22editorMode%22:%22code%22,%22queryType%22:%22range%22,%22hide%22:false%7D,%7B%22refId%22:%22B%22,%22datasource%22:%7B%22type%22:%22loki%22,%22uid%22:%22PCEB727DF2F34084E%22%7D,%22editorMode%22:%22code%22,%22expr%22:%22%7Binvoker%3D%5C%22openshift-internal-ci%2Fperiodic-ci-openshift-release-master-nightly-4.18-upgrade-from-stable-4.17-e2e-aws-upgrade-ovn-single-node%2F1828978182104027136%5C%22%7D%20%7C%20unpack%20%7C%20namespace%3D%5C%22openshift-kube-apiserver-operator%5C%22%20and%20container%3D%5C%22event-exporter%5C%22%20%7C%20json%20%7C%20reason%3D%5C%22RequiredInstallerResourcesMissing%5C%22%22,%22queryType%22:%22range%22,%22hide%22:true%7D%5D,%22range%22:%7B%22from%22:%221724890711711%22,%22to%22:%221724926711713%22%7D%7D

              rh-ee-saldawam Shaza Aldawamneh
              ehila@redhat.com Egli Hila
              None
              None
              Ke Wang Ke Wang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: