Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-282

watcher: metal failing on insights kube-rbac-proxy alerts

    • Icon: Story Story
    • Resolution: Done
    • Icon: Blocker Blocker
    • None
    • None
    • False
    • None
    • False

      I believe https://github.com/openshift/insights-operator/pull/607/ has broken payloads. Specifically the image reference to kube-rbac-proxy: it's being pulled from quay instead of the release payload! I believe it shopuld be registry.ci.openshift.org/openshift:kube-rbac-proxy and added to the image-references file.

       

      e2e-metal-ipi-ovn-ipv6 started failing in https://amd64.ocp.releases.ci.openshift.org/releasestream/4.11.0-0.nightly/release/4.11.0-0.nightly-2022-06-02-202024 blocking payloads on `[sig-instrumentation][Late] Alerts shouldn't report any unexpected alerts in firing or pending state`

      alert KubeContainerWaiting fired for 2955 seconds with labels: {container="kube-rbac-proxy", namespace="openshift-insights", pod="insights-operator-94b79c48c-m8gmq", severity="warning"}
      alert KubeDeploymentReplicasMismatch fired for 4294 seconds with labels: {container="kube-rbac-proxy-main", deployment="insights-operator", endpoint="https-main", job="kube-state-metrics", namespace="openshift-insights", service="kube-state-metrics", severity="warning"}
      alert TargetDown fired for 4294 seconds with labels: {job="metrics", namespace="openshift-insights", service="metrics", severity="warning"}}
      

      Example run: https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/27203/pull-ci-openshift-origin-master-e2e-metal-ipi-ovn-ipv6/1532532468387680256

      CI search shows this is a new problem:

       

              rh-ee-fbabcock Forrest Babcock
              stbenjam Stephen Benjamin
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: