Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-22428

Test failure in Libvirt CI runs- alert/etcdHighCommitDurations was at or above info

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Undefined Undefined
    • None
    • 4.13.z, 4.12.z, 4.11.z, 4.14.z, 4.15
    • Etcd
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • No
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Observing etcd alert issue on 4.11 to 4.13 libvirt runs (This doesn't have etcd mounted on ramdisk). 4.14 and 4.15 jobs has etcd on ramdisk and this issue is not observed in those

      { etcdHighCommitDurations was at or above info for at least 23m54s on platformidentification.JobType
      {Release:"4.13", FromRelease:"", Platform:"libvirt", Architecture:"ppc64le", Network:"ovn", Topology:"ha"}
      (maxAllowed=0s): pending for 0s, firing for 23m54s:
      Aug 25 01:18:26.990 - 568s W alert/etcdHighCommitDurations node/192.168.126.11:9979 ns/openshift-etcd pod/etcd-libvirt-ppc64le-1-0-4-8k97q-master-0 ALERTS
      {alertname="etcdHighCommitDurations", alertstate="firing", endpoint="etcd-metrics", instance="192.168.126.11:9979", job="etcd", namespace="openshift-etcd", pod="etcd-libvirt-ppc64le-1-0-4-8k97q-master-0", prometheus="openshift-monitoring/k8s", service="etcd", severity="warning"}
      Aug 25 01:18:26.990 - 598s W alert/etcdHighCommitDurations node/192.168.126.12:9979 ns/openshift-etcd pod/etcd-libvirt-ppc64le-1-0-4-8k97q-master-1 ALERTS
      {alertname="etcdHighCommitDurations", alertstate="firing", endpoint="etcd-metrics", instance="192.168.126.12:9979", job="etcd", namespace="openshift-etcd", pod="etcd-libvirt-ppc64le-1-0-4-8k97q-master-1", prometheus="openshift-monitoring/k8s", service="etcd", severity="warning"}
      Aug 25 01:18:56.990 - 268s W alert/etcdHighCommitDurations node/192.168.126.13:9979 ns/openshift-etcd pod/etcd-libvirt-ppc64le-1-0-4-8k97q-master-2 ALERTS{alertname="etcdHighCommitDurations", alertstate="firing", endpoint="etcd-metrics", instance="192.168.126.13:9979", job="etcd", namespace="openshift-etcd", pod="etcd-libvirt-ppc64le-1-0-4-8k97q-master-2", prometheus="openshift-monitoring/k8s", service="etcd", severity="warning"}}
      

      Initial debugging is added as part of the following defect- https://issues.redhat.com/browse/OCPBUGS-18104 

              kviswana Kishen V (Inactive)
              shgokul Shilpa Gokul
              None
              None
              Ge Liu Ge Liu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: