Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-22428

Test failure in Libvirt CI runs- alert/etcdHighCommitDurations was at or above info

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Undefined Undefined
    • None
    • 4.13.z, 4.12.z, 4.11.z, 4.14.z, 4.15
    • Etcd
    • None
    • No
    • False
    • Hide

      None

      Show
      None

      Observing etcd alert issue on 4.11 to 4.13 libvirt runs (This doesn't have etcd mounted on ramdisk). 4.14 and 4.15 jobs has etcd on ramdisk and this issue is not observed in those

      { etcdHighCommitDurations was at or above info for at least 23m54s on platformidentification.JobType
      {Release:"4.13", FromRelease:"", Platform:"libvirt", Architecture:"ppc64le", Network:"ovn", Topology:"ha"}
      (maxAllowed=0s): pending for 0s, firing for 23m54s:
      Aug 25 01:18:26.990 - 568s W alert/etcdHighCommitDurations node/192.168.126.11:9979 ns/openshift-etcd pod/etcd-libvirt-ppc64le-1-0-4-8k97q-master-0 ALERTS
      {alertname="etcdHighCommitDurations", alertstate="firing", endpoint="etcd-metrics", instance="192.168.126.11:9979", job="etcd", namespace="openshift-etcd", pod="etcd-libvirt-ppc64le-1-0-4-8k97q-master-0", prometheus="openshift-monitoring/k8s", service="etcd", severity="warning"}
      Aug 25 01:18:26.990 - 598s W alert/etcdHighCommitDurations node/192.168.126.12:9979 ns/openshift-etcd pod/etcd-libvirt-ppc64le-1-0-4-8k97q-master-1 ALERTS
      {alertname="etcdHighCommitDurations", alertstate="firing", endpoint="etcd-metrics", instance="192.168.126.12:9979", job="etcd", namespace="openshift-etcd", pod="etcd-libvirt-ppc64le-1-0-4-8k97q-master-1", prometheus="openshift-monitoring/k8s", service="etcd", severity="warning"}
      Aug 25 01:18:56.990 - 268s W alert/etcdHighCommitDurations node/192.168.126.13:9979 ns/openshift-etcd pod/etcd-libvirt-ppc64le-1-0-4-8k97q-master-2 ALERTS{alertname="etcdHighCommitDurations", alertstate="firing", endpoint="etcd-metrics", instance="192.168.126.13:9979", job="etcd", namespace="openshift-etcd", pod="etcd-libvirt-ppc64le-1-0-4-8k97q-master-2", prometheus="openshift-monitoring/k8s", service="etcd", severity="warning"}}
      

      Initial debugging is added as part of the following defect- https://issues.redhat.com/browse/OCPBUGS-18104 

            kviswana Kishen V
            shgokul Shilpa Gokul
            ge liu ge liu
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: