Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-2604

Increase ControlPlane disk IOPS on GCP

XMLWordPrintable

    • None
    • Proposed
    • False
    • Hide

      None

      Show
      None

      Background:

      https://issues.redhat.com/browse/OCPBUGS-1130

      https://github.com/openshift/release/pull/33248

       

      Description of problem:

      In the CI we figured that etcd is having high latencies on WAL appends, this triggered a wide range of alerts. 

We chased this down to slow disks where we suspect the sequential IOPS to not be enough anymore. In order for us to increase IOPS we can increase the size.

      Version-Release number of selected component (if applicable):

      We suspect this to be > 4.8, but with 4.11/4.12 it's the most noticeable:
https://testgrid.k8s.io/redhat-openshift-ocp-release-4.12-informing#periodic-ci-openshift-release-master-ci-4.12-upgrade-from-stable-4.11-e2e-gcp-sdn-upgrade&width=20

      How reproducible:

      About every second of the above jobs are failing due to that.

      Steps to Reproduce:

      1.
2.
3.

      Actual results:

       

      Expected results:

      etcd not tripping over high disk latency anymore.

      Additional info:

       

              Unassigned Unassigned
              tjungblu@redhat.com Thomas Jungblut
              Gaoyun Pei Gaoyun Pei
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: