Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-305

Investigate: [bz-etcd][invariant] alert/etcdHighNumberOfLeaderChanges should not be at or above info

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Minor Minor
    • None
    • None
    • False
    • None
    • False

      A handful of failures on aws lately, very rare, but they do indicate that the logic we currently have in the test can trip when we don't really care.

      https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-aws-upgrade/1534289681716350976

      https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-aws-upgrade-workload/1533156263876104192

      https://prow.ci.openshift.org/view/gcs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.11-upgrade-from-stable-4.10-e2e-aws-upgrade/1531959480453959680

      We allow 10min * num revisions today, and most of the half dozen fails this month are within 5s - 1 minute. One was 25 minutes.

      Extending to 15min per revision should handle these outliers and preserve our ability to detect massive changes.

            rhn-engineering-dgoodwin Devan Goodwin
            rhn-engineering-dgoodwin Devan Goodwin
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: