Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-214

Investigate how openshift-tests can fail a test but return success

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Minor Minor
    • None
    • None
    • False
    • None
    • False

      We were testing with forcing a failure in upgrade.go to catch a condition where a test didn't run due to a previous failure

       
      err != nil {
      //before returning the err force a failure in a test scenario
      disruption.RecordJUnit(f, "[sig-cluster-lifecycle] intentional test failure", func() (error, bool)

      { framework.Logf("Validating failure occurs with err") return fmt.Errorf("Validating failure occurs %s: %s", "upgrade", "forced failure (err)"), false }

      )
      recordClusterEvent(kubeClient, uid, "Upgrade", "UpgradeFailed", fmt.Sprintf("failed to acknowledge version: %v", err), true)
      return err
      } else {
      //force the failure here as well just to make sure we get our signal
      disruption.RecordJUnit(f, "[sig-cluster-lifecycle] intentional test failure", func() (error, bool)

      { framework.Logf("Validating failure occurs without err") return fmt.Errorf("Validating failure occurs %s: %s", "upgrade", "forced failure (no err)"), false }

      )
      }
       

      The else condition above fired and the test failed. However the job succeeded. Question is how did the job succeed with a failed test.
       

      https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/27059/pull-ci-openshift-origin-master-e2e-gcp-upgrade/1519062970204688384

            rh-ee-fbabcock Forrest Babcock
            rhn-engineering-dgoodwin Devan Goodwin
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: