Uploaded image for project: 'MicroShift'
  1. MicroShift
  2. USHIFT-2796

Periodics::CNCF test can take more than current CI step timeout (2h)

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 2
    • None
    • None
    • uShift Sprint 251
    • None
    • None
    • None

      Found in job: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-microshift-main-ocp-metal-nightly/1772609594904809472 

       

      + 14:06:19.516929849 microshift/test/bin/ci_phase_test.sh:32 	parallel --results '/home/ec2-user/microshift/_output/test-images/scenario-info/{/.}/run.log' --joblog /home/ec2-user/microshift/_output/test-images/test_jobs.txt bash -x ./bin/scenario.sh run ::: /home/ec2-user/microshift/_output/scenarios-to-run/cos9-src@rpm-standard1.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-crel@upgrade-fails.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-prel@upgrade-fails-and-rolls-back.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@fips.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@greenboot.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@isolated-net.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@manual-rollback.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@offline.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@optional.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@osconfig.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@rpm-standard1.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@rpm-standard2.sh /home/ec2-user/microshift/_output/scenarios-to-run/el92-src@upgrade-fails-cannot-backup.sh /home/ec2-user/microshift/_output/scenarios-to-run/el93-crel@upgrade-to-el93-src.sh /home/ec2-user/microshift/_output/scenarios-to-run/el93-src@standard-suite1.sh /home/ec2-user/microshift/_output/scenarios-to-run/el93-src@standard-suite2.sh /home/ec2-user/microshift/_output/scenarios-to-run/el93-src@upgrade-from-el92-prel.sh /home/ec2-user/microshift/_output/scenarios-to-run/el93-src@upgrade-from-el92-src.sh
      
      {"component":"entrypoint","file":"k8s.io/test-infra/prow/entrypoint/run.go:169","func":"k8s.io/test-infra/prow/entrypoint.Options.ExecuteProcess","level":"error","msg":"Process did not finish before 2h0m0s timeout","severity":"error","time":"2024-03-26T16:06:03Z"}
      
      ++ 16:06:03.888660980 	finalize
      ++ 16:06:03.890694843 	scp -r ec2-user@35.88.108.3:/home/ec2-user/microshift/_output/test-images/scenario-info /logs/artifacts304 

      CNCF test log: https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/logs/periodic-ci-openshift-microshift-main-ocp-metal-nightly/1772609594904809472/artifacts/ocp-metal-nightly/openshift-microshift-e2e-metal-tests/artifacts/scenario-info/el92-src@cncf-conformance/run.log

      +++ 14:06:19.757252364 ./bin/scenario.sh:13 	dirname ./bin/scenario.sh
      ...
      ...
      ...
      + 16:06:39.497848279 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:99 	true
      + 16:06:39.499105062 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:100 	/home/ec2-user/go/bin/sonobuoy status --json
      ++ 16:06:39.542648940 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:101 	jq -r .status /home/ec2-user/microshift/_output/test-images/scenario-info/el92-src@cncf-conformance/cncf_status.json
      + 16:06:39.566979136 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:101 	'[' running '!=' running ']'
      ++ 16:06:39.568817307 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:105 	date +%s
      + 16:06:39.570993862 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:105 	now=1711469199
      + 16:06:39.572216722 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:106 	'[' 6972 -ge 7400 ']'
      + 16:06:39.573521990 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:113 	jq '.plugins[] | select(.plugin=="e2e") | .["result-counts"], .progress' /home/ec2-user/microshift/_output/test-images/scenario-info/el92-src@cncf-conformance/cncf_status.json
      null
      {
        "name": "e2e",
        "node": "global",
        "timestamp": "2024-03-26T14:10:40.513088195Z",
        "msg": "",
        "total": 388,
        "completed": 0
      }
      + 16:06:39.597585300 microshift/_output/scenarios-to-run/el92-src@cncf-conformance.sh:114 	sleep 60 

       

      Proposed solution: increase timeout of "metal-tests" step to 2h30m

              pmatusza@redhat.com Patryk Matuszak
              pmatusza@redhat.com Patryk Matuszak
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: