Uploaded image for project: 'Agent-based deployment for OpenShift Installer'
  1. Agent-based deployment for OpenShift Installer
  2. AGENT-678

Concurrency issue in agent integration tests

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Sprint 239

      The agent integration tests is failing with different errors when run multiple times locally:

      Local Run 1:

       

      level=fatal msg=failed to fetch Agent Installer PXE Files: failed to fetch dependency of "Agent Installer PXE Files": failed to generate asset "Agent Installer Artifacts": lstat /home/rwsu/.cache/agent/files_cache/libnmstate.so.2: no such file or directory
      [exit status 1]
      FAIL: testdata/agent/pxe/configurations/sno.txt:3: unexpected command failure

       

      Local Run 2:

       

      level=fatal msg=failed to fetch Agent Installer PXE Files: failed to fetch dependency of "Agent Installer PXE Files": failed to generate asset "Agent Installer Artifacts": file /usr/bin/agent-tui was not found
      [exit status 1]
      FAIL: testdata/agent/pxe/configurations/sno.txt:3: unexpected command failure
      

       

      In the [CI|https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_installer/7299/pull-ci-openshift-installer-master-agent-integration-tests/1677347591739674624,] it has failed in this PR multiple times with this error: 

      level=fatal msg=failed to fetch Agent Installer PXE Files: failed to fetch dependency of "Agent Installer PXE Files": failed to generate asset "Agent Installer Artifacts": lstat /.cache/agent/files_cache/agent-tui: no such file or directory   32  [exit status 1]   33  FAIL: testdata/agent/pxe/configurations/sno.txt:3: unexpected command failure 

      I believe the issue is the integration tests are running in parallel, and the extractFileFromImage function in pkg/asset/agent/image/oc.go problematic because the cache is being cleared and then files extracted to the same path. When the tests run in parallel, another test could clear the cached files and when the current test tries to read the file from the cached directory, it has disappeared.

      Adding 

      -parallel 1

      to ./hack/go-integration-test.sh eliminates the errors, so that why I think it is an concurrency issue.
       

            rwsu1@redhat.com Richard Su
            rwsu1@redhat.com Richard Su
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: