Uploaded image for project: 'Distributed Tracing'
  1. Distributed Tracing
  2. TRACING-3339

Fix Jaeger failing tests on Prow CI

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Undefined
    • None
    • None
    • Jaeger
    • None
    • 1
    • False
    • None
    • False
    • Tracing Sprint # 239, Tracing Sprint # 240

    Description

      The following test cases are failing on Prow CI, which runs the test cases from a pod.

      elasticsearch_es_rollover_autoprov         failed
      elasticsearch_es_index_cleaner_autoprov     failed
      miscellaneous_collector_autoscale             failed
      streaming_streaming_with_autoprovisioning_autoscale failed

      Logs for the test run:

      https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift_release/40442/rehearse-40442-periodic-ci-jaegertracing-jaeger-operator-main-jaeger-product-ocp-14-lp-interop-jaeger-interop-aws/1675778951244419072/artifacts/jaeger-interop-aws/distributed-tracing-tests-jaeger/build-log.txt

      How to test this out from a OpenShift pod to simulate the Prow CI run:

      *Create the test runner image from the jaeger-operator repo.

      docker build -t quay.io/rhn_support_ikanse/jaeger-tests-runner:latest -f tests/Dockerfile

      *Build the asserts-e2e image which is used for Jaeger tests.

      docker build -t quay.io/rhn_support_ikanse/asserts-e2e:latest -f Dockerfile.asserts .

      *Run a deployment using the image tests runner image.
      *Make sure that you install any dependency operators like OpenTelemetry, Elasticsearch, AMQ streams for running the tests. 

      *oc rsh in the running pod and run the steps from this script to execute the e2e tests.

      export KUBECONFIG=/path/to/a/kubeconfig/for our test cluster, you can use the same cluster on which the pod is running as a test cluster. 

      Make sure to specify the ASSERT_IMG and TIMEOUT vars. Set the ARTIFACT_DIR var to any writable location like /tmp/artifacts. 

      #!/bin/bash
set -o errexit
set -o pipefail
# Unset environment variables which conflict with kuttl
unset NAMESPACE
# Copy the jaeger-operator repo files to a writable directory by kuttl
cp -R /tmp/jaeger-operator /tmp/jaeger-tests && cd /tmp/jaeger-tests
# Replace with patched files for running Jaeger tests on Prow CI
curl -o tests/e2e/Makefile -L https://raw.githubusercontent.com/IshwarKanse/test-files/main/jaeger-operator/tests/e2e/Makefile
curl -o hack/run-e2e-test-suite.sh -L https://raw.githubusercontent.com/IshwarKanse/test-files/main/jaeger-operator/hack/run-e2e-test-suite.sh
curl -o hack/install/install-kuttl.sh -L https://raw.githubusercontent.com/IshwarKanse/test-files/main/jaeger-operator/hack/install/install-kuttl.sh
./hack/install/install-kuttl.sh
# Run the e2e tests
make run-e2e-tests ASSERT_IMG=$ASSERT_IMG VERBOSE=true USE_KIND_CLUSTER=false SKIP_ES_EXTERNAL=true JAEGER_OLM=true KAFKA_OLM=true PROMETHEUS_OLM=true CI=true PIPELINE=true E2E_TESTS_TIMEOUT=$TIMEOUT

      Link to the script:
      https://github.com/openshift/release/pull/40442/files#diff-88ee480b9dbc678aac9308cecbfb88cbd258db98533104823a34ed24a91f4196

       

       

       

      Attachments

        Activity

          People

            rvargasp@redhat.com Ruben Vargas Palma
            rhn-support-ikanse Ishwar Kanse
            Ishwar Kanse Ishwar Kanse
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: