exec: [go test -json -tags=integ -timeout 180m ./tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2157291 2025-12-04T10:00:53.231660Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:00:53.231736Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:00:53.262838Z info tf === Test Framework Settings === 2025-12-04T10:00:53.262950Z info tf TestID: telemetry_tracing_otelcollector RunID: ae739202-d1a0-4f61-b748-127e2e6b83a7 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:00:53.262958Z info tf =============================== 2025-12-04T10:00:53.286318Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:00:53.301004Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:00:53.301040Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:00:53.301072Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:00:53.301085Z info tf === BEGIN: Building clusters === 2025-12-04T10:00:53.305008Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:00:53.305020Z info tf === DONE: Building clusters === 2025-12-04T10:00:53.305042Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:00:53.336185Z info tf === SUCCESS: context setup function === 2025-12-04T10:00:53.336240Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:00:53.336249Z info tf === Istio Component Config === 2025-12-04T10:00:53.336304Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:00:53.336312Z info tf ================================ 2025-12-04T10:00:53.342591Z info tf skipping deployment as specified in the config 2025-12-04T10:00:53.342674Z info tf === SUCCEEDED: Deploy Istio in 6.394858ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:00:53.342694Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:00:53.587661Z info tf === SUCCEEDED: Create namespace echo in 244.958044ms === 2025-12-04T10:00:53.677177Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:00:53.962914Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:00:53.970642Z info sync complete name=pod controller attempt=3 time=7.63276ms 2025-12-04T10:00:53.987358Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:00:53.994367Z info sync complete name=pod controller attempt=3 time=6.949041ms 2025-12-04T10:00:54.063919Z info klog Caches are synced controller=echo 2025-12-04T10:00:54.087512Z info klog Caches are synced controller=echo 2025-12-04T10:01:03.593250Z info tf === SUCCEEDED: Deploy echo instances in 9.916058907s === 2025-12-04T10:01:04.765926Z info tf Checking pods ready... 2025-12-04T10:01:04.765977Z info tf Checking pods ready... 2025-12-04T10:01:04.807911Z info tf [ 0] zipkin-74cc7d9448-hm77k Pending (Pending) 2025-12-04T10:01:05.008783Z info tf Checking pods ready... 2025-12-04T10:01:05.008825Z info tf Checking pods ready... 2025-12-04T10:01:05.018003Z info tf [ 0] zipkin-74cc7d9448-hm77k Pending (Pending) 2025-12-04T10:01:05.418154Z info tf Checking pods ready... 2025-12-04T10:01:05.418209Z info tf Checking pods ready... 2025-12-04T10:01:05.425225Z info tf [ 0] zipkin-74cc7d9448-hm77k Pending (Pending) 2025-12-04T10:01:06.226159Z info tf Checking pods ready... 2025-12-04T10:01:06.226227Z info tf Checking pods ready... 2025-12-04T10:01:06.232098Z info tf [ 0] zipkin-74cc7d9448-hm77k Pending (Pending) 2025-12-04T10:01:07.832541Z info tf Checking pods ready... 2025-12-04T10:01:07.832587Z info tf Checking pods ready... 2025-12-04T10:01:07.837131Z info tf [ 0] zipkin-74cc7d9448-hm77k Pending (Pending) 2025-12-04T10:01:11.037806Z info tf Checking pods ready... 2025-12-04T10:01:11.037876Z info tf Checking pods ready... 2025-12-04T10:01:11.044812Z info tf [ 0] zipkin-74cc7d9448-hm77k Pending (Pending) 2025-12-04T10:01:14.245288Z info tf Checking pods ready... 2025-12-04T10:01:14.245315Z info tf Checking pods ready... 2025-12-04T10:01:14.249898Z info tf [ 0] zipkin-74cc7d9448-hm77k Running (container not ready: 'zipkin') 2025-12-04T10:01:17.450603Z info tf Checking pods ready... 2025-12-04T10:01:17.450639Z info tf Checking pods ready... 2025-12-04T10:01:17.454869Z info tf [ 0] zipkin-74cc7d9448-hm77k Running (container not ready: 'zipkin') 2025-12-04T10:01:20.655025Z info tf Checking pods ready... 2025-12-04T10:01:20.655067Z info tf Checking pods ready... 2025-12-04T10:01:20.660142Z info tf [ 0] zipkin-74cc7d9448-hm77k Running (Ready) 2025-12-04T10:01:20.805990Z info tf Checking pods ready... 2025-12-04T10:01:20.806028Z info tf Checking pods ready... 2025-12-04T10:01:20.824561Z info tf Failed retrieving pods: no matching pod found for selectors: [app=opentelemetry-collector] 2025-12-04T10:01:21.025277Z info tf Checking pods ready... 2025-12-04T10:01:21.025353Z info tf Checking pods ready... 2025-12-04T10:01:21.031821Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:21.432037Z info tf Checking pods ready... 2025-12-04T10:01:21.432085Z info tf Checking pods ready... 2025-12-04T10:01:21.438056Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:22.238311Z info tf Checking pods ready... 2025-12-04T10:01:22.238337Z info tf Checking pods ready... 2025-12-04T10:01:22.244883Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:23.845549Z info tf Checking pods ready... 2025-12-04T10:01:23.845609Z info tf Checking pods ready... 2025-12-04T10:01:23.849941Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:27.050794Z info tf Checking pods ready... 2025-12-04T10:01:27.050825Z info tf Checking pods ready... 2025-12-04T10:01:27.055075Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:30.255427Z info tf Checking pods ready... 2025-12-04T10:01:30.255472Z info tf Checking pods ready... 2025-12-04T10:01:30.279164Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:33.479647Z info tf Checking pods ready... 2025-12-04T10:01:33.479696Z info tf Checking pods ready... 2025-12-04T10:01:33.485430Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Pending (Pending) 2025-12-04T10:01:36.685876Z info tf Checking pods ready... 2025-12-04T10:01:36.685915Z info tf Checking pods ready... 2025-12-04T10:01:36.792916Z info tf [ 0] opentelemetry-collector-749d46955f-jqm77 Running (Ready) 2025-12-04T10:01:37.655205Z info tf === DONE: Setup: 'telemetry_tracing_otelcollector' (44.350148262s) === 2025-12-04T10:01:37.655242Z info tf === BEGIN: Test Run: 'telemetry_tracing_otelcollector' === === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0 2025-12-04T10:01:37.726526Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:04:07.727600Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0] (2m30.00104514s)' === 2025-12-04T10:04:07.727728Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0... 2025-12-04T10:04:07.727728Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0... 2025-12-04T10:04:07.911581Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/62584f5e01-fd5d-492f-b3fc-31c80fdf78dc 2025-12-04T10:04:07.911671Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1832s 2025-12-04T10:04:07.912319Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.059176Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/623aa6ccf0-378a-4865-8997-3490c0fad356 2025-12-04T10:04:08.059487Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3310s 2025-12-04T10:04:08.060026Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.157141Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/62719ce95a-7085-418a-8097-e1742ccdfbfb 2025-12-04T10:04:08.157244Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4287s 2025-12-04T10:04:08.157877Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0 (150.46s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0 (150.46s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter 2025-12-04T10:01:37.656566Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter]' === 2025-12-04T10:04:08.189389Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter] (2m30.532819684s)' === 2025-12-04T10:04:08.189487Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter... 2025-12-04T10:04:08.189520Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter... 2025-12-04T10:04:08.348071Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/620e2516ae-e053-4db1-9edf-ba9e5e55d075 2025-12-04T10:04:08.348190Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1584s 2025-12-04T10:04:08.348835Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.470669Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/62d93d7862-f958-4605-9459-26ff67b6cdd6 2025-12-04T10:04:08.470922Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2811s 2025-12-04T10:04:08.471533Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.592662Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/621a7e33f3-0749-4b48-b0e8-b2156846d4a5 --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter (150.94s) 2025-12-04T10:04:08.592778Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4030s FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter (150.94s) === RUN TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 2025-12-04T10:04:08.743784Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:06:38.744981Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0] (2m30.001194009s)' === 2025-12-04T10:06:38.745093Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:06:38.745102Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:06:38.917989Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64e5a6f993-1c36-44e0-aafc-b1f041b49109 2025-12-04T10:06:38.918288Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1726s 2025-12-04T10:06:38.919035Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.030177Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/644cf5cbbf-4653-4a95-9d2c-26795f69f680 2025-12-04T10:06:39.030310Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2846s 2025-12-04T10:06:39.031037Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.140183Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/6436107e83-cd71-492c-8ec3-dc660fc0834a 2025-12-04T10:06:39.140332Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3946s 2025-12-04T10:06:39.141217Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 (150.57s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 (150.57s) === RUN TestProxyTracingOpenTelemetryProvider/http_exporter 2025-12-04T10:04:08.593023Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter]' === 2025-12-04T10:04:08.593344Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.313585Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter] (2m30.720548881s)' === 2025-12-04T10:06:39.313779Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:06:39.313784Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:06:39.444428Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64024440a6-ff60-4eac-b393-aca73b44462a 2025-12-04T10:06:39.444887Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1308s 2025-12-04T10:06:39.445739Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.569577Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64eb32dc7f-c8b1-4fe1-8ae2-aaf44c655861 2025-12-04T10:06:39.569709Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2556s 2025-12-04T10:06:39.570604Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.687878Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64323dd572-9f25-4bfa-a1bd-d32fdd388a11 2025-12-04T10:06:39.688163Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3740s 2025-12-04T10:06:39.689113Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/http_exporter (151.37s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/http_exporter (151.37s) === RUN TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0 2025-12-04T10:06:39.987947Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:09:09.989171Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0] (2m30.001214427s)' === 2025-12-04T10:09:09.989357Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0... 2025-12-04T10:09:09.989357Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0... 2025-12-04T10:09:10.148536Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/653480cb46-22a5-4dbf-b287-3a839bb2a813 2025-12-04T10:09:10.148834Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1587s 2025-12-04T10:09:10.149795Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:10.252780Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/652d2a0416-176c-46e7-8232-51cd34a75bad 2025-12-04T10:09:10.252877Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2627s 2025-12-04T10:09:10.253942Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:10.361711Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/65163bad77-d299-4e01-b23c-cf1afe1444dc 2025-12-04T10:09:10.361920Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3717s 2025-12-04T10:09:10.362570Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0 (150.76s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0 (150.76s) === RUN TestProxyTracingOpenTelemetryProvider/resource_detectors 2025-12-04T10:06:39.965485Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors]' === 2025-12-04T10:09:10.749250Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors] (2m30.783760071s)' === 2025-12-04T10:09:10.749396Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/resource_detectors... 2025-12-04T10:09:10.749397Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/resource_detectors... 2025-12-04T10:09:10.898012Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/65919e5e69-1a3c-40cc-b9e3-503434b8b329 2025-12-04T10:09:10.898425Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1488s 2025-12-04T10:09:10.898916Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:11.023631Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/65fce1903a-6d9d-4ecc-a475-1077adb16652 2025-12-04T10:09:11.023779Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2741s 2025-12-04T10:09:11.024599Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:11.140504Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/658a14066c-1076-4eac-a95f-f5e5833de1e1 2025-12-04T10:09:11.140647Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3909s 2025-12-04T10:09:11.141481Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/resource_detectors (151.54s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/resource_detectors (151.54s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 2025-12-04T10:09:11.516561Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:11:41.516893Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0] (2m30.00032346s)' === 2025-12-04T10:11:41.517064Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:11:41.517085Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:11:41.644372Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66d5986c66-bb5e-4dfd-b67b-b1485f8545b1 2025-12-04T10:11:41.644794Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1271s 2025-12-04T10:11:41.645268Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:41.812822Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66aa59ce37-87e0-4c77-94ca-91684f8d015c 2025-12-04T10:11:41.813058Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2953s 2025-12-04T10:11:41.813895Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:41.930244Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/6603d53e05-75f5-443c-bb70-81daba53f08c 2025-12-04T10:11:41.930387Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4127s 2025-12-04T10:11:41.931281Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (151.34s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (151.34s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata 2025-12-04T10:09:11.501563Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata]' === 2025-12-04T10:11:42.851273Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata] (2m31.349702413s)' === 2025-12-04T10:11:42.851402Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:11:42.851412Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:11:43.067874Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66c9ec527a-b580-4d64-9862-fbec4b036b55 2025-12-04T10:11:43.068042Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2164s 2025-12-04T10:11:43.068888Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:43.208391Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66fa0a42eb-aa3d-476d-9b82-b615b9ec06af 2025-12-04T10:11:43.208537Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3569s 2025-12-04T10:11:43.209255Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:43.324096Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/663de96a8f-be27-47d0-97e3-fa2dd6e3badb 2025-12-04T10:11:43.324426Z info tf istioctl ([x internal-debug --all configz]): completed after 0.4727s 2025-12-04T10:11:43.325101Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (152.30s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (152.30s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0 2025-12-04T10:11:43.819137Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:14:13.819823Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0] (2m30.000677904s)' === 2025-12-04T10:14:13.819990Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0... 2025-12-04T10:14:13.820015Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0... 2025-12-04T10:14:13.966477Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:11:43Z/67e326ac71-e513-4440-9eec-49f718e10e44 2025-12-04T10:14:13.966824Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1457s 2025-12-04T10:14:13.967532Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:14:14.090253Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:11:43Z/6738441e87-b855-404a-a948-5604b6be0bf9 2025-12-04T10:14:14.090383Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2691s 2025-12-04T10:14:14.091151Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:14:14.242633Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:11:43Z/6721f8138e-0fd9-44a6-969e-fe4ac9a051fc 2025-12-04T10:14:14.242859Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4216s 2025-12-04T10:14:14.243598Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0 (151.28s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0 (151.28s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth 2025-12-04T10:11:43.805759Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth]' === 2025-12-04T10:14:15.102579Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth] (2m31.296811056s)' === --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth (151.31s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth (151.31s) === RUN TestProxyTracingOpenTelemetryProvider 2025-12-04T10:01:37.656124Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:14:15.111199Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (12m37.455070547s)' === --- FAIL: TestProxyTracingOpenTelemetryProvider (757.46s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider (757.46s) === RUN TestGatewayTracing/primary-0 2025-12-04T10:14:15.145150Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0]' === tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:162: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:16:45.145718Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0] (2m30.000561817s)' === --- FAIL: TestGatewayTracing/primary-0 (150.00s) FAIL tests/integration/telemetry/tracing/otelcollector.TestGatewayTracing/primary-0 (150.00s) === RUN TestGatewayTracing 2025-12-04T10:14:15.111849Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing]' === 2025-12-04T10:16:45.145819Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing] (2m30.033968736s)' === --- FAIL: TestGatewayTracing (150.06s) FAIL tests/integration/telemetry/tracing/otelcollector.TestGatewayTracing (150.06s) 2025-12-04T10:16:45.169838Z info tf === FAILED: Test Run: 'telemetry_tracing_otelcollector' (exitCode: 1) === 2025-12-04T10:16:45.169864Z info tf === Suite "telemetry_tracing_otelcollector" run time: 15m51.864828308s === 2025-12-04T10:16:45.169881Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/trace.yaml 2025-12-04T10:16:45.170861Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:16:45.170873Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:16:45.318094Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:16:45Z/6938adbe5d-4390-4ba7-9b91-f81ec833db0f 2025-12-04T10:16:45.318291Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1471s 2025-12-04T10:16:45.319106Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:16:45.451236Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:16:45Z/6949b652b5-23b8-4ecf-b23c-3b2cc0669692 2025-12-04T10:16:45.451433Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2802s 2025-12-04T10:16:45.452257Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:16:45.563470Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:16:45Z/69251b3798-9ecd-4028-baaf-c360a2598290 2025-12-04T10:16:45.563711Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3925s 2025-12-04T10:16:45.564389Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:16:46.877915Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:16:46.878038Z info tf === SUCCEEDED: Cleanup Istio in 114.899µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector DONE 13 tests, 13 failures in 953.682s exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^grpc_exporter$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2160854 2025-12-04T10:16:59.108258Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:16:59.108359Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:16:59.146924Z info tf === Test Framework Settings === 2025-12-04T10:16:59.147002Z info tf TestID: telemetry_tracing_otelcollector RunID: 3fdd8e09-4315-44d9-985c-26c5128e03a7 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:16:59.147007Z info tf =============================== 2025-12-04T10:16:59.147176Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:16:59.147216Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:16:59.147228Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:16:59.147245Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:16:59.147253Z info tf === BEGIN: Building clusters === 2025-12-04T10:16:59.151103Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:16:59.151113Z info tf === DONE: Building clusters === 2025-12-04T10:16:59.151132Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:16:59.353447Z info tf === SUCCESS: context setup function === 2025-12-04T10:16:59.353480Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:16:59.353492Z info tf === Istio Component Config === 2025-12-04T10:16:59.353529Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:16:59.353533Z info tf ================================ 2025-12-04T10:16:59.357663Z info tf skipping deployment as specified in the config 2025-12-04T10:16:59.357694Z info tf === SUCCEEDED: Deploy Istio in 4.205353ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:16:59.357707Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:16:59.367145Z info tf === SUCCEEDED: Create namespace echo in 9.423955ms === 2025-12-04T10:16:59.399328Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:16:59.495432Z error tf === FAILED: Deploy echo instances === 2025-12-04T10:16:59.495498Z error tf 2 errors occurred: * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.3282125833.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.2068586247.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-12-04T10:16:59.495562Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:16:59.495582Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:16:59.589335Z error tf Test setup error: 2 errors occurred: * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.3282125833.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.2068586247.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-12-04T10:16:59.589438Z info tf === FAILED: Setup: 'telemetry_tracing_otelcollector' (2 errors occurred: * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.3282125833.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.2068586247.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated ) === 2025-12-04T10:16:59.589452Z error tf Exiting due to setup failure: 2 errors occurred: * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.3282125833.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.2068586247.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-12-04T10:16:59.589495Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:16:59.589503Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:16:59.617348Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:16:59.617499Z info tf === SUCCEEDED: Cleanup Istio in 139.163µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^http_exporter$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2162186 2025-12-04T10:17:12.396612Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:17:12.396677Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:17:12.442609Z info tf === Test Framework Settings === 2025-12-04T10:17:12.442790Z info tf TestID: telemetry_tracing_otelcollector RunID: b3d785aa-3843-4079-bea3-aac6b902553a NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:17:12.442799Z info tf =============================== 2025-12-04T10:17:12.442841Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:17:12.442899Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:17:12.442919Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:17:12.442944Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:17:12.442957Z info tf === BEGIN: Building clusters === 2025-12-04T10:17:12.446974Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:17:12.446985Z info tf === DONE: Building clusters === 2025-12-04T10:17:12.447018Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:17:12.489017Z info tf === SUCCESS: context setup function === 2025-12-04T10:17:12.489074Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:17:12.489084Z info tf === Istio Component Config === 2025-12-04T10:17:12.489146Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:17:12.489152Z info tf ================================ 2025-12-04T10:17:12.492350Z info tf skipping deployment as specified in the config 2025-12-04T10:17:12.492368Z info tf === SUCCEEDED: Deploy Istio in 3.291637ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:17:12.492376Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:17:12.580343Z info tf === SUCCEEDED: Create namespace echo in 87.936212ms === 2025-12-04T10:17:12.718684Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:17:12.925482Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:17:12.933098Z info sync complete name=pod controller attempt=3 time=7.540786ms 2025-12-04T10:17:12.933313Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:17:12.935975Z info sync complete name=pod controller attempt=2 time=2.628053ms 2025-12-04T10:17:13.026722Z info klog Caches are synced controller=echo 2025-12-04T10:17:13.034297Z info klog Caches are synced controller=echo 2025-12-04T10:17:22.540967Z info tf === SUCCEEDED: Deploy echo instances in 9.822213823s === 2025-12-04T10:17:24.661432Z info tf Checking pods ready... 2025-12-04T10:17:24.661512Z info tf Checking pods ready... 2025-12-04T10:17:24.892947Z info tf [ 0] zipkin-74cc7d9448-54dw2 Pending (Pending) 2025-12-04T10:17:25.093491Z info tf Checking pods ready... 2025-12-04T10:17:25.093659Z info tf Checking pods ready... 2025-12-04T10:17:25.098669Z info tf [ 0] zipkin-74cc7d9448-54dw2 Pending (Pending) 2025-12-04T10:17:25.499514Z info tf Checking pods ready... 2025-12-04T10:17:25.499605Z info tf Checking pods ready... 2025-12-04T10:17:25.529094Z info tf [ 0] zipkin-74cc7d9448-54dw2 Pending (Pending) 2025-12-04T10:17:26.330277Z info tf Checking pods ready... 2025-12-04T10:17:26.330371Z info tf Checking pods ready... 2025-12-04T10:17:26.355397Z info tf [ 0] zipkin-74cc7d9448-54dw2 Pending (Pending) 2025-12-04T10:17:27.956133Z info tf Checking pods ready... 2025-12-04T10:17:27.956215Z info tf Checking pods ready... 2025-12-04T10:17:27.960901Z info tf [ 0] zipkin-74cc7d9448-54dw2 Running (container not ready: 'zipkin') 2025-12-04T10:17:31.161105Z info tf Checking pods ready... 2025-12-04T10:17:31.161195Z info tf Checking pods ready... 2025-12-04T10:17:31.167409Z info tf [ 0] zipkin-74cc7d9448-54dw2 Running (container not ready: 'zipkin') 2025-12-04T10:17:34.368360Z info tf Checking pods ready... 2025-12-04T10:17:34.368413Z info tf Checking pods ready... 2025-12-04T10:17:34.372894Z info tf [ 0] zipkin-74cc7d9448-54dw2 Running (container not ready: 'zipkin') 2025-12-04T10:17:37.573588Z info tf Checking pods ready... 2025-12-04T10:17:37.573660Z info tf Checking pods ready... 2025-12-04T10:17:37.577791Z info tf [ 0] zipkin-74cc7d9448-54dw2 Running (Ready) 2025-12-04T10:17:37.725144Z info tf Checking pods ready... 2025-12-04T10:17:37.725194Z info tf Checking pods ready... 2025-12-04T10:17:37.731345Z info tf Failed retrieving pods: no matching pod found for selectors: [app=opentelemetry-collector] 2025-12-04T10:17:37.931987Z info tf Checking pods ready... 2025-12-04T10:17:37.932070Z info tf Checking pods ready... 2025-12-04T10:17:37.937842Z info tf [ 0] opentelemetry-collector-749d46955f-r9z99 Pending (Pending) 2025-12-04T10:17:38.338061Z info tf Checking pods ready... 2025-12-04T10:17:38.338117Z info tf Checking pods ready... 2025-12-04T10:17:38.343443Z info tf [ 0] opentelemetry-collector-749d46955f-r9z99 Pending (Pending) 2025-12-04T10:17:39.144620Z info tf Checking pods ready... 2025-12-04T10:17:39.144693Z info tf Checking pods ready... 2025-12-04T10:17:39.151271Z info tf [ 0] opentelemetry-collector-749d46955f-r9z99 Pending (Pending) 2025-12-04T10:17:40.751763Z info tf Checking pods ready... 2025-12-04T10:17:40.751831Z info tf Checking pods ready... 2025-12-04T10:17:40.756747Z info tf [ 0] opentelemetry-collector-749d46955f-r9z99 Running (Ready) 2025-12-04T10:17:40.795664Z info tf === DONE: Setup: 'telemetry_tracing_otelcollector' (28.348625282s) === 2025-12-04T10:17:40.795705Z info tf === BEGIN: Test Run: 'telemetry_tracing_otelcollector' === === RUN TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 2025-12-04T10:17:40.807992Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:20:10.809274Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0] (2m30.001276351s)' === 2025-12-04T10:20:10.809395Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:20:10.809395Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:20:10.928638Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80cbd73392-a6dc-43c4-855a-d08a3f3caf9a 2025-12-04T10:20:10.928923Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1192s 2025-12-04T10:20:10.929416Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.055083Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80484703d2-3e47-407b-bf25-83bb76a222e9 2025-12-04T10:20:11.055323Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2455s 2025-12-04T10:20:11.056113Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.161974Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80e1ae8510-ea54-46ee-9d64-b67cc8d4e7e1 2025-12-04T10:20:11.162125Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3523s 2025-12-04T10:20:11.162704Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 (150.38s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 (re-run 1) (150.38s) === RUN TestProxyTracingOpenTelemetryProvider/http_exporter 2025-12-04T10:17:40.796096Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter]' === 2025-12-04T10:20:11.187578Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter] (2m30.391473409s)' === 2025-12-04T10:20:11.187699Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:20:11.187713Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:20:11.327274Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80151100d3-b33b-4542-8938-fc8597b87a1c 2025-12-04T10:20:11.327717Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1398s 2025-12-04T10:20:11.328267Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.455441Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/8013f148f7-45b8-4e67-832f-9513be59b99c 2025-12-04T10:20:11.455549Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2676s 2025-12-04T10:20:11.456314Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/http_exporter (150.72s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/http_exporter (re-run 1) (150.72s) === RUN TestProxyTracingOpenTelemetryProvider 2025-12-04T10:17:40.795984Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:20:11.515814Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (2m30.719817632s)' === 2025-12-04T10:20:11.515995Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:20:11.516001Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:20:11.613060Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/8096ed22d1-5472-48cf-af26-a6d75b637474 2025-12-04T10:20:11.613196Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4253s 2025-12-04T10:20:11.613855Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.735801Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81cf713575-2c5b-47dd-bd13-f2143a4ac748 2025-12-04T10:20:11.736170Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2193s 2025-12-04T10:20:11.736644Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.833507Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81998dcf9f-6a07-46c2-8be5-3095425a89ba 2025-12-04T10:20:11.833647Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3167s 2025-12-04T10:20:11.834424Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider (151.06s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider (re-run 1) (151.06s) 2025-12-04T10:20:11.857488Z info tf === FAILED: Test Run: 'telemetry_tracing_otelcollector' (exitCode: 1) === 2025-12-04T10:20:11.857587Z info tf === Suite "telemetry_tracing_otelcollector" run time: 2m59.410576412s === 2025-12-04T10:20:11.857607Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/trace.yaml 2025-12-04T10:20:11.857842Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:20:11.857858Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:20:11.987677Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/818da01944-4f5b-4c4b-87e7-6cd2cf198685 2025-12-04T10:20:11.987812Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4709s 2025-12-04T10:20:11.988558Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:12.115448Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81c68486bb-4867-4414-9f3a-1d53ab71d7a0 2025-12-04T10:20:12.115758Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2577s 2025-12-04T10:20:12.116484Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:12.210323Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81d5b8f471-6673-45b3-9ef2-8edf44fe44bd 2025-12-04T10:20:12.210511Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3524s 2025-12-04T10:20:12.211337Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:12.337837Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81e6f56402-c9be-4236-90a2-1ae68aea19a9 2025-12-04T10:20:12.338004Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4799s 2025-12-04T10:20:12.338981Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:12.826170Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:20:12.826339Z info tf === SUCCEEDED: Cleanup Istio in 154.592µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^resource_detectors$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2163885 2025-12-04T10:20:24.295753Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:20:24.295824Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:20:24.326653Z info tf === Test Framework Settings === 2025-12-04T10:20:24.326729Z info tf TestID: telemetry_tracing_otelcollector RunID: d7f618e8-0e0e-4279-9534-cd8803e1735c NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:20:24.326734Z info tf =============================== 2025-12-04T10:20:24.326763Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:20:24.326802Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:20:24.326814Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:20:24.326831Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:20:24.326839Z info tf === BEGIN: Building clusters === 2025-12-04T10:20:24.330731Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:20:24.330738Z info tf === DONE: Building clusters === 2025-12-04T10:20:24.330754Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:20:24.480942Z info tf === SUCCESS: context setup function === 2025-12-04T10:20:24.481028Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:20:24.481042Z info tf === Istio Component Config === 2025-12-04T10:20:24.481101Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:20:24.481117Z info tf ================================ 2025-12-04T10:20:24.486394Z info tf skipping deployment as specified in the config 2025-12-04T10:20:24.486423Z info tf === SUCCEEDED: Deploy Istio in 5.388524ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:20:24.486435Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:20:24.497707Z info tf === SUCCEEDED: Create namespace echo in 11.257471ms === 2025-12-04T10:20:24.532069Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:20:24.798676Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:20:24.801166Z info sync complete name=pod controller attempt=2 time=2.416679ms 2025-12-04T10:20:24.813956Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:20:24.816716Z info sync complete name=pod controller attempt=2 time=2.682485ms 2025-12-04T10:20:24.899322Z info klog Caches are synced controller=echo 2025-12-04T10:20:24.914939Z info klog Caches are synced controller=echo 2025-12-04T10:30:24.915683Z error tf === FAILED: Deploy echo instances === 2025-12-04T10:30:24.915773Z error tf 2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) 2025-12-04T10:30:24.915855Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:30:24.915856Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:30:24.933168Z error tf Test setup error: 2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) 2025-12-04T10:30:24.933246Z info tf === FAILED: Setup: 'telemetry_tracing_otelcollector' (2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) ) === 2025-12-04T10:30:24.933254Z error tf Exiting due to setup failure: 2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) 2025-12-04T10:30:24.933281Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:30:24.933303Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:30:25.034837Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:30:25.038059Z info tf === SUCCEEDED: Cleanup Istio in 3.192566ms [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^grpc_exporter_with_initial_metadata$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2166095 2025-12-04T10:30:38.185548Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:30:38.185604Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:30:38.216540Z info tf === Test Framework Settings === 2025-12-04T10:30:38.216609Z info tf TestID: telemetry_tracing_otelcollector RunID: 9ecd3ede-ee9b-44cc-bdd1-3620723301db NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:30:38.216614Z info tf =============================== 2025-12-04T10:30:38.216819Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:30:38.216857Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:30:38.216870Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:30:38.216885Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:30:38.216893Z info tf === BEGIN: Building clusters === 2025-12-04T10:30:38.220627Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:30:38.220635Z info tf === DONE: Building clusters === 2025-12-04T10:30:38.220650Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:30:38.250116Z info tf === SUCCESS: context setup function === 2025-12-04T10:30:38.250142Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:30:38.250154Z info tf === Istio Component Config === 2025-12-04T10:30:38.250214Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:30:38.250223Z info tf ================================ 2025-12-04T10:30:38.253825Z info tf skipping deployment as specified in the config 2025-12-04T10:30:38.253841Z info tf === SUCCEEDED: Deploy Istio in 3.697543ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:30:38.253849Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:30:38.348334Z info tf === SUCCEEDED: Create namespace echo in 94.466571ms === 2025-12-04T10:30:38.424228Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:30:38.639538Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:30:38.639800Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:30:38.641652Z info sync complete name=pod controller attempt=2 time=2.044927ms 2025-12-04T10:30:38.642409Z info sync complete name=pod controller attempt=2 time=2.581955ms 2025-12-04T10:30:38.740302Z info klog Caches are synced controller=echo 2025-12-04T10:30:38.740310Z info klog Caches are synced controller=echo 2025-12-04T10:30:46.645888Z info tf === SUCCEEDED: Deploy echo instances in 8.221665813s === 2025-12-04T10:30:46.805876Z info tf Checking pods ready... 2025-12-04T10:30:46.805924Z info tf Checking pods ready... 2025-12-04T10:30:46.826200Z info tf [ 0] zipkin-74cc7d9448-dpvvj Pending (Pending) 2025-12-04T10:30:47.026959Z info tf Checking pods ready... 2025-12-04T10:30:47.027101Z info tf Checking pods ready... 2025-12-04T10:30:47.032191Z info tf [ 0] zipkin-74cc7d9448-dpvvj Pending (Pending) 2025-12-04T10:30:47.432336Z info tf Checking pods ready... 2025-12-04T10:30:47.432412Z info tf Checking pods ready... 2025-12-04T10:30:47.439426Z info tf [ 0] zipkin-74cc7d9448-dpvvj Pending (Pending) 2025-12-04T10:30:48.239867Z info tf Checking pods ready... 2025-12-04T10:30:48.239916Z info tf Checking pods ready... 2025-12-04T10:30:48.245698Z info tf [ 0] zipkin-74cc7d9448-dpvvj Pending (Pending) 2025-12-04T10:30:49.846834Z info tf Checking pods ready... 2025-12-04T10:30:49.846868Z info tf Checking pods ready... 2025-12-04T10:30:49.851156Z info tf [ 0] zipkin-74cc7d9448-dpvvj Pending (Pending) 2025-12-04T10:30:53.051477Z info tf Checking pods ready... 2025-12-04T10:30:53.051542Z info tf Checking pods ready... 2025-12-04T10:30:53.056489Z info tf [ 0] zipkin-74cc7d9448-dpvvj Pending (Pending) 2025-12-04T10:30:56.256928Z info tf Checking pods ready... 2025-12-04T10:30:56.256971Z info tf Checking pods ready... 2025-12-04T10:30:56.262092Z info tf [ 0] zipkin-74cc7d9448-dpvvj Running (container not ready: 'zipkin') 2025-12-04T10:30:59.462574Z info tf Checking pods ready... 2025-12-04T10:30:59.462610Z info tf Checking pods ready... 2025-12-04T10:30:59.468391Z info tf [ 0] zipkin-74cc7d9448-dpvvj Running (container not ready: 'zipkin') 2025-12-04T10:31:02.668514Z info tf Checking pods ready... 2025-12-04T10:31:02.668561Z info tf Checking pods ready... 2025-12-04T10:31:02.682449Z info tf [ 0] zipkin-74cc7d9448-dpvvj Running (Ready) 2025-12-04T10:31:03.714811Z info tf Checking pods ready... 2025-12-04T10:31:03.714833Z info tf Checking pods ready... 2025-12-04T10:31:03.801087Z info tf Failed retrieving pods: no matching pod found for selectors: [app=opentelemetry-collector] 2025-12-04T10:31:04.001807Z info tf Checking pods ready... 2025-12-04T10:31:04.001879Z info tf Checking pods ready... 2025-12-04T10:31:04.160264Z info tf [ 0] opentelemetry-collector-749d46955f-97f9x Pending (Pending) 2025-12-04T10:31:04.560543Z info tf Checking pods ready... 2025-12-04T10:31:04.560575Z info tf Checking pods ready... 2025-12-04T10:31:04.949722Z info tf [ 0] opentelemetry-collector-749d46955f-97f9x Pending (Pending) 2025-12-04T10:31:05.750793Z info tf Checking pods ready... 2025-12-04T10:31:05.750825Z info tf Checking pods ready... 2025-12-04T10:31:05.768709Z info tf [ 0] opentelemetry-collector-749d46955f-97f9x Pending (Pending) 2025-12-04T10:31:07.368944Z info tf Checking pods ready... 2025-12-04T10:31:07.368986Z info tf Checking pods ready... 2025-12-04T10:31:08.067211Z info tf [ 0] opentelemetry-collector-749d46955f-97f9x Running (Ready) 2025-12-04T10:31:08.336666Z info tf === DONE: Setup: 'telemetry_tracing_otelcollector' (30.116002421s) === 2025-12-04T10:31:08.336715Z info tf === BEGIN: Test Run: 'telemetry_tracing_otelcollector' === === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 2025-12-04T10:31:08.371986Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:33:38.373115Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0] (2m30.001120534s)' === 2025-12-04T10:33:38.373301Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:33:38.373305Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:33:38.498878Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/959be5ca7f-8963-45cb-ad3b-e82632374b3a 2025-12-04T10:33:38.499355Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1257s 2025-12-04T10:33:38.501177Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:38.604711Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/951cc824a9-3759-4f92-beca-2739e00cd2bb 2025-12-04T10:33:38.604811Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2311s 2025-12-04T10:33:38.605585Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:38.699103Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/95f7043f7a-484b-4f10-a13c-2247a08a990d 2025-12-04T10:33:38.699530Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3258s 2025-12-04T10:33:38.699961Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (150.35s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (re-run 1) (150.35s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata 2025-12-04T10:31:08.337413Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata]' === 2025-12-04T10:33:38.725953Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata] (2m30.388533806s)' === 2025-12-04T10:33:38.726015Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:33:38.726030Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:33:38.909416Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/95ef904da0-0188-4918-a1c6-6de10d0e44af 2025-12-04T10:33:38.909763Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1835s 2025-12-04T10:33:38.910284Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.011510Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/95ebd54c82-b429-4e66-a4e2-add216dc570f 2025-12-04T10:33:39.011722Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2854s 2025-12-04T10:33:39.012524Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (150.73s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (re-run 1) (150.73s) === RUN TestProxyTracingOpenTelemetryProvider 2025-12-04T10:31:08.337109Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:33:39.066979Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (2m30.729865536s)' === 2025-12-04T10:33:39.067058Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:33:39.067172Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:33:39.173237Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/962c6b60ae-aadd-4eda-8e69-8180e3eeee11 2025-12-04T10:33:39.173345Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4471s 2025-12-04T10:33:39.173970Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.270457Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/96bd269ba7-9b0b-4162-b746-31a6aa0f3e12 2025-12-04T10:33:39.270569Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2030s 2025-12-04T10:33:39.271401Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.376398Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/9697e483e9-c32a-48f9-af31-1604d4c74a6e 2025-12-04T10:33:39.376628Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3091s 2025-12-04T10:33:39.377300Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider (151.10s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider (re-run 1) (151.10s) 2025-12-04T10:33:39.432955Z info tf === FAILED: Test Run: 'telemetry_tracing_otelcollector' (exitCode: 1) === 2025-12-04T10:33:39.433087Z info tf === Suite "telemetry_tracing_otelcollector" run time: 3m1.212427784s === 2025-12-04T10:33:39.433118Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/trace.yaml 2025-12-04T10:33:39.433600Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:33:39.433602Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:33:39.516043Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/964f97ace1-add7-4243-9b3f-69c08cfd9e92 2025-12-04T10:33:39.516438Z info tf istioctl ([x internal-debug --all configz]): completed after 0.4489s 2025-12-04T10:33:39.516848Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.622176Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/96e8a49ff0-e443-48f0-a8c6-8196cadf6f20 2025-12-04T10:33:39.622436Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1886s 2025-12-04T10:33:39.623071Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.725982Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/96e504ecb1-c8df-4727-8612-2b885a02572d 2025-12-04T10:33:39.726086Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2922s 2025-12-04T10:33:39.726756Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.932434Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/965165ba58-bce4-4507-a5c4-e7c6417af7d7 2025-12-04T10:33:39.932597Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4987s 2025-12-04T10:33:39.933536Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:40.244098Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:33:40.244147Z info tf === SUCCEEDED: Cleanup Istio in 43.481µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^grpc_exporter_with_auth$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2167647 2025-12-04T10:33:51.880901Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:33:51.880961Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:33:51.917546Z info tf === Test Framework Settings === 2025-12-04T10:33:51.917631Z info tf TestID: telemetry_tracing_otelcollector RunID: 06e3e71e-ad26-4415-a13d-4413262c8776 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:33:51.917637Z info tf =============================== 2025-12-04T10:33:51.917668Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:33:51.917711Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:33:51.917724Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:33:51.917742Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:33:51.917750Z info tf === BEGIN: Building clusters === 2025-12-04T10:33:51.921566Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:33:51.921574Z info tf === DONE: Building clusters === 2025-12-04T10:33:51.921591Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:33:52.520764Z info tf === SUCCESS: context setup function === 2025-12-04T10:33:52.520829Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:33:52.520841Z info tf === Istio Component Config === 2025-12-04T10:33:52.520905Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:33:52.520913Z info tf ================================ 2025-12-04T10:33:52.524180Z info tf skipping deployment as specified in the config 2025-12-04T10:33:52.524205Z info tf === SUCCEEDED: Deploy Istio in 3.371115ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:33:52.524217Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:33:52.632934Z info tf === SUCCEEDED: Create namespace echo in 108.705702ms === 2025-12-04T10:33:52.793255Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:33:53.056394Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:33:53.063491Z info sync complete name=pod controller attempt=3 time=7.004235ms 2025-12-04T10:33:53.157242Z info klog Caches are synced controller=echo 2025-12-04T10:43:53.158161Z error tf === FAILED: Deploy echo instances === 2025-12-04T10:43:53.158300Z error tf 2 errors occurred: * services "client-cluster-0" not found * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) 2025-12-04T10:43:53.158394Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:43:53.158402Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:43:53.169840Z error tf Test setup error: 2 errors occurred: * services "client-cluster-0" not found * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) 2025-12-04T10:43:53.169909Z info tf === FAILED: Setup: 'telemetry_tracing_otelcollector' (2 errors occurred: * services "client-cluster-0" not found * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) ) === 2025-12-04T10:43:53.169928Z error tf Exiting due to setup failure: 2 errors occurred: * services "client-cluster-0" not found * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) 2025-12-04T10:43:53.169965Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:43:53.169972Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:43:53.192176Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:43:53.192812Z info tf === SUCCEEDED: Cleanup Istio in 624.087µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestGatewayTracing$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2170151 2025-12-04T10:44:07.247343Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:44:07.247420Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:44:07.285041Z info tf === Test Framework Settings === 2025-12-04T10:44:07.285118Z info tf TestID: telemetry_tracing_otelcollector RunID: 0931f176-9f38-45a6-9ef7-c871edc928f4 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:44:07.285124Z info tf =============================== 2025-12-04T10:44:07.285257Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:44:07.285299Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:44:07.285311Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:44:07.285326Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:44:07.285334Z info tf === BEGIN: Building clusters === 2025-12-04T10:44:07.290305Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:44:07.290317Z info tf === DONE: Building clusters === 2025-12-04T10:44:07.290343Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:44:07.501341Z info tf === SUCCESS: context setup function === 2025-12-04T10:44:07.501389Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:44:07.501405Z info tf === Istio Component Config === 2025-12-04T10:44:07.501459Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:44:07.501467Z info tf ================================ 2025-12-04T10:44:07.506064Z info tf skipping deployment as specified in the config 2025-12-04T10:44:07.506092Z info tf === SUCCEEDED: Deploy Istio in 4.69565ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:44:07.506104Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:44:07.551957Z info tf === SUCCEEDED: Create namespace echo in 45.831512ms === 2025-12-04T10:44:08.337012Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:44:10.863379Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:44:10.865449Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:44:10.870238Z info sync complete name=pod controller attempt=3 time=6.767818ms 2025-12-04T10:44:10.872847Z info sync complete name=pod controller attempt=3 time=7.323267ms 2025-12-04T10:44:10.964515Z info klog Caches are synced controller=echo 2025-12-04T10:44:10.966053Z info klog Caches are synced controller=echo 2025-12-04T10:44:20.471214Z info tf === SUCCEEDED: Deploy echo instances in 12.134188128s === 2025-12-04T10:44:22.925712Z info tf Checking pods ready... 2025-12-04T10:44:22.925789Z info tf Checking pods ready... 2025-12-04T10:44:23.024338Z info tf [ 0] zipkin-74cc7d9448-xjc88 Pending (Pending) 2025-12-04T10:44:23.225100Z info tf Checking pods ready... 2025-12-04T10:44:23.225242Z info tf Checking pods ready... 2025-12-04T10:44:23.362835Z info tf [ 0] zipkin-74cc7d9448-xjc88 Pending (Pending) 2025-12-04T10:44:23.763812Z info tf Checking pods ready... 2025-12-04T10:44:23.763906Z info tf Checking pods ready... 2025-12-04T10:44:24.053081Z info tf [ 0] zipkin-74cc7d9448-xjc88 Pending (Pending) 2025-12-04T10:44:24.853381Z info tf Checking pods ready... 2025-12-04T10:44:24.853452Z info tf Checking pods ready... 2025-12-04T10:44:24.890919Z info tf [ 0] zipkin-74cc7d9448-xjc88 Running (container not ready: 'zipkin') 2025-12-04T10:44:26.491853Z info tf Checking pods ready... 2025-12-04T10:44:26.491956Z info tf Checking pods ready... 2025-12-04T10:44:26.763065Z info tf [ 0] zipkin-74cc7d9448-xjc88 Running (container not ready: 'zipkin') 2025-12-04T10:44:29.963820Z info tf Checking pods ready... 2025-12-04T10:44:29.963874Z info tf Checking pods ready... 2025-12-04T10:44:29.968324Z info tf [ 0] zipkin-74cc7d9448-xjc88 Running (container not ready: 'zipkin') 2025-12-04T10:44:33.168968Z info tf Checking pods ready... 2025-12-04T10:44:33.169038Z info tf Checking pods ready... 2025-12-04T10:44:33.175926Z info tf [ 0] zipkin-74cc7d9448-xjc88 Running (Ready) 2025-12-04T10:44:33.341356Z info tf Checking pods ready... 2025-12-04T10:44:33.341410Z info tf Checking pods ready... 2025-12-04T10:44:33.353194Z info tf Failed retrieving pods: no matching pod found for selectors: [app=opentelemetry-collector] 2025-12-04T10:44:33.553337Z info tf Checking pods ready... 2025-12-04T10:44:33.553430Z info tf Checking pods ready... 2025-12-04T10:44:33.558146Z info tf [ 0] opentelemetry-collector-749d46955f-trxjp Pending (Pending) 2025-12-04T10:44:33.958438Z info tf Checking pods ready... 2025-12-04T10:44:33.958482Z info tf Checking pods ready... 2025-12-04T10:44:33.962495Z info tf [ 0] opentelemetry-collector-749d46955f-trxjp Pending (Pending) 2025-12-04T10:44:34.763135Z info tf Checking pods ready... 2025-12-04T10:44:34.763202Z info tf Checking pods ready... 2025-12-04T10:44:34.767298Z info tf [ 0] opentelemetry-collector-749d46955f-trxjp Pending (Pending) 2025-12-04T10:44:36.368234Z info tf Checking pods ready... 2025-12-04T10:44:36.368304Z info tf Checking pods ready... 2025-12-04T10:44:36.373596Z info tf [ 0] opentelemetry-collector-749d46955f-trxjp Running (Ready) 2025-12-04T10:44:36.420347Z info tf === DONE: Setup: 'telemetry_tracing_otelcollector' (29.129985084s) === 2025-12-04T10:44:36.420396Z info tf === BEGIN: Test Run: 'telemetry_tracing_otelcollector' === === RUN TestGatewayTracing/primary-0 2025-12-04T10:44:36.447165Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0]' === tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:162: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:47:06.448441Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0] (2m30.001269987s)' === 2025-12-04T10:47:06.448560Z error tf === Dumping Namespace echo State for TestGatewayTracing/primary-0... 2025-12-04T10:47:06.448591Z error tf === Dumping Istio Deployment State for TestGatewayTracing/primary-0... 2025-12-04T10:47:06.597815Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/108b7c90833-706a-462e-99e6-dca812745203 2025-12-04T10:47:06.598190Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1489s 2025-12-04T10:47:06.598972Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestGatewayTracing/primary-0 (150.22s) FAIL tests/integration/telemetry/tracing/otelcollector.TestGatewayTracing/primary-0 (re-run 1) (150.22s) === RUN TestGatewayTracing 2025-12-04T10:44:36.420657Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing]' === 2025-12-04T10:47:06.669965Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing] (2m30.249298708s)' === 2025-12-04T10:47:06.670095Z error tf === Dumping Namespace echo State for TestGatewayTracing... 2025-12-04T10:47:06.670111Z error tf === Dumping Istio Deployment State for TestGatewayTracing... 2025-12-04T10:47:06.707815Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/108e4f2a5ca-1dd7-4311-9322-17b355f52b5a 2025-12-04T10:47:06.707937Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2586s 2025-12-04T10:47:06.708601Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:06.841285Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/1080746f40c-1f08-473f-9521-41cef56fd945 2025-12-04T10:47:06.841528Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3922s 2025-12-04T10:47:06.842141Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestGatewayTracing (150.52s) FAIL tests/integration/telemetry/tracing/otelcollector.TestGatewayTracing (re-run 1) (150.52s) 2025-12-04T10:47:06.940924Z info tf === FAILED: Test Run: 'telemetry_tracing_otelcollector' (exitCode: 1) === 2025-12-04T10:47:06.941019Z info tf === Suite "telemetry_tracing_otelcollector" run time: 2m59.650677928s === 2025-12-04T10:47:06.941038Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/trace.yaml 2025-12-04T10:47:06.941514Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:47:06.941518Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:47:06.960347Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/108ab823e49-1614-4766-bcde-3135b0473d0c 2025-12-04T10:47:06.960851Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2905s 2025-12-04T10:47:06.961817Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:07.139664Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:47:07Z/1095cba826b-1ab4-4b69-9011-4bef4673932a 2025-12-04T10:47:07.139798Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4694s 2025-12-04T10:47:07.140584Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:07.305746Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:47:07Z/109084e72bf-eb8b-4e23-bcf8-1e703bafa4c5 2025-12-04T10:47:07.305923Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.6355s 2025-12-04T10:47:07.306817Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:07.404954Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:47:07Z/109764ef681-e405-4eab-8570-50cfbe55c0eb 2025-12-04T10:47:07.405362Z info tf istioctl ([x internal-debug --all configz]): completed after 0.4635s 2025-12-04T10:47:07.405856Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:07.515409Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:47:07Z/109b9c12cb1-103a-48b2-8b3d-9640e90055ab 2025-12-04T10:47:07.515552Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.5737s 2025-12-04T10:47:07.516292Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:07.717924Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:47:07Z/1093c6768d7-0bcc-4ef2-9c2c-df65f6e10c55 2025-12-04T10:47:07.718071Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.7762s 2025-12-04T10:47:07.718843Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:07.998509Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:47:07.998562Z info tf === SUCCEEDED: Cleanup Istio in 45.455µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector DONE 2 runs, 21 tests, 21 failures in 2774.803s exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^http_exporter$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2171790 2025-12-04T10:47:19.997172Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:47:19.997231Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:47:20.033305Z info tf === Test Framework Settings === 2025-12-04T10:47:20.033436Z info tf TestID: telemetry_tracing_otelcollector RunID: 3f46e2dc-910a-4438-a4d0-d6e8ee2790f9 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:47:20.033445Z info tf =============================== 2025-12-04T10:47:20.033483Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:47:20.033535Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:47:20.033554Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:47:20.033578Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:47:20.033590Z info tf === BEGIN: Building clusters === 2025-12-04T10:47:20.038639Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:47:20.038650Z info tf === DONE: Building clusters === 2025-12-04T10:47:20.038676Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:47:20.231223Z info tf === SUCCESS: context setup function === 2025-12-04T10:47:20.231297Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:47:20.231308Z info tf === Istio Component Config === 2025-12-04T10:47:20.231360Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:47:20.231366Z info tf ================================ 2025-12-04T10:47:20.234526Z info tf skipping deployment as specified in the config 2025-12-04T10:47:20.234549Z info tf === SUCCEEDED: Deploy Istio in 3.249064ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:47:20.234557Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:47:20.376471Z info tf === SUCCEEDED: Create namespace echo in 141.896566ms === 2025-12-04T10:47:20.641300Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:47:20.760852Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:47:20.763471Z info sync complete name=pod controller attempt=2 time=2.525689ms 2025-12-04T10:47:20.861462Z info klog Caches are synced controller=echo 2025-12-04T10:47:20.884857Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:47:20.892045Z info sync complete name=pod controller attempt=3 time=7.122307ms 2025-12-04T10:47:20.986021Z info klog Caches are synced controller=echo 2025-12-04T10:57:20.986824Z error tf === FAILED: Deploy echo instances === 2025-12-04T10:57:20.986912Z error tf 2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) 2025-12-04T10:57:20.986980Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:57:20.986983Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:57:20.998706Z error tf Test setup error: 2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) 2025-12-04T10:57:20.998880Z info tf === FAILED: Setup: 'telemetry_tracing_otelcollector' (2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) ) === 2025-12-04T10:57:20.998891Z error tf Exiting due to setup failure: 2 errors occurred: * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/client-cluster-0) * timeout while waiting after 378 attempts (last error: no workloads ready for echo echo/server) 2025-12-04T10:57:21.000253Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:57:21.000295Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T10:57:21.026212Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:57:21.027499Z info tf === SUCCEEDED: Cleanup Istio in 1.274492ms [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestProxyTracingOpenTelemetryProvider$/^grpc_exporter_with_initial_metadata$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2174146 2025-12-04T10:57:34.549718Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:57:34.549773Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T10:57:34.583008Z info tf === Test Framework Settings === 2025-12-04T10:57:34.583096Z info tf TestID: telemetry_tracing_otelcollector RunID: 035711da-33ba-4032-9927-c2ba2db264d4 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T10:57:34.583101Z info tf =============================== 2025-12-04T10:57:34.583242Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T10:57:34.583281Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T10:57:34.583292Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T10:57:34.583309Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T10:57:34.583315Z info tf === BEGIN: Building clusters === 2025-12-04T10:57:34.587079Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:57:34.587087Z info tf === DONE: Building clusters === 2025-12-04T10:57:34.587102Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T10:57:34.642749Z info tf === SUCCESS: context setup function === 2025-12-04T10:57:34.642807Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T10:57:34.642818Z info tf === Istio Component Config === 2025-12-04T10:57:34.642871Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T10:57:34.642879Z info tf ================================ 2025-12-04T10:57:34.646034Z info tf skipping deployment as specified in the config 2025-12-04T10:57:34.646051Z info tf === SUCCEEDED: Deploy Istio in 3.24053ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T10:57:34.646059Z info tf === BEGIN: Create namespace echo === 2025-12-04T10:57:34.718048Z info tf === SUCCEEDED: Create namespace echo in 71.971329ms === 2025-12-04T10:57:34.779074Z info tf === BEGIN: Deploy echo instances === 2025-12-04T10:57:35.016209Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:57:35.021657Z info klog Waiting for caches to sync controller=echo 2025-12-04T10:57:35.022960Z info sync complete name=pod controller attempt=3 time=6.697177ms 2025-12-04T10:57:35.028972Z info sync complete name=pod controller attempt=3 time=7.240637ms 2025-12-04T10:57:35.116492Z info klog Caches are synced controller=echo 2025-12-04T10:57:35.122186Z info klog Caches are synced controller=echo 2025-12-04T10:57:46.271415Z info tf === SUCCEEDED: Deploy echo instances in 11.4922699s === 2025-12-04T10:57:46.856836Z info tf Checking pods ready... 2025-12-04T10:57:46.856894Z info tf Checking pods ready... 2025-12-04T10:57:46.877163Z info tf [ 0] zipkin-74cc7d9448-kcp85 Pending (Pending) 2025-12-04T10:57:47.077990Z info tf Checking pods ready... 2025-12-04T10:57:47.078123Z info tf Checking pods ready... 2025-12-04T10:57:47.083278Z info tf [ 0] zipkin-74cc7d9448-kcp85 Pending (Pending) 2025-12-04T10:57:47.483597Z info tf Checking pods ready... 2025-12-04T10:57:47.483687Z info tf Checking pods ready... 2025-12-04T10:57:47.487986Z info tf [ 0] zipkin-74cc7d9448-kcp85 Pending (Pending) 2025-12-04T10:57:48.288166Z info tf Checking pods ready... 2025-12-04T10:57:48.288244Z info tf Checking pods ready... 2025-12-04T10:57:48.293209Z info tf [ 0] zipkin-74cc7d9448-kcp85 Running (container not ready: 'zipkin') 2025-12-04T10:57:49.893881Z info tf Checking pods ready... 2025-12-04T10:57:49.893952Z info tf Checking pods ready... 2025-12-04T10:57:49.901492Z info tf [ 0] zipkin-74cc7d9448-kcp85 Running (container not ready: 'zipkin') 2025-12-04T10:57:53.101734Z info tf Checking pods ready... 2025-12-04T10:57:53.101831Z info tf Checking pods ready... 2025-12-04T10:57:53.106009Z info tf [ 0] zipkin-74cc7d9448-kcp85 Running (container not ready: 'zipkin') 2025-12-04T10:57:56.306500Z info tf Checking pods ready... 2025-12-04T10:57:56.306579Z info tf Checking pods ready... 2025-12-04T10:57:56.381601Z info tf [ 0] zipkin-74cc7d9448-kcp85 Running (Ready) 2025-12-04T10:57:56.773016Z info tf Checking pods ready... 2025-12-04T10:57:56.773044Z info tf Checking pods ready... 2025-12-04T10:57:56.777606Z info tf Failed retrieving pods: no matching pod found for selectors: [app=opentelemetry-collector] 2025-12-04T10:57:56.978192Z info tf Checking pods ready... 2025-12-04T10:57:56.978330Z info tf Checking pods ready... 2025-12-04T10:57:56.993709Z info tf [ 0] opentelemetry-collector-749d46955f-dpz8m Pending (Pending) 2025-12-04T10:57:57.394271Z info tf Checking pods ready... 2025-12-04T10:57:57.394339Z info tf Checking pods ready... 2025-12-04T10:57:57.399708Z info tf [ 0] opentelemetry-collector-749d46955f-dpz8m Pending (Pending) 2025-12-04T10:57:58.200021Z info tf Checking pods ready... 2025-12-04T10:57:58.200124Z info tf Checking pods ready... 2025-12-04T10:57:58.204853Z info tf [ 0] opentelemetry-collector-749d46955f-dpz8m Running (Ready) 2025-12-04T10:57:58.245480Z info tf === DONE: Setup: 'telemetry_tracing_otelcollector' (23.658361388s) === 2025-12-04T10:57:58.245544Z info tf === BEGIN: Test Run: 'telemetry_tracing_otelcollector' === === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 2025-12-04T10:57:58.256337Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T11:00:28.257725Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0] (2m30.001377815s)' === 2025-12-04T11:00:28.257982Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T11:00:28.257982Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T11:00:28.388914Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/12224e81abf-f39a-4d70-9567-8a8a464f1174 2025-12-04T11:00:28.389254Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1308s 2025-12-04T11:00:28.389784Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:28.506540Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/122d935def6-51b0-444f-a1c5-00507fb0a53c 2025-12-04T11:00:28.506691Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2483s 2025-12-04T11:00:28.507401Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (150.34s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (re-run 2) (150.34s) === RUN TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata 2025-12-04T10:57:58.246061Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata]' === 2025-12-04T11:00:28.599099Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata] (2m30.353030195s)' === 2025-12-04T11:00:28.599206Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T11:00:28.599217Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T11:00:28.613430Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/12288263bc3-92fa-4bdb-8c5d-805fb8151450 2025-12-04T11:00:28.613593Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3552s 2025-12-04T11:00:28.614356Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:28.749182Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/122483bd306-b91f-498c-bfc0-2baf8e88b7b4 2025-12-04T11:00:28.749306Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1499s 2025-12-04T11:00:28.749915Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:28.856623Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/122b3b0e8da-46b8-48ff-92fe-8b19b2abc868 2025-12-04T11:00:28.857042Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2576s 2025-12-04T11:00:28.857434Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (150.70s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (re-run 2) (150.70s) === RUN TestProxyTracingOpenTelemetryProvider 2025-12-04T10:57:58.245848Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T11:00:28.943105Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (2m30.69724764s)' === 2025-12-04T11:00:28.943236Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T11:00:28.943233Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T11:00:28.962850Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/1225d79acfb-7540-4a88-a668-56a00ffe57cb 2025-12-04T11:00:28.962962Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3636s 2025-12-04T11:00:28.964943Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.149451Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/1237263c6ec-0e01-4965-a1ed-6ca035aeda6e 2025-12-04T11:00:29.149716Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2063s 2025-12-04T11:00:29.150369Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.267656Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123ca391ec9-8c5e-4178-9260-8a6219a8640e 2025-12-04T11:00:29.267863Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3244s 2025-12-04T11:00:29.268404Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.366345Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123e58932c0-e74c-4f7b-8def-3becf40a3a2d 2025-12-04T11:00:29.366457Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4230s 2025-12-04T11:00:29.367298Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestProxyTracingOpenTelemetryProvider (151.19s) FAIL tests/integration/telemetry/tracing/otelcollector.TestProxyTracingOpenTelemetryProvider (re-run 2) (151.19s) 2025-12-04T11:00:29.431710Z info tf === FAILED: Test Run: 'telemetry_tracing_otelcollector' (exitCode: 1) === 2025-12-04T11:00:29.431739Z info tf === Suite "telemetry_tracing_otelcollector" run time: 2m54.844638164s === 2025-12-04T11:00:29.431754Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/trace.yaml 2025-12-04T11:00:29.431903Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T11:00:29.431920Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T11:00:29.598324Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123dfc93181-ce41-4491-850f-1b930d8226fb 2025-12-04T11:00:29.598697Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1666s 2025-12-04T11:00:29.599243Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.734169Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123a252c775-4894-442b-a93f-59d27bc0aac0 2025-12-04T11:00:29.734254Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3021s 2025-12-04T11:00:29.734912Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.918063Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123c6b77ec0-e84a-46c7-a62b-b6e2adfb4649 2025-12-04T11:00:29.918337Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4862s 2025-12-04T11:00:29.919337Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:30.583397Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T11:00:30.583500Z info tf === SUCCEEDED: Cleanup Istio in 92.755µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector exec: [go test -json -test.run=^TestGatewayTracing$/^primary-0$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/tracing/otelcollector -args -istio.test.skipWorkloads=vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts] go test pid: 2175540 2025-12-04T11:00:42.626255Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T11:00:42.626356Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3.x/istio-integration-test/istio/out/linux_ppc64le does not exist 2025-12-04T11:00:42.673048Z info tf === Test Framework Settings === 2025-12-04T11:00:42.673162Z info tf TestID: telemetry_tracing_otelcollector RunID: a2096547-4a4b-41d6-94ad-6822ef7c521a NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-04T11:00:42.673171Z info tf =============================== 2025-12-04T11:00:42.673208Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector- 2025-12-04T11:00:42.673265Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-04T11:00:42.673283Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-04T11:00:42.673309Z info tf Using KubeConfigs: [/root/upi/cC/auth/kubeconfig]. 2025-12-04T11:00:42.673322Z info tf === BEGIN: Building clusters === 2025-12-04T11:00:42.679233Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T11:00:42.679243Z info tf === DONE: Building clusters === 2025-12-04T11:00:42.679269Z info tf === BEGIN: Setup: 'telemetry_tracing_otelcollector' === 2025-12-04T11:00:42.749091Z info tf === SUCCESS: context setup function === 2025-12-04T11:00:42.749160Z info tf === BEGIN: Deploy Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T11:00:42.749174Z info tf === Istio Component Config === 2025-12-04T11:00:42.749238Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-04T11:00:42.749248Z info tf ================================ 2025-12-04T11:00:42.752889Z info tf skipping deployment as specified in the config 2025-12-04T11:00:42.752907Z info tf === SUCCEEDED: Deploy Istio in 3.743363ms [Suite=telemetry_tracing_otelcollector]=== 2025-12-04T11:00:42.752915Z info tf === BEGIN: Create namespace echo === 2025-12-04T11:00:42.805441Z info tf === SUCCEEDED: Create namespace echo in 52.508017ms === 2025-12-04T11:00:42.878109Z info tf === BEGIN: Deploy echo instances === 2025-12-04T11:00:42.924087Z error tf === FAILED: Deploy echo instances === 2025-12-04T11:00:42.924164Z error tf 2 errors occurred: * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.1618048769.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.338451205.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-12-04T11:00:42.924218Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T11:00:42.924233Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T11:00:42.936897Z error tf Test setup error: 2 errors occurred: * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.1618048769.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.338451205.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-12-04T11:00:42.936977Z info tf === FAILED: Setup: 'telemetry_tracing_otelcollector' (2 errors occurred: * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.1618048769.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.338451205.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated ) === 2025-12-04T11:00:42.936987Z error tf Exiting due to setup failure: 2 errors occurred: * failed deploying echo server.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.1618048769.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "server-v1" is forbidden: unable to create new content in namespace echo because it is being terminated * failed deploying echo client-cluster-0.echo.svc.cluster.local to cluster cluster-0: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-tracing-otelcollector/artifacts/telemetry-tracing-otelcollector-/_suite_context/Deployment.338451205.yaml] to ns echo in cluster cluster-0: apply: patch: deployments.apps "client-cluster-0-v1" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-12-04T11:00:42.937019Z error tf === Dumping Namespace echo State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T11:00:42.937037Z error tf === Dumping Istio Deployment State for [suite(telemetry_tracing_otelcollector)]... 2025-12-04T11:00:43.042584Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_tracing_otelcollector] === 2025-12-04T11:00:43.042663Z info tf === SUCCEEDED: Cleanup Istio in 73.705µs [Suite=telemetry_tracing_otelcollector] === FAIL tests/integration/telemetry/tracing/otelcollector === Failed === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0 (150.46s) 2025-12-04T10:01:37.726526Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:04:07.727600Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0] (2m30.00104514s)' === 2025-12-04T10:04:07.727728Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0... 2025-12-04T10:04:07.727728Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter/primary-0... 2025-12-04T10:04:07.911581Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/62584f5e01-fd5d-492f-b3fc-31c80fdf78dc 2025-12-04T10:04:07.911671Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1832s 2025-12-04T10:04:07.912319Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.059176Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/623aa6ccf0-378a-4865-8997-3490c0fad356 2025-12-04T10:04:08.059487Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3310s 2025-12-04T10:04:08.060026Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.157141Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/62719ce95a-7085-418a-8097-e1742ccdfbfb 2025-12-04T10:04:08.157244Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4287s 2025-12-04T10:04:08.157877Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter (150.94s) 2025-12-04T10:01:37.656566Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter]' === 2025-12-04T10:04:08.189389Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter] (2m30.532819684s)' === 2025-12-04T10:04:08.189487Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter... 2025-12-04T10:04:08.189520Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter... 2025-12-04T10:04:08.348071Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/620e2516ae-e053-4db1-9edf-ba9e5e55d075 2025-12-04T10:04:08.348190Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1584s 2025-12-04T10:04:08.348835Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.470669Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/62d93d7862-f958-4605-9459-26ff67b6cdd6 2025-12-04T10:04:08.470922Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2811s 2025-12-04T10:04:08.471533Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:04:08.592662Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:01:37Z/621a7e33f3-0749-4b48-b0e8-b2156846d4a5 2025-12-04T10:04:08.592778Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4030s === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 (150.57s) 2025-12-04T10:04:08.743784Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:06:38.744981Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0] (2m30.001194009s)' === 2025-12-04T10:06:38.745093Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:06:38.745102Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:06:38.917989Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64e5a6f993-1c36-44e0-aafc-b1f041b49109 2025-12-04T10:06:38.918288Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1726s 2025-12-04T10:06:38.919035Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.030177Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/644cf5cbbf-4653-4a95-9d2c-26795f69f680 2025-12-04T10:06:39.030310Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2846s 2025-12-04T10:06:39.031037Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.140183Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/6436107e83-cd71-492c-8ec3-dc660fc0834a 2025-12-04T10:06:39.140332Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3946s 2025-12-04T10:06:39.141217Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/http_exporter (151.37s) 2025-12-04T10:04:08.593023Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter]' === 2025-12-04T10:04:08.593344Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.313585Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter] (2m30.720548881s)' === 2025-12-04T10:06:39.313779Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:06:39.313784Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:06:39.444428Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64024440a6-ff60-4eac-b393-aca73b44462a 2025-12-04T10:06:39.444887Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1308s 2025-12-04T10:06:39.445739Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.569577Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64eb32dc7f-c8b1-4fe1-8ae2-aaf44c655861 2025-12-04T10:06:39.569709Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2556s 2025-12-04T10:06:39.570604Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:06:39.687878Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:04:08Z/64323dd572-9f25-4bfa-a1bd-d32fdd388a11 2025-12-04T10:06:39.688163Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3740s 2025-12-04T10:06:39.689113Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0 (150.76s) 2025-12-04T10:06:39.987947Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:09:09.989171Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0] (2m30.001214427s)' === 2025-12-04T10:09:09.989357Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0... 2025-12-04T10:09:09.989357Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/resource_detectors/primary-0... 2025-12-04T10:09:10.148536Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/653480cb46-22a5-4dbf-b287-3a839bb2a813 2025-12-04T10:09:10.148834Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1587s 2025-12-04T10:09:10.149795Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:10.252780Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/652d2a0416-176c-46e7-8232-51cd34a75bad 2025-12-04T10:09:10.252877Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2627s 2025-12-04T10:09:10.253942Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:10.361711Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/65163bad77-d299-4e01-b23c-cf1afe1444dc 2025-12-04T10:09:10.361920Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3717s 2025-12-04T10:09:10.362570Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/resource_detectors (151.54s) 2025-12-04T10:06:39.965485Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors]' === 2025-12-04T10:09:10.749250Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/resource_detectors] (2m30.783760071s)' === 2025-12-04T10:09:10.749396Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/resource_detectors... 2025-12-04T10:09:10.749397Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/resource_detectors... 2025-12-04T10:09:10.898012Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/65919e5e69-1a3c-40cc-b9e3-503434b8b329 2025-12-04T10:09:10.898425Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1488s 2025-12-04T10:09:10.898916Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:11.023631Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/65fce1903a-6d9d-4ecc-a475-1077adb16652 2025-12-04T10:09:11.023779Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2741s 2025-12-04T10:09:11.024599Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:09:11.140504Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:06:40Z/658a14066c-1076-4eac-a95f-f5e5833de1e1 2025-12-04T10:09:11.140647Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3909s 2025-12-04T10:09:11.141481Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (151.34s) 2025-12-04T10:09:11.516561Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:11:41.516893Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0] (2m30.00032346s)' === 2025-12-04T10:11:41.517064Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:11:41.517085Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:11:41.644372Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66d5986c66-bb5e-4dfd-b67b-b1485f8545b1 2025-12-04T10:11:41.644794Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1271s 2025-12-04T10:11:41.645268Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:41.812822Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66aa59ce37-87e0-4c77-94ca-91684f8d015c 2025-12-04T10:11:41.813058Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2953s 2025-12-04T10:11:41.813895Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:41.930244Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/6603d53e05-75f5-443c-bb70-81daba53f08c 2025-12-04T10:11:41.930387Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4127s 2025-12-04T10:11:41.931281Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (152.30s) 2025-12-04T10:09:11.501563Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata]' === 2025-12-04T10:11:42.851273Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata] (2m31.349702413s)' === 2025-12-04T10:11:42.851402Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:11:42.851412Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:11:43.067874Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66c9ec527a-b580-4d64-9862-fbec4b036b55 2025-12-04T10:11:43.068042Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2164s 2025-12-04T10:11:43.068888Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:43.208391Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/66fa0a42eb-aa3d-476d-9b82-b615b9ec06af 2025-12-04T10:11:43.208537Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3569s 2025-12-04T10:11:43.209255Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:11:43.324096Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:09:11Z/663de96a8f-be27-47d0-97e3-fa2dd6e3badb 2025-12-04T10:11:43.324426Z info tf istioctl ([x internal-debug --all configz]): completed after 0.4727s 2025-12-04T10:11:43.325101Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0 (151.28s) 2025-12-04T10:11:43.819137Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:14:13.819823Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0] (2m30.000677904s)' === 2025-12-04T10:14:13.819990Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0... 2025-12-04T10:14:13.820015Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth/primary-0... 2025-12-04T10:14:13.966477Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:11:43Z/67e326ac71-e513-4440-9eec-49f718e10e44 2025-12-04T10:14:13.966824Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1457s 2025-12-04T10:14:13.967532Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:14:14.090253Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:11:43Z/6738441e87-b855-404a-a948-5604b6be0bf9 2025-12-04T10:14:14.090383Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2691s 2025-12-04T10:14:14.091151Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:14:14.242633Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:11:43Z/6721f8138e-0fd9-44a6-969e-fe4ac9a051fc 2025-12-04T10:14:14.242859Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4216s 2025-12-04T10:14:14.243598Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth (151.31s) 2025-12-04T10:11:43.805759Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth]' === 2025-12-04T10:14:15.102579Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_auth] (2m31.296811056s)' === === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider (757.46s) 2025-12-04T10:01:37.656124Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:14:15.111199Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (12m37.455070547s)' === === FAIL: tests/integration/telemetry/tracing/otelcollector TestGatewayTracing/primary-0 (150.00s) 2025-12-04T10:14:15.145150Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0]' === tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:162: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:16:45.145718Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0] (2m30.000561817s)' === === FAIL: tests/integration/telemetry/tracing/otelcollector TestGatewayTracing (150.06s) 2025-12-04T10:14:15.111849Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing]' === 2025-12-04T10:16:45.145819Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing] (2m30.033968736s)' === === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0 (re-run 1) (150.38s) 2025-12-04T10:17:40.807992Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:20:10.809274Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0] (2m30.001276351s)' === 2025-12-04T10:20:10.809395Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:20:10.809395Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter/primary-0... 2025-12-04T10:20:10.928638Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80cbd73392-a6dc-43c4-855a-d08a3f3caf9a 2025-12-04T10:20:10.928923Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1192s 2025-12-04T10:20:10.929416Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.055083Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80484703d2-3e47-407b-bf25-83bb76a222e9 2025-12-04T10:20:11.055323Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2455s 2025-12-04T10:20:11.056113Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.161974Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80e1ae8510-ea54-46ee-9d64-b67cc8d4e7e1 2025-12-04T10:20:11.162125Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3523s 2025-12-04T10:20:11.162704Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/http_exporter (re-run 1) (150.72s) 2025-12-04T10:17:40.796096Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter]' === 2025-12-04T10:20:11.187578Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/http_exporter] (2m30.391473409s)' === 2025-12-04T10:20:11.187699Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:20:11.187713Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/http_exporter... 2025-12-04T10:20:11.327274Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/80151100d3-b33b-4542-8938-fc8597b87a1c 2025-12-04T10:20:11.327717Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1398s 2025-12-04T10:20:11.328267Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.455441Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/8013f148f7-45b8-4e67-832f-9513be59b99c 2025-12-04T10:20:11.455549Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2676s 2025-12-04T10:20:11.456314Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider (re-run 1) (151.06s) 2025-12-04T10:17:40.795984Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:20:11.515814Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (2m30.719817632s)' === 2025-12-04T10:20:11.515995Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:20:11.516001Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:20:11.613060Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:17:40Z/8096ed22d1-5472-48cf-af26-a6d75b637474 2025-12-04T10:20:11.613196Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4253s 2025-12-04T10:20:11.613855Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.735801Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81cf713575-2c5b-47dd-bd13-f2143a4ac748 2025-12-04T10:20:11.736170Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2193s 2025-12-04T10:20:11.736644Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:20:11.833507Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:20:11Z/81998dcf9f-6a07-46c2-8be5-3095425a89ba 2025-12-04T10:20:11.833647Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3167s 2025-12-04T10:20:11.834424Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (re-run 1) (150.35s) 2025-12-04T10:31:08.371986Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:33:38.373115Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0] (2m30.001120534s)' === 2025-12-04T10:33:38.373301Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:33:38.373305Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T10:33:38.498878Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/959be5ca7f-8963-45cb-ad3b-e82632374b3a 2025-12-04T10:33:38.499355Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1257s 2025-12-04T10:33:38.501177Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:38.604711Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/951cc824a9-3759-4f92-beca-2739e00cd2bb 2025-12-04T10:33:38.604811Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2311s 2025-12-04T10:33:38.605585Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:38.699103Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/95f7043f7a-484b-4f10-a13c-2247a08a990d 2025-12-04T10:33:38.699530Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3258s 2025-12-04T10:33:38.699961Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (re-run 1) (150.73s) 2025-12-04T10:31:08.337413Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata]' === 2025-12-04T10:33:38.725953Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata] (2m30.388533806s)' === 2025-12-04T10:33:38.726015Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:33:38.726030Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T10:33:38.909416Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/95ef904da0-0188-4918-a1c6-6de10d0e44af 2025-12-04T10:33:38.909763Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1835s 2025-12-04T10:33:38.910284Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.011510Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:31:08Z/95ebd54c82-b429-4e66-a4e2-add216dc570f 2025-12-04T10:33:39.011722Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2854s 2025-12-04T10:33:39.012524Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider (re-run 1) (151.10s) 2025-12-04T10:31:08.337109Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T10:33:39.066979Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (2m30.729865536s)' === 2025-12-04T10:33:39.067058Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:33:39.067172Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T10:33:39.173237Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/962c6b60ae-aadd-4eda-8e69-8180e3eeee11 2025-12-04T10:33:39.173345Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4471s 2025-12-04T10:33:39.173970Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.270457Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/96bd269ba7-9b0b-4162-b746-31a6aa0f3e12 2025-12-04T10:33:39.270569Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2030s 2025-12-04T10:33:39.271401Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:33:39.376398Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:33:39Z/9697e483e9-c32a-48f9-af31-1604d4c74a6e 2025-12-04T10:33:39.376628Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3091s 2025-12-04T10:33:39.377300Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestGatewayTracing/primary-0 (re-run 1) (150.22s) 2025-12-04T10:44:36.447165Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0]' === tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:164: Sending from cluster-0 ingress... tracing_test.go:162: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T10:47:06.448441Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing/primary-0] (2m30.001269987s)' === 2025-12-04T10:47:06.448560Z error tf === Dumping Namespace echo State for TestGatewayTracing/primary-0... 2025-12-04T10:47:06.448591Z error tf === Dumping Istio Deployment State for TestGatewayTracing/primary-0... 2025-12-04T10:47:06.597815Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/108b7c90833-706a-462e-99e6-dca812745203 2025-12-04T10:47:06.598190Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1489s 2025-12-04T10:47:06.598972Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestGatewayTracing (re-run 1) (150.52s) 2025-12-04T10:44:36.420657Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestGatewayTracing]' === 2025-12-04T10:47:06.669965Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestGatewayTracing] (2m30.249298708s)' === 2025-12-04T10:47:06.670095Z error tf === Dumping Namespace echo State for TestGatewayTracing... 2025-12-04T10:47:06.670111Z error tf === Dumping Istio Deployment State for TestGatewayTracing... 2025-12-04T10:47:06.707815Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/108e4f2a5ca-1dd7-4311-9322-17b355f52b5a 2025-12-04T10:47:06.707937Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2586s 2025-12-04T10:47:06.708601Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T10:47:06.841285Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:44:36Z/1080746f40c-1f08-473f-9521-41cef56fd945 2025-12-04T10:47:06.841528Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3922s 2025-12-04T10:47:06.842141Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0 (re-run 2) (150.34s) 2025-12-04T10:57:58.256337Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0]' === tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:120: Sending from cluster-0... tracing_test.go:119: retry.UntilSuccessOrFail: timeout while waiting after 50 attempts (last error: cannot get traces from zipkin: zipkin API returned status code: 404) 2025-12-04T11:00:28.257725Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0] (2m30.001377815s)' === 2025-12-04T11:00:28.257982Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T11:00:28.257982Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata/primary-0... 2025-12-04T11:00:28.388914Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/12224e81abf-f39a-4d70-9567-8a8a464f1174 2025-12-04T11:00:28.389254Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1308s 2025-12-04T11:00:28.389784Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:28.506540Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/122d935def6-51b0-444f-a1c5-00507fb0a53c 2025-12-04T11:00:28.506691Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.2483s 2025-12-04T11:00:28.507401Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata (re-run 2) (150.70s) 2025-12-04T10:57:58.246061Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata]' === 2025-12-04T11:00:28.599099Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata] (2m30.353030195s)' === 2025-12-04T11:00:28.599206Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T11:00:28.599217Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider/grpc_exporter_with_initial_metadata... 2025-12-04T11:00:28.613430Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/12288263bc3-92fa-4bdb-8c5d-805fb8151450 2025-12-04T11:00:28.613593Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3552s 2025-12-04T11:00:28.614356Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:28.749182Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/122483bd306-b91f-498c-bfc0-2baf8e88b7b4 2025-12-04T11:00:28.749306Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1499s 2025-12-04T11:00:28.749915Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:28.856623Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/122b3b0e8da-46b8-48ff-92fe-8b19b2abc868 2025-12-04T11:00:28.857042Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2576s 2025-12-04T11:00:28.857434Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/tracing/otelcollector TestProxyTracingOpenTelemetryProvider (re-run 2) (151.19s) 2025-12-04T10:57:58.245848Z info tf === BEGIN: Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider]' === tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig tracing_test.go:132: got traces [] from Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cC/auth/kubeconfig 2025-12-04T11:00:28.943105Z info tf === DONE (failed): Test: 'telemetry_tracing_otelcollector[TestProxyTracingOpenTelemetryProvider] (2m30.69724764s)' === 2025-12-04T11:00:28.943236Z error tf === Dumping Namespace echo State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T11:00:28.943233Z error tf === Dumping Istio Deployment State for TestProxyTracingOpenTelemetryProvider... 2025-12-04T11:00:28.962850Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T10:57:58Z/1225d79acfb-7540-4a88-a668-56a00ffe57cb 2025-12-04T11:00:28.962962Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3636s 2025-12-04T11:00:28.964943Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.149451Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/1237263c6ec-0e01-4965-a1ed-6ca035aeda6e 2025-12-04T11:00:29.149716Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2063s 2025-12-04T11:00:29.150369Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.267656Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123ca391ec9-8c5e-4178-9260-8a6219a8640e 2025-12-04T11:00:29.267863Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3244s 2025-12-04T11:00:29.268404Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-04T11:00:29.366345Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-04T11:00:29Z/123e58932c0-e74c-4f7b-8def-3becf40a3a2d 2025-12-04T11:00:29.366457Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4230s 2025-12-04T11:00:29.367298Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF DONE 3 runs, 24 tests, 24 failures in 3589.846s exec: go version