exec: [go test -json -tags=integ -timeout 180m ./tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 4106811 2025-08-01T13:21:27.628570Z warn unable to resolve TARGET_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:21:27.628641Z warn unable to resolve LOCAL_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:21:27.659201Z info tf === Test Framework Settings === 2025-08-01T13:21:27.659301Z info tf TestID: telemetry_api RunID: d4225795-5b18-4f3e-ab0b-f9969f53042f NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-08-01T13:21:27.659306Z info tf =============================== 2025-08-01T13:21:27.659675Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-d42257955b184f3ea 2025-08-01T13:21:27.659805Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-08-01T13:21:27.659824Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-08-01T13:21:27.659853Z info tf Using KubeConfigs: [/root/upi/cO/auth/kubeconfig]. 2025-08-01T13:21:27.659866Z info tf === BEGIN: Building clusters === 2025-08-01T13:21:27.665251Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cO/auth/kubeconfig 2025-08-01T13:21:27.665305Z info tf === DONE: Building clusters === 2025-08-01T13:21:27.665329Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-08-01T13:21:27.665357Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-08-01T13:21:27.665366Z info tf === Istio Component Config === 2025-08-01T13:21:27.665406Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-08-01T13:21:27.665412Z info tf ================================ 2025-08-01T13:21:27.668126Z info tf skipping deployment as specified in the config 2025-08-01T13:21:27.668147Z info tf === SUCCEEDED: Deploy Istio in 2.783496ms [Suite=telemetry_api]=== 2025-08-01T13:21:27.693684Z info tf === BEGIN: Deploy registry redirector server === 2025-08-01T13:21:27.693715Z info tf === BEGIN: Create namespace registry-redirector === 2025-08-01T13:21:27.706055Z info tf === SUCCEEDED: Create namespace registry-redirector in 12.332733ms === 2025-08-01T13:21:27.754487Z info tf Checking pods ready... 2025-08-01T13:21:27.754519Z info tf Checking pods ready... 2025-08-01T13:21:27.758837Z info tf No pods found... 2025-08-01T13:21:27.959581Z info tf Checking pods ready... 2025-08-01T13:21:27.959633Z info tf Checking pods ready... 2025-08-01T13:21:27.980855Z info tf [ 0] registry-redirector-7f6bd4c5df-tk6qc Pending (Pending) 2025-08-01T13:21:28.381873Z info tf Checking pods ready... 2025-08-01T13:21:28.381920Z info tf Checking pods ready... 2025-08-01T13:21:28.387358Z info tf [ 0] registry-redirector-7f6bd4c5df-tk6qc Pending (Pending) 2025-08-01T13:21:29.187440Z info tf Checking pods ready... 2025-08-01T13:21:29.187489Z info tf Checking pods ready... 2025-08-01T13:21:29.239418Z info tf [ 0] registry-redirector-7f6bd4c5df-tk6qc Pending (Pending) 2025-08-01T13:21:30.840358Z info tf Checking pods ready... 2025-08-01T13:21:30.840387Z info tf Checking pods ready... 2025-08-01T13:21:30.843794Z info tf [ 0] registry-redirector-7f6bd4c5df-tk6qc Pending (Pending) 2025-08-01T13:21:34.044840Z info tf Checking pods ready... 2025-08-01T13:21:34.044921Z info tf Checking pods ready... 2025-08-01T13:21:34.048744Z info tf [ 0] registry-redirector-7f6bd4c5df-tk6qc Running (Ready) 2025-08-01T13:21:34.053853Z info tf registry redirector server address: 172.30.225.228:1338 2025-08-01T13:21:34.120743Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-08-01T13:21:34.120841Z info tf === BEGIN: Create namespace external === 2025-08-01T13:21:34.120860Z info tf === BEGIN: Create namespace echo === 2025-08-01T13:21:34.127643Z info tf === SUCCEEDED: Create namespace external in 6.809953ms === 2025-08-01T13:21:34.135194Z info tf === SUCCEEDED: Create namespace echo in 14.345052ms === 2025-08-01T13:21:34.275661Z info tf === BEGIN: Deploy echo instances === 2025-08-01T13:21:34.639587Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.652009Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.685836Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.689774Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.690319Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.698658Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.699073Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.699058Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.699114Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.699333Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:21:34.740475Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.753122Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.786422Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.790115Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.790567Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.799112Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.799190Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.799172Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.799230Z info klog Caches are synced controller=echo 2025-08-01T13:21:34.799493Z info klog Caches are synced controller=echo 2025-08-01T13:22:41.930267Z info tf === SUCCEEDED: Deploy echo instances in 1m7.654596187s === 2025-08-01T13:22:42.226571Z info tf Checking pods ready... 2025-08-01T13:22:42.226645Z info tf Checking pods ready... 2025-08-01T13:22:42.254053Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-08-01T13:22:42.454884Z info tf Checking pods ready... 2025-08-01T13:22:42.454993Z info tf Checking pods ready... 2025-08-01T13:22:42.471054Z info tf [ 0] prometheus-689cc795d4-g4ps8 Pending (Pending) 2025-08-01T13:22:42.871257Z info tf Checking pods ready... 2025-08-01T13:22:42.871308Z info tf Checking pods ready... 2025-08-01T13:22:42.878567Z info tf [ 0] prometheus-689cc795d4-g4ps8 Pending (Pending) 2025-08-01T13:22:43.679295Z info tf Checking pods ready... 2025-08-01T13:22:43.679328Z info tf Checking pods ready... 2025-08-01T13:22:43.689759Z info tf [ 0] prometheus-689cc795d4-g4ps8 Pending (Pending) 2025-08-01T13:22:45.290193Z info tf Checking pods ready... 2025-08-01T13:22:45.290220Z info tf Checking pods ready... 2025-08-01T13:22:45.300169Z info tf [ 0] prometheus-689cc795d4-g4ps8 Pending (Pending) 2025-08-01T13:22:48.500969Z info tf Checking pods ready... 2025-08-01T13:22:48.500998Z info tf Checking pods ready... 2025-08-01T13:22:48.505956Z info tf [ 0] prometheus-689cc795d4-g4ps8 Running (Ready) 2025-08-01T13:22:48.590342Z info tf === DONE: Setup: 'telemetry_api' (1m20.925001572s) === 2025-08-01T13:22:48.590374Z info tf === BEGIN: Test Run: 'telemetry_api' === PASS tests/integration/telemetry/api.TestAccessLogs/enabled (0.63s) PASS tests/integration/telemetry/api.TestAccessLogs/enabled-with-targetref (6.26s) PASS tests/integration/telemetry/api.TestAccessLogs/disabled (2.13s) PASS tests/integration/telemetry/api.TestAccessLogs (9.02s) PASS tests/integration/telemetry/api.TestAccessLogsFilter (3.72s) PASS tests/integration/telemetry/api.TestAccessLogsMode/client (11.28s) PASS tests/integration/telemetry/api.TestAccessLogsMode/server (11.37s) PASS tests/integration/telemetry/api.TestAccessLogsMode/client-and-server (11.34s) PASS tests/integration/telemetry/api.TestAccessLogsMode (33.99s) PASS tests/integration/telemetry/api.TestAccessLogsDefaultProvider/disabled (8.36s) PASS tests/integration/telemetry/api.TestAccessLogsDefaultProvider/enabled (1.23s) PASS tests/integration/telemetry/api.TestAccessLogsDefaultProvider (9.59s) PASS tests/integration/telemetry/api.TestAccessLogWithFilterState (1.10s) PASS tests/integration/telemetry/api.TestDashboard/istio-mesh-dashboard.json (0.06s) PASS tests/integration/telemetry/api.TestDashboard/istio-service-dashboard.json (0.28s) PASS tests/integration/telemetry/api.TestDashboard/istio-workload-dashboard.json (0.26s) PASS tests/integration/telemetry/api.TestDashboard/istio-performance-dashboard.json (6.84s) PASS tests/integration/telemetry/api.TestDashboard (7.70s) PASS tests/integration/telemetry/api.TestIstioctlMetrics (0.13s) PASS tests/integration/telemetry/api.TestStatsFilter/default (2.09s) === RUN TestStatsFilter/additional-labels 2025-08-01T13:23:55.933742Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 8 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 4 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 6 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 5 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-08-01T13:25:15.952440Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.018694093s)' === 2025-08-01T13:25:15.952582Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-08-01T13:25:15.952601Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-08-01T13:25:15.952595Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-08-01T13:25:15.952619Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-08-01T13:25:16.317551Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:23:56Z/9190587996-6e3d-4725-9bff-5b84cc6f81c8 2025-08-01T13:25:16.317906Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3650s 2025-08-01T13:25:16.318382Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:16.554466Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:23:56Z/91da330819-760f-4f2f-9358-62e2d44caa11 2025-08-01T13:25:16.554585Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.6017s 2025-08-01T13:25:16.555342Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:16.820430Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:23:56Z/91d254bf70-7351-4226-9ed0-c4fd1e90f5d7 2025-08-01T13:25:16.820558Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.8676s 2025-08-01T13:25:16.821336Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter/additional-labels (82.47s) FAIL tests/integration/telemetry/api.TestStatsFilter/additional-labels (82.47s) PASS tests/integration/telemetry/api.TestStatsFilter/mockprom-to-metrics (0.06s) === RUN TestStatsFilter 2025-08-01T13:23:53.842655Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-08-01T13:25:18.470187Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m24.627531065s)' === 2025-08-01T13:25:18.470263Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-08-01T13:25:18.470276Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-08-01T13:25:18.470298Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-08-01T13:25:18.470305Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-08-01T13:25:21.063493Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:18Z/927cb6ffd4-d650-4c53-814e-83820b4ec31d 2025-08-01T13:25:21.063621Z info tf istioctl ([x internal-debug --all mcsz]): completed after 2.5929s 2025-08-01T13:25:21.064473Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:21.248654Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:18Z/92d7433f53-5a2a-465b-bb3c-19032ff46301 2025-08-01T13:25:21.248803Z info tf istioctl ([x internal-debug --all clusterz]): completed after 2.7781s 2025-08-01T13:25:21.249556Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:21.337016Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:18Z/92fd8294a5-c860-4f54-8dce-df6544fc1ccd 2025-08-01T13:25:21.337338Z info tf istioctl ([x internal-debug --all configz]): completed after 2.8666s 2025-08-01T13:25:21.338047Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter (88.06s) FAIL tests/integration/telemetry/api.TestStatsFilter (88.06s) PASS tests/integration/telemetry/api.TestStatsTCPFilter/default (6.09s) === RUN TestStatsTCPFilter/additional-labels 2025-08-01T13:25:27.987352Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-08-01T13:26:47.999532Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.012171233s)' === 2025-08-01T13:26:47.999680Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:47.999687Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:47.999671Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:47.999671Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:48.317342Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:28Z/93c86d6754-9b61-4656-9d0e-f8f90cd555e4 2025-08-01T13:26:48.317518Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3174s 2025-08-01T13:26:48.318411Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:48.598270Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:28Z/93c6aca51d-b016-4025-ae16-22302cf2065c 2025-08-01T13:26:48.598590Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5984s 2025-08-01T13:26:48.599417Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:48.761648Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:28Z/930725d6dd-9e1d-463b-9fa6-79c0e1a45053 2025-08-01T13:26:48.762340Z info tf istioctl ([x internal-debug --all configz]): completed after 0.7621s 2025-08-01T13:26:48.763313Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter/additional-labels (82.90s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter/additional-labels (82.90s) === RUN TestStatsTCPFilter 2025-08-01T13:25:21.900161Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-08-01T13:26:50.885196Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m28.985028269s)' === 2025-08-01T13:26:50.885302Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-08-01T13:26:50.885313Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-08-01T13:26:50.885344Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-08-01T13:26:50.885365Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-08-01T13:26:51.265937Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:26:50Z/94c13423be-ae93-44a3-941f-4dca859c7ef5 2025-08-01T13:26:51.266050Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3805s 2025-08-01T13:26:51.266658Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:51.393659Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:26:50Z/94de069d26-e586-43d7-bf40-993557f07056 2025-08-01T13:26:51.393735Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5081s 2025-08-01T13:26:51.394447Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:51.549381Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:26:50Z/94a218a84c-a55a-456b-9497-9e177c062a4b 2025-08-01T13:26:51.549830Z info tf istioctl ([x internal-debug --all configz]): completed after 0.6642s 2025-08-01T13:26:51.550498Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter (92.05s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter (92.05s) PASS tests/integration/telemetry/api.TestGRPCCountMetrics/istio_request_messages_total (2.12s) PASS tests/integration/telemetry/api.TestGRPCCountMetrics/istio_response_messages_total (0.03s) PASS tests/integration/telemetry/api.TestGRPCCountMetrics (2.15s) PASS tests/integration/telemetry/api.TestImagePullPolicy/Delete_WasmPlugin_wasm-test-module (0.04s) PASS tests/integration/telemetry/api.TestImagePullPolicy/Delete_WasmPlugin_wasm-test-module#01 (0.01s) PASS tests/integration/telemetry/api.TestImagePullPolicy (0.06s) PASS tests/integration/telemetry/api.TestGatewaySelection/Delete_WasmPlugin_wasm-test-module (0.02s) PASS tests/integration/telemetry/api.TestGatewaySelection (0.06s) PASS tests/integration/telemetry/api.TestImagePullPolicyWithHTTP (0.00s) PASS tests/integration/telemetry/api.TestBadWasmRemoteLoad (10.25s) PASS tests/integration/telemetry/api.TestBadWasmWithFailOpen (24.18s) 2025-08-01T13:27:30.633565Z info tf === FAILED: Test Run: 'telemetry_api' (exitCode: 1) === 2025-08-01T13:27:30.633611Z info tf === Suite "telemetry_api" run time: 6m2.968283335s === 2025-08-01T13:27:30.633630Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/trace.yaml 2025-08-01T13:27:30.634185Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-08-01T13:27:30.634229Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-08-01T13:27:30.634192Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-08-01T13:27:30.634223Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-08-01T13:27:31.013150Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:30Z/98d798d7e5-8645-40b6-8154-9550f9b26883 2025-08-01T13:27:31.013457Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3790s 2025-08-01T13:27:31.013776Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:31.176664Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:30Z/981353015b-bfce-4521-aec8-d8559eb187d4 2025-08-01T13:27:31.176855Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.5424s 2025-08-01T13:27:31.177642Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:31.337840Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:30Z/982dfd8800-4afe-4aea-ad54-1c59851b1e0a 2025-08-01T13:27:31.337998Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.7035s 2025-08-01T13:27:31.338640Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:33.919594Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-08-01T13:27:33.919711Z info tf === SUCCEEDED: Cleanup Istio in 99.837µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api DONE 37 tests, 4 failures in 377.927s exec: [go test -json -test.run=^TestStatsFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 4108525 2025-08-01T13:27:47.594880Z warn unable to resolve TARGET_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:27:47.594961Z warn unable to resolve LOCAL_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:27:47.631510Z info tf === Test Framework Settings === 2025-08-01T13:27:47.631682Z info tf TestID: telemetry_api RunID: 50a75f98-73a0-474c-800a-2c68ef4038a3 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-08-01T13:27:47.631694Z info tf =============================== 2025-08-01T13:27:47.631989Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-50a75f9873a0474c8 2025-08-01T13:27:47.632143Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-08-01T13:27:47.632164Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-08-01T13:27:47.632195Z info tf Using KubeConfigs: [/root/upi/cO/auth/kubeconfig]. 2025-08-01T13:27:47.632212Z info tf === BEGIN: Building clusters === 2025-08-01T13:27:47.644202Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cO/auth/kubeconfig 2025-08-01T13:27:47.644234Z info tf === DONE: Building clusters === 2025-08-01T13:27:47.644276Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-08-01T13:27:47.644328Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-08-01T13:27:47.644342Z info tf === Istio Component Config === 2025-08-01T13:27:47.644416Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-08-01T13:27:47.644428Z info tf ================================ 2025-08-01T13:27:47.652392Z info tf skipping deployment as specified in the config 2025-08-01T13:27:47.652469Z info tf === SUCCEEDED: Deploy Istio in 8.133159ms [Suite=telemetry_api]=== 2025-08-01T13:27:47.674876Z info tf === BEGIN: Deploy registry redirector server === 2025-08-01T13:27:47.674918Z info tf === BEGIN: Create namespace registry-redirector === 2025-08-01T13:27:47.686940Z info tf === SUCCEEDED: Create namespace registry-redirector in 11.999927ms === 2025-08-01T13:27:47.832411Z info tf Checking pods ready... 2025-08-01T13:27:47.832449Z info tf Checking pods ready... 2025-08-01T13:27:47.847581Z info tf No pods found... 2025-08-01T13:27:48.048354Z info tf Checking pods ready... 2025-08-01T13:27:48.048453Z info tf Checking pods ready... 2025-08-01T13:27:48.081512Z info tf [ 0] registry-redirector-7f6bd4c5df-86kp6 Pending (Pending) 2025-08-01T13:27:48.482422Z info tf Checking pods ready... 2025-08-01T13:27:48.482468Z info tf Checking pods ready... 2025-08-01T13:27:48.486635Z info tf [ 0] registry-redirector-7f6bd4c5df-86kp6 Pending (Pending) 2025-08-01T13:27:49.286814Z info tf Checking pods ready... 2025-08-01T13:27:49.286909Z info tf Checking pods ready... 2025-08-01T13:27:49.292532Z info tf [ 0] registry-redirector-7f6bd4c5df-86kp6 Pending (Pending) 2025-08-01T13:27:50.893452Z info tf Checking pods ready... 2025-08-01T13:27:50.893485Z info tf Checking pods ready... 2025-08-01T13:27:50.908807Z info tf [ 0] registry-redirector-7f6bd4c5df-86kp6 Pending (Pending) 2025-08-01T13:27:54.110087Z info tf Checking pods ready... 2025-08-01T13:27:54.110170Z info tf Checking pods ready... 2025-08-01T13:27:54.113602Z info tf [ 0] registry-redirector-7f6bd4c5df-86kp6 Running (Ready) 2025-08-01T13:27:54.118028Z info tf registry redirector server address: 172.30.207.89:1338 2025-08-01T13:27:54.164013Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-08-01T13:27:54.164174Z info tf === BEGIN: Create namespace external === 2025-08-01T13:27:54.164177Z info tf === BEGIN: Create namespace echo === 2025-08-01T13:27:54.173862Z info tf === SUCCEEDED: Create namespace external in 9.700121ms === 2025-08-01T13:27:54.175677Z info tf === SUCCEEDED: Create namespace echo in 11.489058ms === 2025-08-01T13:27:54.244005Z info tf === BEGIN: Deploy echo instances === 2025-08-01T13:27:54.260149Z error tf === FAILED: Deploy echo instances === 2025-08-01T13:27:54.260234Z error tf failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-50a75f9873a0474c8/_suite_context/ServiceAccount-Service.4180151417.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "b" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-08-01T13:27:54.260322Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-08-01T13:27:54.260355Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-08-01T13:27:54.260327Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-08-01T13:27:54.260360Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-08-01T13:27:54.382868Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:52Z/104316b57fd-90b2-48d3-a76b-bbcd3ddf635a 2025-08-01T13:27:54.383131Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1222s 2025-08-01T13:27:54.383896Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:54.487573Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:52Z/10496191bbb-157d-4624-9dbc-8e1f9945146a 2025-08-01T13:27:54.487712Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2268s 2025-08-01T13:27:54.488337Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:54.574772Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:52Z/1044375296d-dcfe-4f97-b545-755724254157 2025-08-01T13:27:54.574899Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3140s 2025-08-01T13:27:54.575645Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:54.602344Z error tf Test setup error: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-50a75f9873a0474c8/_suite_context/ServiceAccount-Service.4180151417.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "b" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-08-01T13:27:54.602372Z info tf === FAILED: Setup: 'telemetry_api' (failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-50a75f9873a0474c8/_suite_context/ServiceAccount-Service.4180151417.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "b" is forbidden: unable to create new content in namespace echo because it is being terminated) === 2025-08-01T13:27:54.602381Z error tf Exiting due to setup failure: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-50a75f9873a0474c8/_suite_context/ServiceAccount-Service.4180151417.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "b" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-08-01T13:27:54.602415Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-08-01T13:27:54.602438Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-08-01T13:27:54.602469Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-08-01T13:27:54.602487Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-08-01T13:27:54.789484Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:52Z/1041584917a-0879-4700-b9c3-b8681d124376 2025-08-01T13:27:54.789777Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1870s 2025-08-01T13:27:54.790459Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:54.898314Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:52Z/1049b66e46a-c2f2-400e-ae57-2a49f12f06b4 2025-08-01T13:27:54.898468Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2956s 2025-08-01T13:27:54.899149Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:55.048081Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:27:52Z/10442741ab9-ecf6-4a27-93f1-55c97dd05639 2025-08-01T13:27:55.048252Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4454s 2025-08-01T13:27:55.049058Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:27:55.076152Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-08-01T13:27:55.076199Z info tf === SUCCEEDED: Cleanup Istio in 42.258µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api exec: [go test -json -test.run=^TestStatsTCPFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 4110054 2025-08-01T13:28:08.165553Z warn unable to resolve TARGET_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:28:08.165609Z warn unable to resolve LOCAL_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:28:08.195815Z info tf === Test Framework Settings === 2025-08-01T13:28:08.195926Z info tf TestID: telemetry_api RunID: 432b9b11-0a0a-4583-a820-60c72267afc7 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-08-01T13:28:08.195931Z info tf =============================== 2025-08-01T13:28:08.196072Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-432b9b110a0a4583a 2025-08-01T13:28:08.196145Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-08-01T13:28:08.196157Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-08-01T13:28:08.196174Z info tf Using KubeConfigs: [/root/upi/cO/auth/kubeconfig]. 2025-08-01T13:28:08.196180Z info tf === BEGIN: Building clusters === 2025-08-01T13:28:08.199521Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cO/auth/kubeconfig 2025-08-01T13:28:08.199530Z info tf === DONE: Building clusters === 2025-08-01T13:28:08.199545Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-08-01T13:28:08.199566Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-08-01T13:28:08.199572Z info tf === Istio Component Config === 2025-08-01T13:28:08.199630Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-08-01T13:28:08.199635Z info tf ================================ 2025-08-01T13:28:08.204327Z info tf skipping deployment as specified in the config 2025-08-01T13:28:08.204400Z info tf === SUCCEEDED: Deploy Istio in 4.828281ms [Suite=telemetry_api]=== 2025-08-01T13:28:08.221730Z info tf === BEGIN: Deploy registry redirector server === 2025-08-01T13:28:08.221758Z info tf === BEGIN: Create namespace registry-redirector === 2025-08-01T13:28:08.233647Z info tf === SUCCEEDED: Create namespace registry-redirector in 11.877543ms === 2025-08-01T13:28:08.281007Z info tf Checking pods ready... 2025-08-01T13:28:08.281039Z info tf Checking pods ready... 2025-08-01T13:28:08.289362Z info tf No pods found... 2025-08-01T13:28:08.490174Z info tf Checking pods ready... 2025-08-01T13:28:08.490253Z info tf Checking pods ready... 2025-08-01T13:28:08.493676Z info tf [ 0] registry-redirector-7f6bd4c5df-4xkj6 Pending (Pending) 2025-08-01T13:28:08.894440Z info tf Checking pods ready... 2025-08-01T13:28:08.894584Z info tf Checking pods ready... 2025-08-01T13:28:08.898844Z info tf [ 0] registry-redirector-7f6bd4c5df-4xkj6 Pending (Pending) 2025-08-01T13:28:09.698965Z info tf Checking pods ready... 2025-08-01T13:28:09.699126Z info tf Checking pods ready... 2025-08-01T13:28:09.703616Z info tf [ 0] registry-redirector-7f6bd4c5df-4xkj6 Pending (Pending) 2025-08-01T13:28:11.304582Z info tf Checking pods ready... 2025-08-01T13:28:11.304673Z info tf Checking pods ready... 2025-08-01T13:28:11.308662Z info tf [ 0] registry-redirector-7f6bd4c5df-4xkj6 Running (container not ready: 'registry-redirector') 2025-08-01T13:28:14.510032Z info tf Checking pods ready... 2025-08-01T13:28:14.510080Z info tf Checking pods ready... 2025-08-01T13:28:14.514677Z info tf [ 0] registry-redirector-7f6bd4c5df-4xkj6 Running (Ready) 2025-08-01T13:28:14.520034Z info tf registry redirector server address: 172.30.122.42:1338 2025-08-01T13:28:14.557727Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-08-01T13:28:14.557833Z info tf === BEGIN: Create namespace external === 2025-08-01T13:28:14.557888Z info tf === BEGIN: Create namespace echo === 2025-08-01T13:28:14.568097Z info tf === SUCCEEDED: Create namespace external in 10.26581ms === 2025-08-01T13:28:14.569926Z info tf === SUCCEEDED: Create namespace echo in 12.078648ms === 2025-08-01T13:28:14.637997Z info tf === BEGIN: Deploy echo instances === 2025-08-01T13:28:14.910686Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.922832Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.950129Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.998859Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.998831Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.999321Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.999505Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.999553Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:14.999690Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:15.000078Z info klog Waiting for caches to sync controller=echo 2025-08-01T13:28:15.011485Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.023331Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.050952Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.099756Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.099787Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.099754Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.099754Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.099778Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.099868Z info klog Caches are synced controller=echo 2025-08-01T13:28:15.100137Z info klog Caches are synced controller=echo 2025-08-01T13:28:45.480676Z info tf echo pod echo/a-v1-b558c7b5f-q7m2g (in cluster cluster-0) transitioned to NOT READY. Pod Status=Running 2025-08-01T13:28:48.615010Z info tf === SUCCEEDED: Deploy echo instances in 33.977000873s === 2025-08-01T13:28:48.927479Z info tf Checking pods ready... 2025-08-01T13:28:48.927570Z info tf Checking pods ready... 2025-08-01T13:28:48.938670Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-08-01T13:28:49.139663Z info tf Checking pods ready... 2025-08-01T13:28:49.139741Z info tf Checking pods ready... 2025-08-01T13:28:49.161653Z info tf [ 0] prometheus-689cc795d4-cs8tw Pending (Pending) 2025-08-01T13:28:49.562490Z info tf Checking pods ready... 2025-08-01T13:28:49.562586Z info tf Checking pods ready... 2025-08-01T13:28:49.567731Z info tf [ 0] prometheus-689cc795d4-cs8tw Pending (Pending) 2025-08-01T13:28:50.368561Z info tf Checking pods ready... 2025-08-01T13:28:50.368622Z info tf Checking pods ready... 2025-08-01T13:28:50.372683Z info tf [ 0] prometheus-689cc795d4-cs8tw Pending (Pending) 2025-08-01T13:28:51.972830Z info tf Checking pods ready... 2025-08-01T13:28:51.972881Z info tf Checking pods ready... 2025-08-01T13:28:51.976568Z info tf [ 0] prometheus-689cc795d4-cs8tw Pending (Pending) 2025-08-01T13:28:55.177167Z info tf Checking pods ready... 2025-08-01T13:28:55.177251Z info tf Checking pods ready... 2025-08-01T13:28:55.181920Z info tf [ 0] prometheus-689cc795d4-cs8tw Running (Ready) 2025-08-01T13:28:55.234090Z info tf === DONE: Setup: 'telemetry_api' (47.03452666s) === 2025-08-01T13:28:55.234143Z info tf === BEGIN: Test Run: 'telemetry_api' === === RUN TestStatsTCPFilter/additional-labels 2025-08-01T13:28:55.235194Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-08-01T13:30:15.246974Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.01177171s)' === 2025-08-01T13:30:15.247095Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.247101Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.247133Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.247169Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.507417Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:28:55Z/119e106a78f-ba85-4d8e-8d9e-ac3bd8218e1f 2025-08-01T13:30:15.507551Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2516s 2025-08-01T13:30:15.508638Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:15.697550Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:28:55Z/119bedf09e0-01b0-4720-b0fe-b15640dc5f17 2025-08-01T13:30:15.698047Z info tf istioctl ([x internal-debug --all configz]): completed after 0.4421s 2025-08-01T13:30:15.698486Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:15.851753Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:28:55Z/119cc02a2b2-a7d1-450b-ada8-ed688eb685c5 2025-08-01T13:30:15.851930Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5960s 2025-08-01T13:30:15.852605Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter/additional-labels (81.20s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter/additional-labels (re-run 1) (81.20s) === RUN TestStatsTCPFilter 2025-08-01T13:28:55.234909Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-08-01T13:30:16.436594Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m21.201669812s)' === 2025-08-01T13:30:16.436760Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-08-01T13:30:16.436769Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-08-01T13:30:16.436786Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-08-01T13:30:16.436812Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-08-01T13:30:16.686967Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/120b4f70d01-cb74-435e-a433-f68f03b33228 2025-08-01T13:30:16.687316Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2500s 2025-08-01T13:30:16.687905Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:16.893910Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/120690d5e28-4876-48a5-ade6-5d60f3081cc8 2025-08-01T13:30:16.894004Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4566s 2025-08-01T13:30:16.894613Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:17.013148Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/12024d4fe8d-1f86-45b8-bd17-2df4f707a88a 2025-08-01T13:30:17.013259Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5759s 2025-08-01T13:30:17.013817Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter (82.62s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter (re-run 1) (82.62s) 2025-08-01T13:30:17.858703Z info tf === FAILED: Test Run: 'telemetry_api' (exitCode: 1) === 2025-08-01T13:30:17.858775Z info tf === Suite "telemetry_api" run time: 2m9.659227056s === 2025-08-01T13:30:17.858795Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/trace.yaml 2025-08-01T13:30:17.861701Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-08-01T13:30:17.861742Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-08-01T13:30:17.861714Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-08-01T13:30:17.861742Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-08-01T13:30:18.159772Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/12083bacddd-3b09-491b-8e50-8b42ab773104 2025-08-01T13:30:18.159990Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2976s 2025-08-01T13:30:18.160924Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:18.334317Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/12041c909e5-09c5-4e18-8a4c-9888649ff5e3 2025-08-01T13:30:18.334525Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4721s 2025-08-01T13:30:18.335141Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:18.455516Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/120121b1d3a-ff82-470e-b31e-a29de0ac8019 2025-08-01T13:30:18.455926Z info tf istioctl ([x internal-debug --all configz]): completed after 0.5935s 2025-08-01T13:30:18.456278Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:19.205199Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-08-01T13:30:19.205313Z info tf === SUCCEEDED: Cleanup Istio in 102.672µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api DONE 2 runs, 39 tests, 6 failures in 543.362s exec: [go test -json -test.run=^TestStatsTCPFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 4111514 2025-08-01T13:30:35.845036Z warn unable to resolve TARGET_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:30:35.845121Z warn unable to resolve LOCAL_OUT. Dir /root/afsan/istio/out/linux_ppc64le does not exist 2025-08-01T13:30:35.878309Z info tf === Test Framework Settings === 2025-08-01T13:30:35.878434Z info tf TestID: telemetry_api RunID: 1753029f-ec1d-46e2-b095-5f85fa6262d4 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-08-01T13:30:35.878443Z info tf =============================== 2025-08-01T13:30:35.878733Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-1753029fec1d46e2b 2025-08-01T13:30:35.878861Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-08-01T13:30:35.878879Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-08-01T13:30:35.878903Z info tf Using KubeConfigs: [/root/upi/cO/auth/kubeconfig]. 2025-08-01T13:30:35.878913Z info tf === BEGIN: Building clusters === 2025-08-01T13:30:35.882710Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cO/auth/kubeconfig 2025-08-01T13:30:35.882721Z info tf === DONE: Building clusters === 2025-08-01T13:30:35.882748Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-08-01T13:30:35.883192Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-08-01T13:30:35.883201Z info tf === Istio Component Config === 2025-08-01T13:30:35.883253Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-08-01T13:30:35.883263Z info tf ================================ 2025-08-01T13:30:35.885877Z info tf skipping deployment as specified in the config 2025-08-01T13:30:35.885890Z info tf === SUCCEEDED: Deploy Istio in 2.694698ms [Suite=telemetry_api]=== 2025-08-01T13:30:35.921237Z info tf === BEGIN: Deploy registry redirector server === 2025-08-01T13:30:35.921301Z info tf === BEGIN: Create namespace registry-redirector === 2025-08-01T13:30:35.930994Z info tf === SUCCEEDED: Create namespace registry-redirector in 9.673518ms === 2025-08-01T13:30:35.976740Z info tf Checking pods ready... 2025-08-01T13:30:35.976763Z info tf Checking pods ready... 2025-08-01T13:30:35.982095Z info tf No pods found... 2025-08-01T13:30:36.182736Z info tf Checking pods ready... 2025-08-01T13:30:36.182777Z info tf Checking pods ready... 2025-08-01T13:30:36.204679Z info tf [ 0] registry-redirector-7f6bd4c5df-prp2w Pending (Pending) 2025-08-01T13:30:36.605612Z info tf Checking pods ready... 2025-08-01T13:30:36.605652Z info tf Checking pods ready... 2025-08-01T13:30:36.610648Z info tf [ 0] registry-redirector-7f6bd4c5df-prp2w Pending (Pending) 2025-08-01T13:30:37.410758Z info tf Checking pods ready... 2025-08-01T13:30:37.410806Z info tf Checking pods ready... 2025-08-01T13:30:37.415499Z info tf [ 0] registry-redirector-7f6bd4c5df-prp2w Pending (Pending) 2025-08-01T13:30:39.016362Z info tf Checking pods ready... 2025-08-01T13:30:39.016416Z info tf Checking pods ready... 2025-08-01T13:30:39.019659Z info tf [ 0] registry-redirector-7f6bd4c5df-prp2w Running (Ready) 2025-08-01T13:30:39.023842Z info tf registry redirector server address: 172.30.199.95:1338 2025-08-01T13:30:39.057329Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-08-01T13:30:39.057419Z info tf === BEGIN: Create namespace external === 2025-08-01T13:30:39.057421Z info tf === BEGIN: Create namespace echo === 2025-08-01T13:30:39.063041Z info tf === SUCCEEDED: Create namespace echo in 5.623069ms === 2025-08-01T13:30:39.063241Z info tf === SUCCEEDED: Create namespace external in 5.831646ms === 2025-08-01T13:30:39.116628Z info tf === BEGIN: Deploy echo instances === 2025-08-01T13:30:39.135674Z error tf === FAILED: Deploy echo instances === 2025-08-01T13:30:39.135703Z error tf failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-1753029fec1d46e2b/_suite_context/Service-ServiceAccount.2774862617.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "a" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-08-01T13:30:39.135750Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-08-01T13:30:39.135774Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-08-01T13:30:39.135803Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-08-01T13:30:39.135828Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-08-01T13:30:39.281680Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:37Z/125fa1d78e8-e777-451b-9a6e-f3ab35d9b8b2 2025-08-01T13:30:39.281908Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.1457s 2025-08-01T13:30:39.282667Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:39.436818Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:37Z/1252b85a253-3f2b-4d9a-a6ae-268f0c10276d 2025-08-01T13:30:39.436994Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3008s 2025-08-01T13:30:39.437765Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:39.528662Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:37Z/125ed1b32c1-1808-4ec7-87fc-11fd61367caf 2025-08-01T13:30:39.529145Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3929s 2025-08-01T13:30:39.529835Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:39.580169Z error tf Test setup error: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-1753029fec1d46e2b/_suite_context/Service-ServiceAccount.2774862617.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "a" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-08-01T13:30:39.580217Z info tf === FAILED: Setup: 'telemetry_api' (failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-1753029fec1d46e2b/_suite_context/Service-ServiceAccount.2774862617.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "a" is forbidden: unable to create new content in namespace echo because it is being terminated) === 2025-08-01T13:30:39.580227Z error tf Exiting due to setup failure: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-1753029fec1d46e2b/_suite_context/Service-ServiceAccount.2774862617.yaml] to ns echo in cluster cluster-0: apply: patch: serviceaccounts "a" is forbidden: unable to create new content in namespace echo because it is being terminated 2025-08-01T13:30:39.580264Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-08-01T13:30:39.580267Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-08-01T13:30:39.580310Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-08-01T13:30:39.580321Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-08-01T13:30:39.753138Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:37Z/1258314e5ce-a685-4ed1-9a00-c0f3aad86e98 2025-08-01T13:30:39.753401Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1729s 2025-08-01T13:30:39.754331Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:39.853896Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:37Z/1253220101f-6e56-4548-914f-944ece485592 2025-08-01T13:30:39.854089Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2735s 2025-08-01T13:30:39.854954Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:39.953283Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:37Z/125dc2d6759-7008-4420-a8d3-8092ca10d0fc 2025-08-01T13:30:39.953410Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3729s 2025-08-01T13:30:39.954339Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:40.166413Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-08-01T13:30:40.166482Z info tf === SUCCEEDED: Cleanup Istio in 63.064µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api === Failed === FAIL: tests/integration/telemetry/api TestStatsFilter/additional-labels (82.47s) 2025-08-01T13:23:55.933742Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 8 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: istio-system/istio-ingressgateway-5754586674-cwnmk) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 4 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 6 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 7 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 5 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-08-01T13:25:15.952440Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.018694093s)' === 2025-08-01T13:25:15.952582Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-08-01T13:25:15.952601Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-08-01T13:25:15.952595Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-08-01T13:25:15.952619Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-08-01T13:25:16.317551Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:23:56Z/9190587996-6e3d-4725-9bff-5b84cc6f81c8 2025-08-01T13:25:16.317906Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3650s 2025-08-01T13:25:16.318382Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:16.554466Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:23:56Z/91da330819-760f-4f2f-9358-62e2d44caa11 2025-08-01T13:25:16.554585Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.6017s 2025-08-01T13:25:16.555342Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:16.820430Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:23:56Z/91d254bf70-7351-4226-9ed0-c4fd1e90f5d7 2025-08-01T13:25:16.820558Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.8676s 2025-08-01T13:25:16.821336Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsFilter (88.06s) 2025-08-01T13:23:53.842655Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-08-01T13:25:18.470187Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m24.627531065s)' === 2025-08-01T13:25:18.470263Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-08-01T13:25:18.470276Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-08-01T13:25:18.470298Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-08-01T13:25:18.470305Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-08-01T13:25:21.063493Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:18Z/927cb6ffd4-d650-4c53-814e-83820b4ec31d 2025-08-01T13:25:21.063621Z info tf istioctl ([x internal-debug --all mcsz]): completed after 2.5929s 2025-08-01T13:25:21.064473Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:21.248654Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:18Z/92d7433f53-5a2a-465b-bb3c-19032ff46301 2025-08-01T13:25:21.248803Z info tf istioctl ([x internal-debug --all clusterz]): completed after 2.7781s 2025-08-01T13:25:21.249556Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:25:21.337016Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:18Z/92fd8294a5-c860-4f54-8dce-df6544fc1ccd 2025-08-01T13:25:21.337338Z info tf istioctl ([x internal-debug --all configz]): completed after 2.8666s 2025-08-01T13:25:21.338047Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter/additional-labels (82.90s) 2025-08-01T13:25:27.987352Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-589885b6f9-z9fqf) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-6f87bc6866-wvwzr) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-08-01T13:26:47.999532Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.012171233s)' === 2025-08-01T13:26:47.999680Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:47.999687Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:47.999671Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:47.999671Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:26:48.317342Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:28Z/93c86d6754-9b61-4656-9d0e-f8f90cd555e4 2025-08-01T13:26:48.317518Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3174s 2025-08-01T13:26:48.318411Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:48.598270Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:28Z/93c6aca51d-b016-4025-ae16-22302cf2065c 2025-08-01T13:26:48.598590Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5984s 2025-08-01T13:26:48.599417Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:48.761648Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:25:28Z/930725d6dd-9e1d-463b-9fa6-79c0e1a45053 2025-08-01T13:26:48.762340Z info tf istioctl ([x internal-debug --all configz]): completed after 0.7621s 2025-08-01T13:26:48.763313Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter (92.05s) 2025-08-01T13:25:21.900161Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-08-01T13:26:50.885196Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m28.985028269s)' === 2025-08-01T13:26:50.885302Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-08-01T13:26:50.885313Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-08-01T13:26:50.885344Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-08-01T13:26:50.885365Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-08-01T13:26:51.265937Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:26:50Z/94c13423be-ae93-44a3-941f-4dca859c7ef5 2025-08-01T13:26:51.266050Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3805s 2025-08-01T13:26:51.266658Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:51.393659Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:26:50Z/94de069d26-e586-43d7-bf40-993557f07056 2025-08-01T13:26:51.393735Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5081s 2025-08-01T13:26:51.394447Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:26:51.549381Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:26:50Z/94a218a84c-a55a-456b-9497-9e177c062a4b 2025-08-01T13:26:51.549830Z info tf istioctl ([x internal-debug --all configz]): completed after 0.6642s 2025-08-01T13:26:51.550498Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter/additional-labels (re-run 1) (81.20s) 2025-08-01T13:28:55.235194Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-b558c7b5f-q7m2g) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-7d694d5b5d-qjjq5) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-08-01T13:30:15.246974Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.01177171s)' === 2025-08-01T13:30:15.247095Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.247101Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.247133Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.247169Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-08-01T13:30:15.507417Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:28:55Z/119e106a78f-ba85-4d8e-8d9e-ac3bd8218e1f 2025-08-01T13:30:15.507551Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2516s 2025-08-01T13:30:15.508638Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:15.697550Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:28:55Z/119bedf09e0-01b0-4720-b0fe-b15640dc5f17 2025-08-01T13:30:15.698047Z info tf istioctl ([x internal-debug --all configz]): completed after 0.4421s 2025-08-01T13:30:15.698486Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:15.851753Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:28:55Z/119cc02a2b2-a7d1-450b-ada8-ed688eb685c5 2025-08-01T13:30:15.851930Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5960s 2025-08-01T13:30:15.852605Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter (re-run 1) (82.62s) 2025-08-01T13:28:55.234909Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-08-01T13:30:16.436594Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m21.201669812s)' === 2025-08-01T13:30:16.436760Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-08-01T13:30:16.436769Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-08-01T13:30:16.436786Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-08-01T13:30:16.436812Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-08-01T13:30:16.686967Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/120b4f70d01-cb74-435e-a433-f68f03b33228 2025-08-01T13:30:16.687316Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2500s 2025-08-01T13:30:16.687905Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:16.893910Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/120690d5e28-4876-48a5-ade6-5d60f3081cc8 2025-08-01T13:30:16.894004Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4566s 2025-08-01T13:30:16.894613Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-08-01T13:30:17.013148Z info adsc Received type=istio.io/debug count=1 nonce=2025-08-01T13:30:16Z/12024d4fe8d-1f86-45b8-bd17-2df4f707a88a 2025-08-01T13:30:17.013259Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.5759s 2025-08-01T13:30:17.013817Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF DONE 3 runs, 39 tests, 6 failures in 564.191s exec: go version