exec: [go test -json -tags=integ -timeout 180m ./tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 404819 2025-12-09T16:39:01.885671Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T16:39:01.885732Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T16:39:01.929566Z info tf === Test Framework Settings === 2025-12-09T16:39:01.929648Z info tf TestID: telemetry_api RunID: d50689c8-26bf-4618-b770-69eb65b33314 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-09T16:39:01.929655Z info tf =============================== 2025-12-09T16:39:01.931027Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-d50689c826bf4618b 2025-12-09T16:39:01.931131Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-09T16:39:01.931146Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-09T16:39:01.931167Z info tf Using KubeConfigs: [/root/upi/cA/auth/kubeconfig]. 2025-12-09T16:39:01.931177Z info tf === BEGIN: Building clusters === 2025-12-09T16:39:01.935150Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cA/auth/kubeconfig 2025-12-09T16:39:01.935191Z info tf === DONE: Building clusters === 2025-12-09T16:39:01.935213Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-12-09T16:39:01.935241Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-12-09T16:39:01.935250Z info tf === Istio Component Config === 2025-12-09T16:39:01.935305Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-09T16:39:01.935313Z info tf ================================ 2025-12-09T16:39:01.938455Z info tf skipping deployment as specified in the config 2025-12-09T16:39:01.938474Z info tf === SUCCEEDED: Deploy Istio in 3.227746ms [Suite=telemetry_api]=== 2025-12-09T16:39:01.963210Z info tf === BEGIN: Deploy registry redirector server === 2025-12-09T16:39:01.963227Z info tf === BEGIN: Create namespace registry-redirector === 2025-12-09T16:39:01.969783Z info tf === SUCCEEDED: Create namespace registry-redirector in 6.549162ms === 2025-12-09T16:39:01.987831Z info tf Checking pods ready... 2025-12-09T16:39:01.987864Z info tf Checking pods ready... 2025-12-09T16:39:01.991463Z info tf [ 0] registry-redirector-7f6bd4c5df-g6bds Running (Ready) 2025-12-09T16:39:01.997020Z info klog Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2025-12-09T16:39:01.997106Z info tf registry redirector server address: 172.30.252.4:1338 2025-12-09T16:39:02.027178Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-12-09T16:39:02.027293Z info tf === BEGIN: Create namespace external === 2025-12-09T16:39:02.027319Z info tf === BEGIN: Create namespace echo === 2025-12-09T16:39:02.060775Z info tf === SUCCEEDED: Create namespace echo in 33.471756ms === 2025-12-09T16:39:02.060803Z info tf === SUCCEEDED: Create namespace external in 33.516863ms === 2025-12-09T16:39:02.220496Z info tf === BEGIN: Deploy echo instances === 2025-12-09T16:39:02.904883Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.909521Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.911876Z info sync complete name=pod controller attempt=3 time=6.929408ms 2025-12-09T16:39:02.917004Z info sync complete name=pod controller attempt=3 time=7.442994ms 2025-12-09T16:39:02.938005Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.939668Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.945723Z info sync complete name=pod controller attempt=3 time=7.646978ms 2025-12-09T16:39:02.946079Z info sync complete name=pod controller attempt=3 time=6.350182ms 2025-12-09T16:39:02.966395Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.971143Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.973095Z info sync complete name=pod controller attempt=3 time=6.651716ms 2025-12-09T16:39:02.977134Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.977313Z info sync complete name=pod controller attempt=3 time=6.125133ms 2025-12-09T16:39:02.982907Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.984243Z info sync complete name=pod controller attempt=3 time=7.070991ms 2025-12-09T16:39:02.985732Z info sync complete name=pod controller attempt=2 time=2.775837ms 2025-12-09T16:39:02.986471Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:02.986530Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:39:03.001794Z info sync complete name=pod controller attempt=4 time=15.27597ms 2025-12-09T16:39:03.001863Z info sync complete name=pod controller attempt=4 time=15.292456ms 2025-12-09T16:39:03.005382Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.009865Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.038702Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.040172Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.066691Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.072278Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.077852Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.083422Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.086576Z info klog Caches are synced controller=echo 2025-12-09T16:39:03.086667Z info klog Caches are synced controller=echo 2025-12-09T16:39:15.792326Z info tf === SUCCEEDED: Deploy echo instances in 13.571824654s === 2025-12-09T16:39:16.038438Z info tf Checking pods ready... 2025-12-09T16:39:16.038529Z info tf Checking pods ready... 2025-12-09T16:39:16.047143Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-12-09T16:39:16.247603Z info tf Checking pods ready... 2025-12-09T16:39:16.247678Z info tf Checking pods ready... 2025-12-09T16:39:16.421794Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-12-09T16:39:16.822582Z info tf Checking pods ready... 2025-12-09T16:39:16.822657Z info tf Checking pods ready... 2025-12-09T16:39:16.831007Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:17.631862Z info tf Checking pods ready... 2025-12-09T16:39:17.631899Z info tf Checking pods ready... 2025-12-09T16:39:17.637591Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:19.237887Z info tf Checking pods ready... 2025-12-09T16:39:19.237959Z info tf Checking pods ready... 2025-12-09T16:39:19.241933Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:22.442450Z info tf Checking pods ready... 2025-12-09T16:39:22.442487Z info tf Checking pods ready... 2025-12-09T16:39:22.447959Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:25.648335Z info tf Checking pods ready... 2025-12-09T16:39:25.648369Z info tf Checking pods ready... 2025-12-09T16:39:25.653368Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:28.853954Z info tf Checking pods ready... 2025-12-09T16:39:28.854006Z info tf Checking pods ready... 2025-12-09T16:39:28.859515Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:32.060325Z info tf Checking pods ready... 2025-12-09T16:39:32.060348Z info tf Checking pods ready... 2025-12-09T16:39:32.064517Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:35.265151Z info tf Checking pods ready... 2025-12-09T16:39:35.265208Z info tf Checking pods ready... 2025-12-09T16:39:35.271384Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:38.472512Z info tf Checking pods ready... 2025-12-09T16:39:38.472557Z info tf Checking pods ready... 2025-12-09T16:39:38.477140Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:41.678193Z info tf Checking pods ready... 2025-12-09T16:39:41.678217Z info tf Checking pods ready... 2025-12-09T16:39:41.682889Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:44.883874Z info tf Checking pods ready... 2025-12-09T16:39:44.883923Z info tf Checking pods ready... 2025-12-09T16:39:44.888668Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:48.088944Z info tf Checking pods ready... 2025-12-09T16:39:48.088990Z info tf Checking pods ready... 2025-12-09T16:39:48.094951Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:51.296070Z info tf Checking pods ready... 2025-12-09T16:39:51.296106Z info tf Checking pods ready... 2025-12-09T16:39:51.299929Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:54.500683Z info tf Checking pods ready... 2025-12-09T16:39:54.500718Z info tf Checking pods ready... 2025-12-09T16:39:54.504764Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:39:57.705754Z info tf Checking pods ready... 2025-12-09T16:39:57.705781Z info tf Checking pods ready... 2025-12-09T16:39:57.710208Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:00.911129Z info tf Checking pods ready... 2025-12-09T16:40:00.911152Z info tf Checking pods ready... 2025-12-09T16:40:00.929771Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:04.130839Z info tf Checking pods ready... 2025-12-09T16:40:04.130871Z info tf Checking pods ready... 2025-12-09T16:40:04.135699Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:07.336236Z info tf Checking pods ready... 2025-12-09T16:40:07.336283Z info tf Checking pods ready... 2025-12-09T16:40:07.340623Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:10.540734Z info tf Checking pods ready... 2025-12-09T16:40:10.540794Z info tf Checking pods ready... 2025-12-09T16:40:10.548101Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:13.748848Z info tf Checking pods ready... 2025-12-09T16:40:13.748888Z info tf Checking pods ready... 2025-12-09T16:40:13.774729Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:16.975756Z info tf Checking pods ready... 2025-12-09T16:40:16.975792Z info tf Checking pods ready... 2025-12-09T16:40:16.980011Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:20.180457Z info tf Checking pods ready... 2025-12-09T16:40:20.180495Z info tf Checking pods ready... 2025-12-09T16:40:20.185091Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:23.385537Z info tf Checking pods ready... 2025-12-09T16:40:23.385580Z info tf Checking pods ready... 2025-12-09T16:40:23.393519Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:26.594112Z info tf Checking pods ready... 2025-12-09T16:40:26.594140Z info tf Checking pods ready... 2025-12-09T16:40:26.598535Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:29.799294Z info tf Checking pods ready... 2025-12-09T16:40:29.799333Z info tf Checking pods ready... 2025-12-09T16:40:29.804144Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:33.004924Z info tf Checking pods ready... 2025-12-09T16:40:33.004965Z info tf Checking pods ready... 2025-12-09T16:40:33.009079Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:36.210183Z info tf Checking pods ready... 2025-12-09T16:40:36.210228Z info tf Checking pods ready... 2025-12-09T16:40:36.214707Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:39.415129Z info tf Checking pods ready... 2025-12-09T16:40:39.415164Z info tf Checking pods ready... 2025-12-09T16:40:39.420081Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:42.621112Z info tf Checking pods ready... 2025-12-09T16:40:42.621152Z info tf Checking pods ready... 2025-12-09T16:40:42.625994Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:45.827041Z info tf Checking pods ready... 2025-12-09T16:40:45.827084Z info tf Checking pods ready... 2025-12-09T16:40:45.831511Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:49.032064Z info tf Checking pods ready... 2025-12-09T16:40:49.032101Z info tf Checking pods ready... 2025-12-09T16:40:49.036119Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:52.236199Z info tf Checking pods ready... 2025-12-09T16:40:52.236242Z info tf Checking pods ready... 2025-12-09T16:40:52.242308Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:55.443328Z info tf Checking pods ready... 2025-12-09T16:40:55.443359Z info tf Checking pods ready... 2025-12-09T16:40:55.447438Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:40:58.648506Z info tf Checking pods ready... 2025-12-09T16:40:58.648551Z info tf Checking pods ready... 2025-12-09T16:40:58.652693Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:01.853044Z info tf Checking pods ready... 2025-12-09T16:41:01.853082Z info tf Checking pods ready... 2025-12-09T16:41:01.857098Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:05.057755Z info tf Checking pods ready... 2025-12-09T16:41:05.057801Z info tf Checking pods ready... 2025-12-09T16:41:05.062172Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:08.262585Z info tf Checking pods ready... 2025-12-09T16:41:08.262618Z info tf Checking pods ready... 2025-12-09T16:41:08.278471Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:11.479009Z info tf Checking pods ready... 2025-12-09T16:41:11.479036Z info tf Checking pods ready... 2025-12-09T16:41:11.483448Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:14.684430Z info tf Checking pods ready... 2025-12-09T16:41:14.684466Z info tf Checking pods ready... 2025-12-09T16:41:14.688370Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:17.888743Z info tf Checking pods ready... 2025-12-09T16:41:17.888832Z info tf Checking pods ready... 2025-12-09T16:41:17.892901Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:21.093160Z info tf Checking pods ready... 2025-12-09T16:41:21.093186Z info tf Checking pods ready... 2025-12-09T16:41:21.097632Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:24.298073Z info tf Checking pods ready... 2025-12-09T16:41:24.298104Z info tf Checking pods ready... 2025-12-09T16:41:24.302252Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:27.502496Z info tf Checking pods ready... 2025-12-09T16:41:27.502570Z info tf Checking pods ready... 2025-12-09T16:41:27.507702Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:30.708025Z info tf Checking pods ready... 2025-12-09T16:41:30.708060Z info tf Checking pods ready... 2025-12-09T16:41:30.712333Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:33.913436Z info tf Checking pods ready... 2025-12-09T16:41:33.913463Z info tf Checking pods ready... 2025-12-09T16:41:33.917791Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:37.118547Z info tf Checking pods ready... 2025-12-09T16:41:37.118586Z info tf Checking pods ready... 2025-12-09T16:41:37.122609Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:40.323340Z info tf Checking pods ready... 2025-12-09T16:41:40.323383Z info tf Checking pods ready... 2025-12-09T16:41:40.327776Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:43.528181Z info tf Checking pods ready... 2025-12-09T16:41:43.528221Z info tf Checking pods ready... 2025-12-09T16:41:43.532979Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:46.733669Z info tf Checking pods ready... 2025-12-09T16:41:46.733712Z info tf Checking pods ready... 2025-12-09T16:41:46.738265Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:49.938431Z info tf Checking pods ready... 2025-12-09T16:41:49.938476Z info tf Checking pods ready... 2025-12-09T16:41:49.943089Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:53.143501Z info tf Checking pods ready... 2025-12-09T16:41:53.143552Z info tf Checking pods ready... 2025-12-09T16:41:53.149474Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:56.350203Z info tf Checking pods ready... 2025-12-09T16:41:56.350253Z info tf Checking pods ready... 2025-12-09T16:41:56.354405Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:41:59.554542Z info tf Checking pods ready... 2025-12-09T16:41:59.554583Z info tf Checking pods ready... 2025-12-09T16:41:59.559322Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:42:02.759427Z info tf Checking pods ready... 2025-12-09T16:42:02.759454Z info tf Checking pods ready... 2025-12-09T16:42:02.763206Z info tf [ 0] prometheus-575cb4fbc6-phdfb Pending (Pending) 2025-12-09T16:42:05.963957Z info tf Checking pods ready... 2025-12-09T16:42:05.963990Z info tf Checking pods ready... 2025-12-09T16:42:05.967825Z info tf [ 0] prometheus-575cb4fbc6-phdfb Running (Ready) 2025-12-09T16:42:06.006511Z info tf === DONE: Setup: 'telemetry_api' (3m4.071275807s) === 2025-12-09T16:42:06.006587Z info tf === BEGIN: Test Run: 'telemetry_api' === PASS tests/integration/telemetry/api.TestAccessLogs/enabled (0.71s) PASS tests/integration/telemetry/api.TestAccessLogs/enabled-with-targetref (6.30s) PASS tests/integration/telemetry/api.TestAccessLogs/disabled (2.10s) PASS tests/integration/telemetry/api.TestAccessLogs (9.10s) PASS tests/integration/telemetry/api.TestAccessLogsFilter (4.10s) PASS tests/integration/telemetry/api.TestAccessLogsMode/client (11.28s) PASS tests/integration/telemetry/api.TestAccessLogsMode/server (11.30s) PASS tests/integration/telemetry/api.TestAccessLogsMode/client-and-server (11.26s) PASS tests/integration/telemetry/api.TestAccessLogsMode (33.85s) PASS tests/integration/telemetry/api.TestAccessLogsDefaultProvider/disabled (6.20s) PASS tests/integration/telemetry/api.TestAccessLogsDefaultProvider/enabled (1.74s) PASS tests/integration/telemetry/api.TestAccessLogsDefaultProvider (7.94s) PASS tests/integration/telemetry/api.TestAccessLogWithFilterState (0.99s) PASS tests/integration/telemetry/api.TestDashboard/istio-mesh-dashboard.json (0.04s) PASS tests/integration/telemetry/api.TestDashboard/istio-service-dashboard.json (0.22s) PASS tests/integration/telemetry/api.TestDashboard/istio-workload-dashboard.json (0.21s) PASS tests/integration/telemetry/api.TestDashboard/istio-performance-dashboard.json (10.70s) PASS tests/integration/telemetry/api.TestDashboard (11.41s) PASS tests/integration/telemetry/api.TestIstioctlMetrics (0.09s) PASS tests/integration/telemetry/api.TestStatsFilter/default (2.10s) === RUN TestStatsFilter/additional-labels 2025-12-09T16:43:15.597918Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 11 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 5 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-12-09T16:44:35.614667Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.016745189s)' === 2025-12-09T16:44:35.614749Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.614749Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.614803Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.614795Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.797890Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:43:15Z/29393b9b15-0b84-4f62-9568-a9c1daec3c12 2025-12-09T16:44:35.798141Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1831s 2025-12-09T16:44:35.798959Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:35.889166Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:43:15Z/2911f95d32-a6bf-4ae6-92b6-8971ac337cc0 2025-12-09T16:44:35.889276Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2742s 2025-12-09T16:44:35.889924Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:35.987595Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:43:15Z/2973f57d51-e95f-440b-8b48-e5eaf872ea3e 2025-12-09T16:44:35.987714Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3726s 2025-12-09T16:44:35.988483Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter/additional-labels (81.58s) FAIL tests/integration/telemetry/api.TestStatsFilter/additional-labels (81.58s) PASS tests/integration/telemetry/api.TestStatsFilter/mockprom-to-metrics (0.07s) === RUN TestStatsFilter 2025-12-09T16:43:13.495018Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-12-09T16:44:37.247888Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m23.752867376s)' === 2025-12-09T16:44:37.247949Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-12-09T16:44:37.247961Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-12-09T16:44:37.247975Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-12-09T16:44:37.248010Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-12-09T16:44:37.429705Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:37Z/303c2b93e1-cff5-4203-82c8-5c02a9d1ec72 2025-12-09T16:44:37.430025Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1817s 2025-12-09T16:44:37.430711Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:37.540796Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:37Z/301302f5f6-deea-42d4-83ff-8fedad1bac28 2025-12-09T16:44:37.540874Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2925s 2025-12-09T16:44:37.541866Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:37.641815Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:37Z/3097c0b9ef-73f0-4082-b2b5-3776247b1c2d 2025-12-09T16:44:37.641988Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3936s 2025-12-09T16:44:37.642804Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter (85.49s) FAIL tests/integration/telemetry/api.TestStatsFilter (85.49s) PASS tests/integration/telemetry/api.TestStatsTCPFilter/default (4.07s) === RUN TestStatsTCPFilter/additional-labels 2025-12-09T16:44:43.054436Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-12-09T16:46:03.065261Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.010816268s)' === 2025-12-09T16:46:03.065421Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.065429Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.065474Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.065459Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.251583Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:43Z/31c3483293-2677-4b88-b14b-3de54faa4ff9 2025-12-09T16:46:03.251724Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1854s 2025-12-09T16:46:03.252717Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:03.351312Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:43Z/3115cf439b-9f5a-47a8-9cbf-aa84d9ae9864 2025-12-09T16:46:03.351575Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2852s 2025-12-09T16:46:03.352257Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:03.439039Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:43Z/31b04e27e6-bee6-4ab2-9ea1-bbfb93295b4e 2025-12-09T16:46:03.439123Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3727s 2025-12-09T16:46:03.440160Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter/additional-labels (81.77s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter/additional-labels (81.77s) === RUN TestStatsTCPFilter 2025-12-09T16:44:38.987769Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-12-09T16:46:04.827347Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m25.839570821s)' === 2025-12-09T16:46:04.827479Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-12-09T16:46:04.827503Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-12-09T16:46:04.827522Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-12-09T16:46:04.827490Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-12-09T16:46:04.994448Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:04Z/32261ef53d-0e00-4e2b-86d1-2f87f9004527 2025-12-09T16:46:04.994742Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1670s 2025-12-09T16:46:04.995719Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:05.098758Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:04Z/32468f3851-f4ee-473d-880f-4ab8b7048891 2025-12-09T16:46:05.098950Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2712s 2025-12-09T16:46:05.099794Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:05.212126Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:04Z/325e3ab13c-2982-4032-b703-d7c611dcd971 2025-12-09T16:46:05.212309Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3846s 2025-12-09T16:46:05.213455Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter (87.75s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter (87.75s) PASS tests/integration/telemetry/api.TestGRPCCountMetrics/istio_request_messages_total (1.12s) PASS tests/integration/telemetry/api.TestGRPCCountMetrics/istio_response_messages_total (0.04s) PASS tests/integration/telemetry/api.TestGRPCCountMetrics (1.16s) PASS tests/integration/telemetry/api.TestImagePullPolicy/Delete_WasmPlugin_wasm-test-module (0.03s) PASS tests/integration/telemetry/api.TestImagePullPolicy/Delete_WasmPlugin_wasm-test-module#01 (0.03s) PASS tests/integration/telemetry/api.TestImagePullPolicy (0.06s) PASS tests/integration/telemetry/api.TestGatewaySelection/Delete_WasmPlugin_wasm-test-module (0.02s) PASS tests/integration/telemetry/api.TestGatewaySelection (0.08s) PASS tests/integration/telemetry/api.TestImagePullPolicyWithHTTP (0.00s) PASS tests/integration/telemetry/api.TestBadWasmRemoteLoad (7.25s) PASS tests/integration/telemetry/api.TestBadWasmWithFailOpen (20.35s) 2025-12-09T16:46:35.642978Z info tf === FAILED: Test Run: 'telemetry_api' (exitCode: 1) === 2025-12-09T16:46:35.643012Z info tf === Suite "telemetry_api" run time: 7m33.707801709s === 2025-12-09T16:46:35.643024Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/trace.yaml 2025-12-09T16:46:35.644075Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-12-09T16:46:35.644083Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-12-09T16:46:35.644085Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-12-09T16:46:35.644106Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-12-09T16:46:35.816197Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:35Z/36a5eede3d-1787-4e14-a786-66ddfa956117 2025-12-09T16:46:35.816399Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1719s 2025-12-09T16:46:35.817149Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:35.928802Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:35Z/365ed9a9cf-42a6-454d-b2e3-7131811c6cc1 2025-12-09T16:46:35.929003Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2845s 2025-12-09T16:46:35.929602Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:36.033498Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:35Z/362da7f8dd-45cc-44af-884d-a91c17855ea7 2025-12-09T16:46:36.033579Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3890s 2025-12-09T16:46:36.034234Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:37.718161Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-12-09T16:46:37.718295Z info tf === SUCCEEDED: Cleanup Istio in 122.612µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api DONE 37 tests, 4 failures in 467.835s exec: [go test -json -test.run=^TestStatsFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 407258 2025-12-09T16:46:50.420692Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T16:46:50.420749Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T16:46:50.452331Z info tf === Test Framework Settings === 2025-12-09T16:46:50.452414Z info tf TestID: telemetry_api RunID: 9ed0b8c7-942a-4d53-bbbd-f3ed24c13fd4 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-09T16:46:50.452419Z info tf =============================== 2025-12-09T16:46:50.452743Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-9ed0b8c7942a4d53b 2025-12-09T16:46:50.452834Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-09T16:46:50.452847Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-09T16:46:50.452868Z info tf Using KubeConfigs: [/root/upi/cA/auth/kubeconfig]. 2025-12-09T16:46:50.452876Z info tf === BEGIN: Building clusters === 2025-12-09T16:46:50.455837Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cA/auth/kubeconfig 2025-12-09T16:46:50.455846Z info tf === DONE: Building clusters === 2025-12-09T16:46:50.455863Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-12-09T16:46:50.455885Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-12-09T16:46:50.455893Z info tf === Istio Component Config === 2025-12-09T16:46:50.455927Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-09T16:46:50.455931Z info tf ================================ 2025-12-09T16:46:50.458418Z info tf skipping deployment as specified in the config 2025-12-09T16:46:50.458431Z info tf === SUCCEEDED: Deploy Istio in 2.541904ms [Suite=telemetry_api]=== 2025-12-09T16:46:50.478254Z info tf === BEGIN: Deploy registry redirector server === 2025-12-09T16:46:50.478318Z info tf === BEGIN: Create namespace registry-redirector === 2025-12-09T16:46:50.486232Z info tf === SUCCEEDED: Create namespace registry-redirector in 7.910774ms === 2025-12-09T16:46:50.538358Z info tf Checking pods ready... 2025-12-09T16:46:50.538381Z info tf Checking pods ready... 2025-12-09T16:46:50.552726Z info tf No pods found... 2025-12-09T16:46:50.753421Z info tf Checking pods ready... 2025-12-09T16:46:50.753453Z info tf Checking pods ready... 2025-12-09T16:46:50.767218Z info tf [ 0] registry-redirector-7f6bd4c5df-qxb94 Pending (Pending) 2025-12-09T16:46:51.168069Z info tf Checking pods ready... 2025-12-09T16:46:51.168136Z info tf Checking pods ready... 2025-12-09T16:46:51.207072Z info tf [ 0] registry-redirector-7f6bd4c5df-qxb94 Pending (Pending) 2025-12-09T16:46:52.007253Z info tf Checking pods ready... 2025-12-09T16:46:52.007332Z info tf Checking pods ready... 2025-12-09T16:46:52.017294Z info tf [ 0] registry-redirector-7f6bd4c5df-qxb94 Pending (Pending) 2025-12-09T16:46:53.618258Z info tf Checking pods ready... 2025-12-09T16:46:53.618302Z info tf Checking pods ready... 2025-12-09T16:46:53.630694Z info tf [ 0] registry-redirector-7f6bd4c5df-qxb94 Pending (Pending) 2025-12-09T16:46:56.831458Z info tf Checking pods ready... 2025-12-09T16:46:56.831530Z info tf Checking pods ready... 2025-12-09T16:46:56.838822Z info tf [ 0] registry-redirector-7f6bd4c5df-qxb94 Running (Ready) 2025-12-09T16:46:56.851763Z info klog Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2025-12-09T16:46:56.851820Z info tf registry redirector server address: 172.30.155.61:1338 2025-12-09T16:46:56.884895Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-12-09T16:46:56.885022Z info tf === BEGIN: Create namespace external === 2025-12-09T16:46:56.885022Z info tf === BEGIN: Create namespace echo === 2025-12-09T16:46:56.900795Z info tf === SUCCEEDED: Create namespace echo in 15.775675ms === 2025-12-09T16:46:56.915813Z info tf === SUCCEEDED: Create namespace external in 30.796239ms === 2025-12-09T16:46:57.090030Z info tf === BEGIN: Deploy echo instances === 2025-12-09T16:46:58.013375Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.020898Z info sync complete name=pod controller attempt=3 time=7.418704ms 2025-12-09T16:46:58.036985Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.039676Z info sync complete name=pod controller attempt=2 time=2.589084ms 2025-12-09T16:46:58.047998Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.050527Z info sync complete name=pod controller attempt=2 time=2.476526ms 2025-12-09T16:46:58.086909Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.087277Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.088999Z info sync complete name=pod controller attempt=2 time=2.055338ms 2025-12-09T16:46:58.089648Z info sync complete name=pod controller attempt=2 time=2.344805ms 2025-12-09T16:46:58.093475Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.093710Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.095478Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.095623Z info sync complete name=pod controller attempt=2 time=2.089464ms 2025-12-09T16:46:58.095827Z info sync complete name=pod controller attempt=2 time=2.080119ms 2025-12-09T16:46:58.098064Z info sync complete name=pod controller attempt=2 time=2.530853ms 2025-12-09T16:46:58.099460Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.102100Z info sync complete name=pod controller attempt=2 time=2.604546ms 2025-12-09T16:46:58.114079Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.121224Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:46:58.124085Z info sync complete name=pod controller attempt=2 time=2.810942ms 2025-12-09T16:46:58.137542Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.149057Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.187760Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.187761Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.194432Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.194406Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.195850Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.200324Z info klog Caches are synced controller=echo 2025-12-09T16:46:58.221981Z info klog Caches are synced controller=echo 2025-12-09T16:47:10.928274Z info tf === SUCCEEDED: Deploy echo instances in 13.838239153s === 2025-12-09T16:47:11.217937Z info tf Checking pods ready... 2025-12-09T16:47:11.218028Z info tf Checking pods ready... 2025-12-09T16:47:11.223482Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-12-09T16:47:11.424228Z info tf Checking pods ready... 2025-12-09T16:47:11.424293Z info tf Checking pods ready... 2025-12-09T16:47:11.428523Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:11.829485Z info tf Checking pods ready... 2025-12-09T16:47:11.829535Z info tf Checking pods ready... 2025-12-09T16:47:11.834383Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:12.634560Z info tf Checking pods ready... 2025-12-09T16:47:12.634612Z info tf Checking pods ready... 2025-12-09T16:47:12.640421Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:14.240983Z info tf Checking pods ready... 2025-12-09T16:47:14.241038Z info tf Checking pods ready... 2025-12-09T16:47:14.245374Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:17.445730Z info tf Checking pods ready... 2025-12-09T16:47:17.445788Z info tf Checking pods ready... 2025-12-09T16:47:17.451202Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:20.651591Z info tf Checking pods ready... 2025-12-09T16:47:20.651671Z info tf Checking pods ready... 2025-12-09T16:47:20.656472Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:23.856937Z info tf Checking pods ready... 2025-12-09T16:47:23.856981Z info tf Checking pods ready... 2025-12-09T16:47:23.868234Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:27.068693Z info tf Checking pods ready... 2025-12-09T16:47:27.068744Z info tf Checking pods ready... 2025-12-09T16:47:27.073708Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:30.274729Z info tf Checking pods ready... 2025-12-09T16:47:30.274756Z info tf Checking pods ready... 2025-12-09T16:47:30.279729Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:33.480325Z info tf Checking pods ready... 2025-12-09T16:47:33.480383Z info tf Checking pods ready... 2025-12-09T16:47:33.484735Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:36.685417Z info tf Checking pods ready... 2025-12-09T16:47:36.685457Z info tf Checking pods ready... 2025-12-09T16:47:36.690221Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:39.891255Z info tf Checking pods ready... 2025-12-09T16:47:39.891293Z info tf Checking pods ready... 2025-12-09T16:47:39.895481Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:43.095665Z info tf Checking pods ready... 2025-12-09T16:47:43.095695Z info tf Checking pods ready... 2025-12-09T16:47:43.099533Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:46.299834Z info tf Checking pods ready... 2025-12-09T16:47:46.299884Z info tf Checking pods ready... 2025-12-09T16:47:46.305115Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:49.505589Z info tf Checking pods ready... 2025-12-09T16:47:49.505620Z info tf Checking pods ready... 2025-12-09T16:47:49.509729Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:52.710376Z info tf Checking pods ready... 2025-12-09T16:47:52.710433Z info tf Checking pods ready... 2025-12-09T16:47:52.715567Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:55.916326Z info tf Checking pods ready... 2025-12-09T16:47:55.916378Z info tf Checking pods ready... 2025-12-09T16:47:55.921036Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:47:59.121770Z info tf Checking pods ready... 2025-12-09T16:47:59.121880Z info tf Checking pods ready... 2025-12-09T16:47:59.127366Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:02.328481Z info tf Checking pods ready... 2025-12-09T16:48:02.328519Z info tf Checking pods ready... 2025-12-09T16:48:02.333258Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:05.533841Z info tf Checking pods ready... 2025-12-09T16:48:05.533916Z info tf Checking pods ready... 2025-12-09T16:48:05.538529Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:08.739046Z info tf Checking pods ready... 2025-12-09T16:48:08.739101Z info tf Checking pods ready... 2025-12-09T16:48:08.745356Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:11.945789Z info tf Checking pods ready... 2025-12-09T16:48:11.945829Z info tf Checking pods ready... 2025-12-09T16:48:11.950030Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:15.150564Z info tf Checking pods ready... 2025-12-09T16:48:15.150603Z info tf Checking pods ready... 2025-12-09T16:48:15.154350Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:18.355076Z info tf Checking pods ready... 2025-12-09T16:48:18.355106Z info tf Checking pods ready... 2025-12-09T16:48:18.366691Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:21.566778Z info tf Checking pods ready... 2025-12-09T16:48:21.566904Z info tf Checking pods ready... 2025-12-09T16:48:21.571613Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:24.772481Z info tf Checking pods ready... 2025-12-09T16:48:24.772541Z info tf Checking pods ready... 2025-12-09T16:48:24.776857Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:27.977573Z info tf Checking pods ready... 2025-12-09T16:48:27.977652Z info tf Checking pods ready... 2025-12-09T16:48:27.987571Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:31.188275Z info tf Checking pods ready... 2025-12-09T16:48:31.188343Z info tf Checking pods ready... 2025-12-09T16:48:31.192833Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:34.393288Z info tf Checking pods ready... 2025-12-09T16:48:34.393330Z info tf Checking pods ready... 2025-12-09T16:48:34.398240Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:37.598825Z info tf Checking pods ready... 2025-12-09T16:48:37.598874Z info tf Checking pods ready... 2025-12-09T16:48:37.603818Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:40.804903Z info tf Checking pods ready... 2025-12-09T16:48:40.804952Z info tf Checking pods ready... 2025-12-09T16:48:40.812749Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:44.013411Z info tf Checking pods ready... 2025-12-09T16:48:44.013448Z info tf Checking pods ready... 2025-12-09T16:48:44.018431Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:47.218567Z info tf Checking pods ready... 2025-12-09T16:48:47.218640Z info tf Checking pods ready... 2025-12-09T16:48:47.223589Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:50.424375Z info tf Checking pods ready... 2025-12-09T16:48:50.424442Z info tf Checking pods ready... 2025-12-09T16:48:50.429091Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:53.630194Z info tf Checking pods ready... 2025-12-09T16:48:53.630250Z info tf Checking pods ready... 2025-12-09T16:48:53.642229Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:48:56.842709Z info tf Checking pods ready... 2025-12-09T16:48:56.842757Z info tf Checking pods ready... 2025-12-09T16:48:56.847028Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:00.047677Z info tf Checking pods ready... 2025-12-09T16:49:00.047724Z info tf Checking pods ready... 2025-12-09T16:49:00.051871Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:03.251985Z info tf Checking pods ready... 2025-12-09T16:49:03.252031Z info tf Checking pods ready... 2025-12-09T16:49:03.256628Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:06.457414Z info tf Checking pods ready... 2025-12-09T16:49:06.457469Z info tf Checking pods ready... 2025-12-09T16:49:06.462083Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:09.662655Z info tf Checking pods ready... 2025-12-09T16:49:09.662699Z info tf Checking pods ready... 2025-12-09T16:49:09.666806Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:12.867289Z info tf Checking pods ready... 2025-12-09T16:49:12.867390Z info tf Checking pods ready... 2025-12-09T16:49:12.871873Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:16.072890Z info tf Checking pods ready... 2025-12-09T16:49:16.072987Z info tf Checking pods ready... 2025-12-09T16:49:16.078086Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:19.278827Z info tf Checking pods ready... 2025-12-09T16:49:19.278867Z info tf Checking pods ready... 2025-12-09T16:49:19.284137Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:22.484840Z info tf Checking pods ready... 2025-12-09T16:49:22.484917Z info tf Checking pods ready... 2025-12-09T16:49:22.489790Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:25.689976Z info tf Checking pods ready... 2025-12-09T16:49:25.690019Z info tf Checking pods ready... 2025-12-09T16:49:25.693871Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:28.894846Z info tf Checking pods ready... 2025-12-09T16:49:28.894878Z info tf Checking pods ready... 2025-12-09T16:49:28.899109Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:32.099952Z info tf Checking pods ready... 2025-12-09T16:49:32.100003Z info tf Checking pods ready... 2025-12-09T16:49:32.116611Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:35.317394Z info tf Checking pods ready... 2025-12-09T16:49:35.317446Z info tf Checking pods ready... 2025-12-09T16:49:35.322430Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:38.522782Z info tf Checking pods ready... 2025-12-09T16:49:38.522826Z info tf Checking pods ready... 2025-12-09T16:49:38.527183Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:41.728142Z info tf Checking pods ready... 2025-12-09T16:49:41.728189Z info tf Checking pods ready... 2025-12-09T16:49:42.664327Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:45.865065Z info tf Checking pods ready... 2025-12-09T16:49:45.865105Z info tf Checking pods ready... 2025-12-09T16:49:45.871308Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:49.072125Z info tf Checking pods ready... 2025-12-09T16:49:49.072160Z info tf Checking pods ready... 2025-12-09T16:49:49.076374Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:52.277256Z info tf Checking pods ready... 2025-12-09T16:49:52.277284Z info tf Checking pods ready... 2025-12-09T16:49:52.281781Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:55.482138Z info tf Checking pods ready... 2025-12-09T16:49:55.482168Z info tf Checking pods ready... 2025-12-09T16:49:55.486164Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:49:58.687040Z info tf Checking pods ready... 2025-12-09T16:49:58.687093Z info tf Checking pods ready... 2025-12-09T16:49:58.691669Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:01.892065Z info tf Checking pods ready... 2025-12-09T16:50:01.892109Z info tf Checking pods ready... 2025-12-09T16:50:01.896356Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:05.096834Z info tf Checking pods ready... 2025-12-09T16:50:05.096864Z info tf Checking pods ready... 2025-12-09T16:50:05.101259Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:08.301624Z info tf Checking pods ready... 2025-12-09T16:50:08.301654Z info tf Checking pods ready... 2025-12-09T16:50:08.307657Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:11.507958Z info tf Checking pods ready... 2025-12-09T16:50:11.508004Z info tf Checking pods ready... 2025-12-09T16:50:11.512392Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:14.712527Z info tf Checking pods ready... 2025-12-09T16:50:14.712572Z info tf Checking pods ready... 2025-12-09T16:50:14.717077Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:17.917865Z info tf Checking pods ready... 2025-12-09T16:50:17.917899Z info tf Checking pods ready... 2025-12-09T16:50:17.921945Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:21.122383Z info tf Checking pods ready... 2025-12-09T16:50:21.122422Z info tf Checking pods ready... 2025-12-09T16:50:21.126571Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:24.327008Z info tf Checking pods ready... 2025-12-09T16:50:24.327046Z info tf Checking pods ready... 2025-12-09T16:50:24.331924Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:27.532190Z info tf Checking pods ready... 2025-12-09T16:50:27.532232Z info tf Checking pods ready... 2025-12-09T16:50:27.536965Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:30.737658Z info tf Checking pods ready... 2025-12-09T16:50:30.737694Z info tf Checking pods ready... 2025-12-09T16:50:30.741554Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:33.942581Z info tf Checking pods ready... 2025-12-09T16:50:33.942611Z info tf Checking pods ready... 2025-12-09T16:50:33.947246Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:37.148241Z info tf Checking pods ready... 2025-12-09T16:50:37.148284Z info tf Checking pods ready... 2025-12-09T16:50:37.152156Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:40.353167Z info tf Checking pods ready... 2025-12-09T16:50:40.353200Z info tf Checking pods ready... 2025-12-09T16:50:40.357309Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:43.558393Z info tf Checking pods ready... 2025-12-09T16:50:43.558435Z info tf Checking pods ready... 2025-12-09T16:50:43.563164Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:46.763554Z info tf Checking pods ready... 2025-12-09T16:50:46.763633Z info tf Checking pods ready... 2025-12-09T16:50:46.769339Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:49.969984Z info tf Checking pods ready... 2025-12-09T16:50:49.970031Z info tf Checking pods ready... 2025-12-09T16:50:49.975985Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:53.176464Z info tf Checking pods ready... 2025-12-09T16:50:53.176518Z info tf Checking pods ready... 2025-12-09T16:50:53.181522Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:56.381989Z info tf Checking pods ready... 2025-12-09T16:50:56.382031Z info tf Checking pods ready... 2025-12-09T16:50:56.386812Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:50:59.587402Z info tf Checking pods ready... 2025-12-09T16:50:59.587449Z info tf Checking pods ready... 2025-12-09T16:50:59.591711Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:02.792230Z info tf Checking pods ready... 2025-12-09T16:51:02.792280Z info tf Checking pods ready... 2025-12-09T16:51:02.796488Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:05.997255Z info tf Checking pods ready... 2025-12-09T16:51:05.997290Z info tf Checking pods ready... 2025-12-09T16:51:06.001230Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:09.201552Z info tf Checking pods ready... 2025-12-09T16:51:09.201593Z info tf Checking pods ready... 2025-12-09T16:51:09.205598Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:12.406557Z info tf Checking pods ready... 2025-12-09T16:51:12.406688Z info tf Checking pods ready... 2025-12-09T16:51:12.410392Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:15.611290Z info tf Checking pods ready... 2025-12-09T16:51:15.611330Z info tf Checking pods ready... 2025-12-09T16:51:15.615441Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:18.815887Z info tf Checking pods ready... 2025-12-09T16:51:18.815919Z info tf Checking pods ready... 2025-12-09T16:51:18.821020Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:22.021828Z info tf Checking pods ready... 2025-12-09T16:51:22.021907Z info tf Checking pods ready... 2025-12-09T16:51:22.027922Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:25.229044Z info tf Checking pods ready... 2025-12-09T16:51:25.229097Z info tf Checking pods ready... 2025-12-09T16:51:25.234602Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:28.435452Z info tf Checking pods ready... 2025-12-09T16:51:28.435512Z info tf Checking pods ready... 2025-12-09T16:51:28.440923Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:31.641624Z info tf Checking pods ready... 2025-12-09T16:51:31.641680Z info tf Checking pods ready... 2025-12-09T16:51:31.646385Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:34.846790Z info tf Checking pods ready... 2025-12-09T16:51:34.846846Z info tf Checking pods ready... 2025-12-09T16:51:34.852553Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:38.052994Z info tf Checking pods ready... 2025-12-09T16:51:38.053044Z info tf Checking pods ready... 2025-12-09T16:51:38.057993Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:41.258425Z info tf Checking pods ready... 2025-12-09T16:51:41.258482Z info tf Checking pods ready... 2025-12-09T16:51:41.278663Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:44.478978Z info tf Checking pods ready... 2025-12-09T16:51:44.479056Z info tf Checking pods ready... 2025-12-09T16:51:44.484659Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:47.685574Z info tf Checking pods ready... 2025-12-09T16:51:47.685621Z info tf Checking pods ready... 2025-12-09T16:51:47.692353Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:50.893392Z info tf Checking pods ready... 2025-12-09T16:51:50.893446Z info tf Checking pods ready... 2025-12-09T16:51:50.898424Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:54.098820Z info tf Checking pods ready... 2025-12-09T16:51:54.098893Z info tf Checking pods ready... 2025-12-09T16:51:54.104583Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:51:57.305209Z info tf Checking pods ready... 2025-12-09T16:51:57.305277Z info tf Checking pods ready... 2025-12-09T16:51:57.311778Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:00.512190Z info tf Checking pods ready... 2025-12-09T16:52:00.512262Z info tf Checking pods ready... 2025-12-09T16:52:00.516770Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:03.716845Z info tf Checking pods ready... 2025-12-09T16:52:03.716885Z info tf Checking pods ready... 2025-12-09T16:52:03.722086Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:06.922391Z info tf Checking pods ready... 2025-12-09T16:52:06.922441Z info tf Checking pods ready... 2025-12-09T16:52:06.926423Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:10.127031Z info tf Checking pods ready... 2025-12-09T16:52:10.127062Z info tf Checking pods ready... 2025-12-09T16:52:10.131022Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:13.331304Z info tf Checking pods ready... 2025-12-09T16:52:13.331332Z info tf Checking pods ready... 2025-12-09T16:52:13.335098Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:16.535666Z info tf Checking pods ready... 2025-12-09T16:52:16.535715Z info tf Checking pods ready... 2025-12-09T16:52:16.540672Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:19.740813Z info tf Checking pods ready... 2025-12-09T16:52:19.740862Z info tf Checking pods ready... 2025-12-09T16:52:19.746158Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:22.946360Z info tf Checking pods ready... 2025-12-09T16:52:22.946420Z info tf Checking pods ready... 2025-12-09T16:52:22.959038Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:26.159725Z info tf Checking pods ready... 2025-12-09T16:52:26.159781Z info tf Checking pods ready... 2025-12-09T16:52:26.165194Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:29.366186Z info tf Checking pods ready... 2025-12-09T16:52:29.366226Z info tf Checking pods ready... 2025-12-09T16:52:29.371013Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:32.571479Z info tf Checking pods ready... 2025-12-09T16:52:32.571543Z info tf Checking pods ready... 2025-12-09T16:52:32.576558Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:35.776815Z info tf Checking pods ready... 2025-12-09T16:52:35.776847Z info tf Checking pods ready... 2025-12-09T16:52:35.781371Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:38.981815Z info tf Checking pods ready... 2025-12-09T16:52:38.981867Z info tf Checking pods ready... 2025-12-09T16:52:38.986909Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:42.187568Z info tf Checking pods ready... 2025-12-09T16:52:42.187645Z info tf Checking pods ready... 2025-12-09T16:52:42.192642Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:45.393685Z info tf Checking pods ready... 2025-12-09T16:52:45.393723Z info tf Checking pods ready... 2025-12-09T16:52:45.400336Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:48.600603Z info tf Checking pods ready... 2025-12-09T16:52:48.600648Z info tf Checking pods ready... 2025-12-09T16:52:48.606528Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:51.807528Z info tf Checking pods ready... 2025-12-09T16:52:51.807575Z info tf Checking pods ready... 2025-12-09T16:52:51.812552Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:55.012935Z info tf Checking pods ready... 2025-12-09T16:52:55.013026Z info tf Checking pods ready... 2025-12-09T16:52:55.018745Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:52:58.219694Z info tf Checking pods ready... 2025-12-09T16:52:58.219761Z info tf Checking pods ready... 2025-12-09T16:52:58.225375Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:01.425500Z info tf Checking pods ready... 2025-12-09T16:53:01.425546Z info tf Checking pods ready... 2025-12-09T16:53:01.430571Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:04.630947Z info tf Checking pods ready... 2025-12-09T16:53:04.630996Z info tf Checking pods ready... 2025-12-09T16:53:04.635461Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:07.835587Z info tf Checking pods ready... 2025-12-09T16:53:07.835648Z info tf Checking pods ready... 2025-12-09T16:53:07.841344Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:11.042191Z info tf Checking pods ready... 2025-12-09T16:53:11.042242Z info tf Checking pods ready... 2025-12-09T16:53:11.047096Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:14.247413Z info tf Checking pods ready... 2025-12-09T16:53:14.247534Z info tf Checking pods ready... 2025-12-09T16:53:14.252668Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:17.452779Z info tf Checking pods ready... 2025-12-09T16:53:17.452855Z info tf Checking pods ready... 2025-12-09T16:53:17.458321Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:20.658630Z info tf Checking pods ready... 2025-12-09T16:53:20.658697Z info tf Checking pods ready... 2025-12-09T16:53:20.669472Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:23.870076Z info tf Checking pods ready... 2025-12-09T16:53:23.870132Z info tf Checking pods ready... 2025-12-09T16:53:23.875816Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:27.076263Z info tf Checking pods ready... 2025-12-09T16:53:27.076323Z info tf Checking pods ready... 2025-12-09T16:53:27.082254Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:30.283065Z info tf Checking pods ready... 2025-12-09T16:53:30.283159Z info tf Checking pods ready... 2025-12-09T16:53:30.289947Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:33.490823Z info tf Checking pods ready... 2025-12-09T16:53:33.490902Z info tf Checking pods ready... 2025-12-09T16:53:33.496533Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:36.697724Z info tf Checking pods ready... 2025-12-09T16:53:36.697794Z info tf Checking pods ready... 2025-12-09T16:53:36.703554Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:39.903810Z info tf Checking pods ready... 2025-12-09T16:53:39.903864Z info tf Checking pods ready... 2025-12-09T16:53:39.911611Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:43.112130Z info tf Checking pods ready... 2025-12-09T16:53:43.112204Z info tf Checking pods ready... 2025-12-09T16:53:43.117192Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:46.317728Z info tf Checking pods ready... 2025-12-09T16:53:46.317775Z info tf Checking pods ready... 2025-12-09T16:53:46.334287Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:49.535059Z info tf Checking pods ready... 2025-12-09T16:53:49.535111Z info tf Checking pods ready... 2025-12-09T16:53:49.539983Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:52.741126Z info tf Checking pods ready... 2025-12-09T16:53:52.741199Z info tf Checking pods ready... 2025-12-09T16:53:52.746907Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:55.947971Z info tf Checking pods ready... 2025-12-09T16:53:55.948001Z info tf Checking pods ready... 2025-12-09T16:53:55.953642Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:53:59.154748Z info tf Checking pods ready... 2025-12-09T16:53:59.154812Z info tf Checking pods ready... 2025-12-09T16:53:59.161233Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:02.361398Z info tf Checking pods ready... 2025-12-09T16:54:02.361438Z info tf Checking pods ready... 2025-12-09T16:54:02.367476Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:05.568267Z info tf Checking pods ready... 2025-12-09T16:54:05.568336Z info tf Checking pods ready... 2025-12-09T16:54:05.573832Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:08.774764Z info tf Checking pods ready... 2025-12-09T16:54:08.774817Z info tf Checking pods ready... 2025-12-09T16:54:08.780558Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:11.981544Z info tf Checking pods ready... 2025-12-09T16:54:11.981599Z info tf Checking pods ready... 2025-12-09T16:54:11.988131Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:15.188645Z info tf Checking pods ready... 2025-12-09T16:54:15.188685Z info tf Checking pods ready... 2025-12-09T16:54:15.192914Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:18.393612Z info tf Checking pods ready... 2025-12-09T16:54:18.393695Z info tf Checking pods ready... 2025-12-09T16:54:18.400214Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:21.601231Z info tf Checking pods ready... 2025-12-09T16:54:21.601286Z info tf Checking pods ready... 2025-12-09T16:54:21.606363Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:24.806847Z info tf Checking pods ready... 2025-12-09T16:54:24.806908Z info tf Checking pods ready... 2025-12-09T16:54:24.812920Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:28.013800Z info tf Checking pods ready... 2025-12-09T16:54:28.013859Z info tf Checking pods ready... 2025-12-09T16:54:28.018802Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:31.219115Z info tf Checking pods ready... 2025-12-09T16:54:31.219165Z info tf Checking pods ready... 2025-12-09T16:54:31.224949Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:34.426009Z info tf Checking pods ready... 2025-12-09T16:54:34.426047Z info tf Checking pods ready... 2025-12-09T16:54:34.430692Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:37.631343Z info tf Checking pods ready... 2025-12-09T16:54:37.631396Z info tf Checking pods ready... 2025-12-09T16:54:37.637554Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:40.838004Z info tf Checking pods ready... 2025-12-09T16:54:40.838064Z info tf Checking pods ready... 2025-12-09T16:54:40.842952Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:44.043045Z info tf Checking pods ready... 2025-12-09T16:54:44.043118Z info tf Checking pods ready... 2025-12-09T16:54:44.047749Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:47.248443Z info tf Checking pods ready... 2025-12-09T16:54:47.248497Z info tf Checking pods ready... 2025-12-09T16:54:47.256234Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:50.456869Z info tf Checking pods ready... 2025-12-09T16:54:50.456919Z info tf Checking pods ready... 2025-12-09T16:54:50.462622Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:53.663157Z info tf Checking pods ready... 2025-12-09T16:54:53.663238Z info tf Checking pods ready... 2025-12-09T16:54:53.668909Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:54:56.869926Z info tf Checking pods ready... 2025-12-09T16:54:56.870001Z info tf Checking pods ready... 2025-12-09T16:54:56.875489Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:00.076010Z info tf Checking pods ready... 2025-12-09T16:55:00.076066Z info tf Checking pods ready... 2025-12-09T16:55:00.082731Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:03.282776Z info tf Checking pods ready... 2025-12-09T16:55:03.282825Z info tf Checking pods ready... 2025-12-09T16:55:03.287927Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:06.488664Z info tf Checking pods ready... 2025-12-09T16:55:06.488718Z info tf Checking pods ready... 2025-12-09T16:55:06.494061Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:09.694438Z info tf Checking pods ready... 2025-12-09T16:55:09.694512Z info tf Checking pods ready... 2025-12-09T16:55:09.700400Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:12.901551Z info tf Checking pods ready... 2025-12-09T16:55:12.901658Z info tf Checking pods ready... 2025-12-09T16:55:12.908183Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:16.108560Z info tf Checking pods ready... 2025-12-09T16:55:16.108660Z info tf Checking pods ready... 2025-12-09T16:55:16.113357Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:19.313677Z info tf Checking pods ready... 2025-12-09T16:55:19.313727Z info tf Checking pods ready... 2025-12-09T16:55:19.317990Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:22.518649Z info tf Checking pods ready... 2025-12-09T16:55:22.518688Z info tf Checking pods ready... 2025-12-09T16:55:22.524486Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:25.725572Z info tf Checking pods ready... 2025-12-09T16:55:25.725629Z info tf Checking pods ready... 2025-12-09T16:55:25.732446Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:28.933219Z info tf Checking pods ready... 2025-12-09T16:55:28.933258Z info tf Checking pods ready... 2025-12-09T16:55:28.937732Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:32.138125Z info tf Checking pods ready... 2025-12-09T16:55:32.138193Z info tf Checking pods ready... 2025-12-09T16:55:32.142622Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:35.342920Z info tf Checking pods ready... 2025-12-09T16:55:35.342949Z info tf Checking pods ready... 2025-12-09T16:55:35.348680Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:38.549725Z info tf Checking pods ready... 2025-12-09T16:55:38.549765Z info tf Checking pods ready... 2025-12-09T16:55:38.554957Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:41.755675Z info tf Checking pods ready... 2025-12-09T16:55:41.755711Z info tf Checking pods ready... 2025-12-09T16:55:41.763139Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:44.964238Z info tf Checking pods ready... 2025-12-09T16:55:44.964272Z info tf Checking pods ready... 2025-12-09T16:55:44.970461Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:48.171041Z info tf Checking pods ready... 2025-12-09T16:55:48.171119Z info tf Checking pods ready... 2025-12-09T16:55:48.176512Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:51.377287Z info tf Checking pods ready... 2025-12-09T16:55:51.377325Z info tf Checking pods ready... 2025-12-09T16:55:51.389670Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:54.590754Z info tf Checking pods ready... 2025-12-09T16:55:54.590787Z info tf Checking pods ready... 2025-12-09T16:55:54.595264Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:55:57.795854Z info tf Checking pods ready... 2025-12-09T16:55:57.795899Z info tf Checking pods ready... 2025-12-09T16:55:57.800531Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:01.001623Z info tf Checking pods ready... 2025-12-09T16:56:01.001698Z info tf Checking pods ready... 2025-12-09T16:56:01.007771Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:04.208510Z info tf Checking pods ready... 2025-12-09T16:56:04.208562Z info tf Checking pods ready... 2025-12-09T16:56:04.214026Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:07.415035Z info tf Checking pods ready... 2025-12-09T16:56:07.415079Z info tf Checking pods ready... 2025-12-09T16:56:07.420328Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:10.620968Z info tf Checking pods ready... 2025-12-09T16:56:10.621008Z info tf Checking pods ready... 2025-12-09T16:56:10.625805Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:13.826241Z info tf Checking pods ready... 2025-12-09T16:56:13.826273Z info tf Checking pods ready... 2025-12-09T16:56:13.831719Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:17.032040Z info tf Checking pods ready... 2025-12-09T16:56:17.032093Z info tf Checking pods ready... 2025-12-09T16:56:17.037895Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:20.238283Z info tf Checking pods ready... 2025-12-09T16:56:20.238321Z info tf Checking pods ready... 2025-12-09T16:56:20.244549Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:23.445234Z info tf Checking pods ready... 2025-12-09T16:56:23.445272Z info tf Checking pods ready... 2025-12-09T16:56:23.451332Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:26.652275Z info tf Checking pods ready... 2025-12-09T16:56:26.652312Z info tf Checking pods ready... 2025-12-09T16:56:26.658034Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:29.858687Z info tf Checking pods ready... 2025-12-09T16:56:29.858721Z info tf Checking pods ready... 2025-12-09T16:56:29.863594Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:33.064752Z info tf Checking pods ready... 2025-12-09T16:56:33.064802Z info tf Checking pods ready... 2025-12-09T16:56:33.069679Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:36.270273Z info tf Checking pods ready... 2025-12-09T16:56:36.270332Z info tf Checking pods ready... 2025-12-09T16:56:36.275606Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:39.475686Z info tf Checking pods ready... 2025-12-09T16:56:39.475728Z info tf Checking pods ready... 2025-12-09T16:56:39.481325Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:42.681864Z info tf Checking pods ready... 2025-12-09T16:56:42.681906Z info tf Checking pods ready... 2025-12-09T16:56:42.687328Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:45.888205Z info tf Checking pods ready... 2025-12-09T16:56:45.888238Z info tf Checking pods ready... 2025-12-09T16:56:45.892601Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:49.093270Z info tf Checking pods ready... 2025-12-09T16:56:49.093310Z info tf Checking pods ready... 2025-12-09T16:56:49.099578Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:52.300607Z info tf Checking pods ready... 2025-12-09T16:56:52.300637Z info tf Checking pods ready... 2025-12-09T16:56:52.306464Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:55.507087Z info tf Checking pods ready... 2025-12-09T16:56:55.507134Z info tf Checking pods ready... 2025-12-09T16:56:55.512694Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:56:58.713178Z info tf Checking pods ready... 2025-12-09T16:56:58.713225Z info tf Checking pods ready... 2025-12-09T16:56:58.718331Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:57:01.919269Z info tf Checking pods ready... 2025-12-09T16:57:01.919313Z info tf Checking pods ready... 2025-12-09T16:57:01.924332Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:57:05.125148Z info tf Checking pods ready... 2025-12-09T16:57:05.125187Z info tf Checking pods ready... 2025-12-09T16:57:05.131047Z info tf [ 0] prometheus-575cb4fbc6-9ckln Pending (Pending) 2025-12-09T16:57:08.332116Z info tf Checking pods ready... 2025-12-09T16:57:08.332175Z info tf Checking pods ready... 2025-12-09T16:57:08.337959Z info tf [ 0] prometheus-575cb4fbc6-9ckln Running (Ready) 2025-12-09T16:57:08.390465Z info tf === DONE: Setup: 'telemetry_api' (10m17.934591s) === 2025-12-09T16:57:08.390500Z info tf === BEGIN: Test Run: 'telemetry_api' === === RUN TestStatsFilter/additional-labels 2025-12-09T16:57:08.397366Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-12-09T16:58:28.422774Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.0254034s)' === 2025-12-09T16:58:28.422886Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.422889Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.422951Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.422957Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.681325Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:57:08Z/58669fc8df-1cb7-499a-a857-d92fd2317877 2025-12-09T16:58:28.681579Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2580s 2025-12-09T16:58:28.682356Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:28.808583Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:57:08Z/58e42d44de-288e-40ee-91a4-4143795505fa 2025-12-09T16:58:28.808708Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3851s 2025-12-09T16:58:28.809568Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:28.912494Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:57:08Z/584b94c17d-ea47-4951-adf3-8a4e4d0ed33e 2025-12-09T16:58:28.912654Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4890s 2025-12-09T16:58:28.913399Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter/additional-labels (81.18s) FAIL tests/integration/telemetry/api.TestStatsFilter/additional-labels (re-run 1) (81.18s) === RUN TestStatsFilter 2025-12-09T16:57:08.397130Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-12-09T16:58:29.573276Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m21.176139062s)' === 2025-12-09T16:58:29.573471Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-12-09T16:58:29.573471Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-12-09T16:58:29.573478Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-12-09T16:58:29.573478Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-12-09T16:58:29.764191Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/596c5936da-c8d2-421a-95b7-cf9d13fbe6a1 2025-12-09T16:58:29.764399Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1906s 2025-12-09T16:58:29.764944Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:29.861439Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/599f1a828b-b36c-4381-8651-b6dd9c0f05b4 2025-12-09T16:58:29.861557Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2877s 2025-12-09T16:58:29.862515Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:29.963068Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/59298444e7-0c76-432b-a7e1-2dd53c8d700e 2025-12-09T16:58:29.963236Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3893s 2025-12-09T16:58:29.964082Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter (82.39s) FAIL tests/integration/telemetry/api.TestStatsFilter (re-run 1) (82.39s) 2025-12-09T16:58:30.784671Z info tf === FAILED: Test Run: 'telemetry_api' (exitCode: 1) === 2025-12-09T16:58:30.784720Z info tf === Suite "telemetry_api" run time: 11m40.328859405s === 2025-12-09T16:58:30.784732Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/trace.yaml 2025-12-09T16:58:30.786863Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-12-09T16:58:30.786894Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-12-09T16:58:30.786879Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-12-09T16:58:30.786914Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-12-09T16:58:30.950773Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/592db081bc-3ab1-4719-af98-be48c46f2f1d 2025-12-09T16:58:30.951100Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1640s 2025-12-09T16:58:30.951553Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:31.057384Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/59730aa5b6-f0f5-4532-9e6b-dc363fd9a576 2025-12-09T16:58:31.057528Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2704s 2025-12-09T16:58:31.058468Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:31.160764Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/594c705641-75e1-4968-bdd8-40b39796eaa5 2025-12-09T16:58:31.160926Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3738s 2025-12-09T16:58:31.161646Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:32.316847Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-12-09T16:58:32.316940Z info tf === SUCCEEDED: Cleanup Istio in 86.286µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api exec: [go test -json -test.run=^TestStatsTCPFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 409920 2025-12-09T16:58:44.186621Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T16:58:44.186674Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T16:58:44.218520Z info tf === Test Framework Settings === 2025-12-09T16:58:44.218593Z info tf TestID: telemetry_api RunID: ec8fe8be-6353-4224-a3a8-4f866c73e0a8 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-09T16:58:44.218599Z info tf =============================== 2025-12-09T16:58:44.218777Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-ec8fe8be63534224a 2025-12-09T16:58:44.218867Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-09T16:58:44.218877Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-09T16:58:44.218894Z info tf Using KubeConfigs: [/root/upi/cA/auth/kubeconfig]. 2025-12-09T16:58:44.218902Z info tf === BEGIN: Building clusters === 2025-12-09T16:58:44.221874Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cA/auth/kubeconfig 2025-12-09T16:58:44.221882Z info tf === DONE: Building clusters === 2025-12-09T16:58:44.221896Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-12-09T16:58:44.221916Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-12-09T16:58:44.221923Z info tf === Istio Component Config === 2025-12-09T16:58:44.221952Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-09T16:58:44.221957Z info tf ================================ 2025-12-09T16:58:44.224646Z info tf skipping deployment as specified in the config 2025-12-09T16:58:44.224659Z info tf === SUCCEEDED: Deploy Istio in 2.739981ms [Suite=telemetry_api]=== 2025-12-09T16:58:44.245914Z info tf === BEGIN: Deploy registry redirector server === 2025-12-09T16:58:44.245931Z info tf === BEGIN: Create namespace registry-redirector === 2025-12-09T16:58:44.254616Z info tf === SUCCEEDED: Create namespace registry-redirector in 8.675311ms === 2025-12-09T16:58:44.350621Z info tf Checking pods ready... 2025-12-09T16:58:44.350669Z info tf Checking pods ready... 2025-12-09T16:58:44.357466Z info tf No pods found... 2025-12-09T16:58:44.558101Z info tf Checking pods ready... 2025-12-09T16:58:44.558163Z info tf Checking pods ready... 2025-12-09T16:58:44.576091Z info tf [ 0] registry-redirector-7f6bd4c5df-lj5gd Pending (Pending) 2025-12-09T16:58:44.977030Z info tf Checking pods ready... 2025-12-09T16:58:44.977079Z info tf Checking pods ready... 2025-12-09T16:58:44.984591Z info tf [ 0] registry-redirector-7f6bd4c5df-lj5gd Pending (Pending) 2025-12-09T16:58:45.784702Z info tf Checking pods ready... 2025-12-09T16:58:45.784752Z info tf Checking pods ready... 2025-12-09T16:58:45.792252Z info tf [ 0] registry-redirector-7f6bd4c5df-lj5gd Pending (Pending) 2025-12-09T16:58:47.393122Z info tf Checking pods ready... 2025-12-09T16:58:47.393188Z info tf Checking pods ready... 2025-12-09T16:58:47.411593Z info tf [ 0] registry-redirector-7f6bd4c5df-lj5gd Pending (Pending) 2025-12-09T16:58:50.612337Z info tf Checking pods ready... 2025-12-09T16:58:50.612417Z info tf Checking pods ready... 2025-12-09T16:58:50.621408Z info tf [ 0] registry-redirector-7f6bd4c5df-lj5gd Running (Ready) 2025-12-09T16:58:50.634415Z info klog Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2025-12-09T16:58:50.634475Z info tf registry redirector server address: 172.30.24.19:1338 2025-12-09T16:58:50.672457Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-12-09T16:58:50.672560Z info tf === BEGIN: Create namespace external === 2025-12-09T16:58:50.672587Z info tf === BEGIN: Create namespace echo === 2025-12-09T16:58:50.694232Z info tf === SUCCEEDED: Create namespace echo in 21.652252ms === 2025-12-09T16:58:50.709202Z info tf === SUCCEEDED: Create namespace external in 36.612363ms === 2025-12-09T16:58:50.861419Z info tf === BEGIN: Deploy echo instances === 2025-12-09T16:58:51.833634Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.839897Z info sync complete name=pod controller attempt=3 time=6.143306ms 2025-12-09T16:58:51.918459Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.923229Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.924949Z info sync complete name=pod controller attempt=3 time=6.411939ms 2025-12-09T16:58:51.926466Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.928562Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.928550Z info sync complete name=pod controller attempt=2 time=2.057595ms 2025-12-09T16:58:51.929799Z info sync complete name=pod controller attempt=3 time=6.495338ms 2025-12-09T16:58:51.930515Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.930619Z info sync complete name=pod controller attempt=2 time=2.028279ms 2025-12-09T16:58:51.931628Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.933678Z info sync complete name=pod controller attempt=2 time=2.018665ms 2025-12-09T16:58:51.933827Z info klog Caches are synced controller=echo 2025-12-09T16:58:51.937115Z info sync complete name=pod controller attempt=3 time=6.557185ms 2025-12-09T16:58:51.945609Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.952961Z info sync complete name=pod controller attempt=3 time=7.319516ms 2025-12-09T16:58:51.953521Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.960416Z info sync complete name=pod controller attempt=3 time=6.842863ms 2025-12-09T16:58:51.966555Z info klog Waiting for caches to sync controller=echo 2025-12-09T16:58:51.968742Z info sync complete name=pod controller attempt=2 time=2.148933ms 2025-12-09T16:58:52.019579Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.024227Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.026613Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.028989Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.031326Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.032615Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.045970Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.054598Z info klog Caches are synced controller=echo 2025-12-09T16:58:52.067269Z info klog Caches are synced controller=echo 2025-12-09T16:59:12.755206Z info tf === SUCCEEDED: Deploy echo instances in 21.893734259s === 2025-12-09T16:59:13.054605Z info tf Checking pods ready... 2025-12-09T16:59:13.054683Z info tf Checking pods ready... 2025-12-09T16:59:13.067818Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-12-09T16:59:13.268522Z info tf Checking pods ready... 2025-12-09T16:59:13.268613Z info tf Checking pods ready... 2025-12-09T16:59:13.274617Z info tf [ 0] prometheus-575cb4fbc6-64jrh Pending (Pending) 2025-12-09T16:59:13.675129Z info tf Checking pods ready... 2025-12-09T16:59:13.675194Z info tf Checking pods ready... 2025-12-09T16:59:13.679656Z info tf [ 0] prometheus-575cb4fbc6-64jrh Pending (Pending) 2025-12-09T16:59:14.480314Z info tf Checking pods ready... 2025-12-09T16:59:14.480393Z info tf Checking pods ready... 2025-12-09T16:59:14.487165Z info tf [ 0] prometheus-575cb4fbc6-64jrh Pending (Pending) 2025-12-09T16:59:16.087713Z info tf Checking pods ready... 2025-12-09T16:59:16.087796Z info tf Checking pods ready... 2025-12-09T16:59:16.096277Z info tf [ 0] prometheus-575cb4fbc6-64jrh Running (Ready) 2025-12-09T16:59:16.138475Z info tf === DONE: Setup: 'telemetry_api' (31.916566108s) === 2025-12-09T16:59:16.138507Z info tf === BEGIN: Test Run: 'telemetry_api' === === RUN TestStatsTCPFilter/additional-labels 2025-12-09T16:59:16.139390Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-12-09T17:00:36.152634Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.013238516s)' === 2025-12-09T17:00:36.152772Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.152796Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.152808Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.152833Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.341127Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:59:16Z/806e85e992-1294-42e1-8570-f2f048d69666 2025-12-09T17:00:36.341335Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1793s 2025-12-09T17:00:36.342262Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:36.435711Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:59:16Z/80e33c1dd5-3010-4a4b-aa73-45053fb05761 2025-12-09T17:00:36.435820Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2738s 2025-12-09T17:00:36.436731Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:36.530699Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:59:16Z/80b66f56f3-e16a-49f4-8c30-a9e4193cde89 2025-12-09T17:00:36.530780Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3687s 2025-12-09T17:00:36.531521Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter/additional-labels (80.68s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter/additional-labels (re-run 1) (80.68s) === RUN TestStatsTCPFilter 2025-12-09T16:59:16.139147Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-12-09T17:00:36.820960Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m20.681805759s)' === 2025-12-09T17:00:36.821066Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-12-09T17:00:36.821087Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-12-09T17:00:36.821072Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-12-09T17:00:36.821107Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-12-09T17:00:37.030720Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/813fe188b8-5d2b-48d6-97a2-bd919d004772 2025-12-09T17:00:37.031051Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2097s 2025-12-09T17:00:37.031916Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:37.153226Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/815516ce58-2fa2-4280-a3b5-e37e61d3ffde 2025-12-09T17:00:37.153346Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3319s 2025-12-09T17:00:37.154206Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:37.257601Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/8104bceb60-b4d7-4b9c-ad38-7a9e2ff56ca9 2025-12-09T17:00:37.257765Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4363s 2025-12-09T17:00:37.258374Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsTCPFilter (81.33s) FAIL tests/integration/telemetry/api.TestStatsTCPFilter (re-run 1) (81.33s) 2025-12-09T17:00:37.465645Z info tf === FAILED: Test Run: 'telemetry_api' (exitCode: 1) === 2025-12-09T17:00:37.465720Z info tf === Suite "telemetry_api" run time: 1m53.243822503s === 2025-12-09T17:00:37.465733Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/trace.yaml 2025-12-09T17:00:37.468112Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-12-09T17:00:37.468121Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-12-09T17:00:37.468127Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-12-09T17:00:37.468145Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-12-09T17:00:37.646254Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/81ada2b59f-09a4-4786-90f8-eaef24e2a58b 2025-12-09T17:00:37.646551Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1770s 2025-12-09T17:00:37.647297Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:37.760585Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/8151a60a71-8835-4f78-bb88-fcef42f617d5 2025-12-09T17:00:37.760716Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2911s 2025-12-09T17:00:37.761529Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:37.867001Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/81babd9e54-de64-4f40-8a58-fcf210721586 2025-12-09T17:00:37.867105Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3975s 2025-12-09T17:00:37.867978Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:38.578108Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-12-09T17:00:38.578241Z info tf === SUCCEEDED: Cleanup Istio in 123.305µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api DONE 2 runs, 41 tests, 8 failures in 1308.744s exec: [go test -json -test.run=^TestStatsFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 411838 2025-12-09T17:00:55.109137Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T17:00:55.109860Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T17:00:55.141778Z info tf === Test Framework Settings === 2025-12-09T17:00:55.141889Z info tf TestID: telemetry_api RunID: 3029a9fd-d621-412c-84ba-a93b9fea55ae NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-09T17:00:55.141896Z info tf =============================== 2025-12-09T17:00:55.142608Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-3029a9fdd621412c8 2025-12-09T17:00:55.142691Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-09T17:00:55.142704Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-09T17:00:55.142721Z info tf Using KubeConfigs: [/root/upi/cA/auth/kubeconfig]. 2025-12-09T17:00:55.142729Z info tf === BEGIN: Building clusters === 2025-12-09T17:00:55.146883Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cA/auth/kubeconfig 2025-12-09T17:00:55.146932Z info tf === DONE: Building clusters === 2025-12-09T17:00:55.146966Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-12-09T17:00:55.147643Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-12-09T17:00:55.147657Z info tf === Istio Component Config === 2025-12-09T17:00:55.147705Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-09T17:00:55.147712Z info tf ================================ 2025-12-09T17:00:55.151088Z info tf skipping deployment as specified in the config 2025-12-09T17:00:55.151105Z info tf === SUCCEEDED: Deploy Istio in 3.457306ms [Suite=telemetry_api]=== 2025-12-09T17:00:55.175631Z info tf === BEGIN: Deploy registry redirector server === 2025-12-09T17:00:55.175673Z info tf === BEGIN: Create namespace registry-redirector === 2025-12-09T17:00:55.186116Z info tf === SUCCEEDED: Create namespace registry-redirector in 10.43659ms === 2025-12-09T17:00:55.267778Z info tf Checking pods ready... 2025-12-09T17:00:55.267861Z info tf Checking pods ready... 2025-12-09T17:00:55.299729Z info tf No pods found... 2025-12-09T17:00:55.500587Z info tf Checking pods ready... 2025-12-09T17:00:55.500688Z info tf Checking pods ready... 2025-12-09T17:00:55.506982Z info tf [ 0] registry-redirector-7f6bd4c5df-5c9mm Pending (Pending) 2025-12-09T17:00:55.907825Z info tf Checking pods ready... 2025-12-09T17:00:55.907876Z info tf Checking pods ready... 2025-12-09T17:00:55.912826Z info tf [ 0] registry-redirector-7f6bd4c5df-5c9mm Pending (Pending) 2025-12-09T17:00:56.713738Z info tf Checking pods ready... 2025-12-09T17:00:56.713852Z info tf Checking pods ready... 2025-12-09T17:00:56.719263Z info tf [ 0] registry-redirector-7f6bd4c5df-5c9mm Pending (Pending) 2025-12-09T17:00:58.320060Z info tf Checking pods ready... 2025-12-09T17:00:58.320136Z info tf Checking pods ready... 2025-12-09T17:00:58.329887Z info tf [ 0] registry-redirector-7f6bd4c5df-5c9mm Running (Ready) 2025-12-09T17:00:58.341802Z info klog Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2025-12-09T17:00:58.341981Z info tf registry redirector server address: 172.30.205.22:1338 2025-12-09T17:00:58.390923Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-12-09T17:00:58.391081Z info tf === BEGIN: Create namespace external === 2025-12-09T17:00:58.391096Z info tf === BEGIN: Create namespace echo === 2025-12-09T17:00:58.413131Z info tf === SUCCEEDED: Create namespace external in 22.057316ms === 2025-12-09T17:00:58.423132Z info tf === SUCCEEDED: Create namespace echo in 32.035763ms === 2025-12-09T17:00:58.616158Z info tf === BEGIN: Deploy echo instances === 2025-12-09T17:00:59.536662Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.544048Z info sync complete name=pod controller attempt=3 time=7.298851ms 2025-12-09T17:00:59.593780Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.596803Z info sync complete name=pod controller attempt=2 time=2.941751ms 2025-12-09T17:00:59.604733Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.611289Z info sync complete name=pod controller attempt=3 time=6.487447ms 2025-12-09T17:00:59.615451Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.622108Z info sync complete name=pod controller attempt=3 time=6.623809ms 2025-12-09T17:00:59.633499Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.636352Z info sync complete name=pod controller attempt=2 time=2.797273ms 2025-12-09T17:00:59.636823Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.637998Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.640732Z info sync complete name=pod controller attempt=2 time=2.70646ms 2025-12-09T17:00:59.645742Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.652845Z info sync complete name=pod controller attempt=3 time=7.054865ms 2025-12-09T17:00:59.653421Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.660220Z info sync complete name=pod controller attempt=3 time=6.752041ms 2025-12-09T17:00:59.663915Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.670936Z info sync complete name=pod controller attempt=3 time=6.99095ms 2025-12-09T17:00:59.673601Z info klog Waiting for caches to sync controller=echo 2025-12-09T17:00:59.694466Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.705129Z info sync complete name=pod controller attempt=5 time=31.494526ms 2025-12-09T17:00:59.705193Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.715840Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.734255Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.738996Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.746477Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.753963Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.764462Z info klog Caches are synced controller=echo 2025-12-09T17:00:59.774273Z info klog Caches are synced controller=echo 2025-12-09T17:01:15.653542Z info tf === SUCCEEDED: Deploy echo instances in 17.037372532s === 2025-12-09T17:01:15.889934Z info tf Checking pods ready... 2025-12-09T17:01:15.889998Z info tf Checking pods ready... 2025-12-09T17:01:15.894587Z info tf Failed retrieving pods: no matching pod found for selectors: [app.kubernetes.io/name=prometheus] 2025-12-09T17:01:16.095461Z info tf Checking pods ready... 2025-12-09T17:01:16.095568Z info tf Checking pods ready... 2025-12-09T17:01:16.104124Z info tf [ 0] prometheus-575cb4fbc6-x8mkj Pending (Pending) 2025-12-09T17:01:16.504534Z info tf Checking pods ready... 2025-12-09T17:01:16.504601Z info tf Checking pods ready... 2025-12-09T17:01:16.509576Z info tf [ 0] prometheus-575cb4fbc6-x8mkj Pending (Pending) 2025-12-09T17:01:17.310672Z info tf Checking pods ready... 2025-12-09T17:01:17.310738Z info tf Checking pods ready... 2025-12-09T17:01:17.316780Z info tf [ 0] prometheus-575cb4fbc6-x8mkj Pending (Pending) 2025-12-09T17:01:18.917054Z info tf Checking pods ready... 2025-12-09T17:01:18.917123Z info tf Checking pods ready... 2025-12-09T17:01:18.922538Z info tf [ 0] prometheus-575cb4fbc6-x8mkj Running (container not ready: 'prometheus-server') 2025-12-09T17:01:22.123184Z info tf Checking pods ready... 2025-12-09T17:01:22.123229Z info tf Checking pods ready... 2025-12-09T17:01:22.129027Z info tf [ 0] prometheus-575cb4fbc6-x8mkj Running (Ready) 2025-12-09T17:01:22.180415Z info tf === DONE: Setup: 'telemetry_api' (27.033433369s) === 2025-12-09T17:01:22.180470Z info tf === BEGIN: Test Run: 'telemetry_api' === === RUN TestStatsFilter/additional-labels 2025-12-09T17:01:22.181815Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-12-09T17:02:42.203906Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.022084878s)' === 2025-12-09T17:02:42.204073Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.204090Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.204117Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.204087Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.391913Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:01:22Z/103c5d3bd35-177a-4599-bc2e-ceff6390269c 2025-12-09T17:02:42.392183Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1877s 2025-12-09T17:02:42.393072Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:42.497430Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:01:22Z/103c681b4fe-bb04-455c-aede-cb2cd66118c5 2025-12-09T17:02:42.497558Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2931s 2025-12-09T17:02:42.498218Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:42.598743Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:01:22Z/103f04994a6-b778-440d-a4bd-eb56e25f474a 2025-12-09T17:02:42.598843Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3944s 2025-12-09T17:02:42.599731Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter/additional-labels (80.89s) FAIL tests/integration/telemetry/api.TestStatsFilter/additional-labels (re-run 2) (80.89s) === RUN TestStatsFilter 2025-12-09T17:01:22.181142Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-12-09T17:02:43.072011Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m20.890858382s)' === 2025-12-09T17:02:43.072146Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-12-09T17:02:43.072157Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-12-09T17:02:43.072196Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-12-09T17:02:43.072220Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-12-09T17:02:43.295411Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/10497a30cef-9591-4a3c-91c1-511c8b4f350a 2025-12-09T17:02:43.295576Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2230s 2025-12-09T17:02:43.296269Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:43.400817Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/104d75d9da0-fdfa-4d8e-ac99-66b2fe929ced 2025-12-09T17:02:43.401198Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3286s 2025-12-09T17:02:43.401935Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:43.504533Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/104beeac049-3529-4dfc-84dd-076f55a99dc8 2025-12-09T17:02:43.504693Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4321s 2025-12-09T17:02:43.505691Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF --- FAIL: TestStatsFilter (81.81s) FAIL tests/integration/telemetry/api.TestStatsFilter (re-run 2) (81.81s) 2025-12-09T17:02:43.994843Z info tf === FAILED: Test Run: 'telemetry_api' (exitCode: 1) === 2025-12-09T17:02:43.994937Z info tf === Suite "telemetry_api" run time: 1m48.847979277s === 2025-12-09T17:02:43.994969Z info tf Wrote trace to /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/trace.yaml 2025-12-09T17:02:43.995401Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-12-09T17:02:43.995393Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-12-09T17:02:43.995392Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-12-09T17:02:43.995409Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-12-09T17:02:44.292283Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/10411df55b8-b13c-457d-b2f4-c1b7672e309e 2025-12-09T17:02:44.292459Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2962s 2025-12-09T17:02:44.293474Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:44.398434Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/1048caf4fea-9d0c-4bee-9072-d7d03342bc5f 2025-12-09T17:02:44.398586Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4023s 2025-12-09T17:02:44.399263Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:44.507685Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/1041a89b3ca-f653-4676-941e-23eb21680795 2025-12-09T17:02:44.507961Z info tf istioctl ([x internal-debug --all configz]): completed after 0.5117s 2025-12-09T17:02:44.508915Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:45.129389Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-12-09T17:02:45.129496Z info tf === SUCCEEDED: Cleanup Istio in 102.323µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api exec: [go test -json -test.run=^TestStatsTCPFilter$/^additional-labels$ -tags=integ -timeout 180m istio.io/istio/tests/integration/telemetry/api -args -istio.test.skipWorkloads=tproxy,vm -istio.test.openshift -istio.test.kube.helm.values=global.platform=openshift -istio.test.istio.enableCNI=true -istio.test.ci=true -istio.test.env=kube -istio.test.kube.deploy=false -istio.test.stableNamespaces=true -istio.test.kube.deployGatewayAPI=false -istio.test.gatewayConformance.maxTimeToConsistency=180s -istio.test.work_dir=/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts -test.skip TestCustomizeMetrics|TestDashboard/pilot-dashboard.json|TestStatsGatewayServerTCPFilter|TestImagePullPolicy/OCI_initial_creation_with_0.0.1|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_IfNotPresent|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2,_but_0.0.1_is_already_present_and_policy_is_default|TestImagePullPolicy/OCI_upstream_is_upgraded_to_0.0.2._0.0.1_is_already_present_but_policy_is_Always,_so_pull_0.0.2|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestGatewaySelection/OCI_initial_creation_with_latest_for_a_gateway|TestImagePullPolicyWithHTTP] go test pid: 413700 2025-12-09T17:02:56.281949Z warn unable to resolve TARGET_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T17:02:56.282050Z warn unable to resolve LOCAL_OUT. Dir /root/anurag/ossm3/istio/out/linux_ppc64le does not exist 2025-12-09T17:02:56.326452Z info tf === Test Framework Settings === 2025-12-09T17:02:56.326584Z info tf TestID: telemetry_api RunID: 25f4f81e-1206-4708-8f91-fee28b7c8c45 NoCleanup: false BaseDir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts Selector: FailOnDeprecation: false CIMode: true Retries: 0 StableNamespaces: true Revision: SkipWorkloads [tproxy vm] Compatibility: false Revisions: Hub: quay.io/maistra Tag: ibm-p Variant: PullPolicy: Always PullSecret: MaxDumps: 10 HelmRepo: https://istio-release.storage.googleapis.com/charts IPFamilies: [] GatewayConformanceStandardOnly: false GatewayConformanceAllowCRDsMismatch: false 2025-12-09T17:02:56.326591Z info tf =============================== 2025-12-09T17:02:56.326765Z info tf Test run dir: /home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-25f4f81e120647088 2025-12-09T17:02:56.326853Z info tf Test Framework Kubernetes environment Settings: Kubeconfigs: [] LoadBalancerSupported: true MCSControllerEnabled: false ControlPlaneTopology: map[] NetworkTopology: map[] ConfigTopology: map[] 2025-12-09T17:02:56.326865Z info tf Flags istio.test.kube.config and istio.test.kube.topology not specified. 2025-12-09T17:02:56.326882Z info tf Using KubeConfigs: [/root/upi/cA/auth/kubeconfig]. 2025-12-09T17:02:56.326889Z info tf === BEGIN: Building clusters === 2025-12-09T17:02:56.330843Z info tf Built Cluster: Name: cluster-0 StableName: primary-0 PrimaryCluster: cluster-0 ConfigCluster: cluster-0 Network: HTTPProxy: ProxyKubectlOnly: false Filename: /root/upi/cA/auth/kubeconfig 2025-12-09T17:02:56.330852Z info tf === DONE: Building clusters === 2025-12-09T17:02:56.330877Z info tf === BEGIN: Setup: 'telemetry_api' === 2025-12-09T17:02:56.330913Z info tf === BEGIN: Deploy Istio [Suite=telemetry_api] === 2025-12-09T17:02:56.330927Z info tf === Istio Component Config === 2025-12-09T17:02:56.330973Z info tf SystemNamespace: istio-system TelemetryNamespace: istio-system DeployIstio: false DeployEastWestGW: true Values: map[global.hub:quay.io/maistra global.imagePullPolicy:Always global.logging.level:xdsproxy:debug,wasm:debug global.platform:openshift global.tag:ibm-p global.variant:] PrimaryClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml ConfigClusterIOPFile: tests/integration/iop-integration-test-defaults.yaml RemoteClusterIOPFile: tests/integration/iop-remote-integration-test-defaults.yaml BaseIOPFile: tests/integration/base.yaml SkipWaitForValidationWebhook: false DumpKubernetesManifests: false IstiodlessRemotes: true OperatorOptions: map[] EnableCNI: true IngressGatewayServiceName: IngressGatewayServiceNamespace: IngressGatewayIstioLabel: EgressGatewayServiceName: istio-egressgateway EgressGatewayServiceNamespace: istio-system EgressGatewayIstioLabel: egressgateway SharedMeshConfigName: ControlPlaneInstaller: 2025-12-09T17:02:56.330981Z info tf ================================ 2025-12-09T17:02:56.334868Z info tf skipping deployment as specified in the config 2025-12-09T17:02:56.334894Z info tf === SUCCEEDED: Deploy Istio in 3.973442ms [Suite=telemetry_api]=== 2025-12-09T17:02:56.357292Z info tf === BEGIN: Deploy registry redirector server === 2025-12-09T17:02:56.357353Z info tf === BEGIN: Create namespace registry-redirector === 2025-12-09T17:02:56.369481Z info tf === SUCCEEDED: Create namespace registry-redirector in 12.109061ms === 2025-12-09T17:02:56.435277Z info tf Checking pods ready... 2025-12-09T17:02:56.435327Z info tf Checking pods ready... 2025-12-09T17:02:56.447863Z info tf No pods found... 2025-12-09T17:02:56.648810Z info tf Checking pods ready... 2025-12-09T17:02:56.648905Z info tf Checking pods ready... 2025-12-09T17:02:56.658262Z info tf [ 0] registry-redirector-7f6bd4c5df-w9bpw Pending (Pending) 2025-12-09T17:02:57.059181Z info tf Checking pods ready... 2025-12-09T17:02:57.059295Z info tf Checking pods ready... 2025-12-09T17:02:57.066383Z info tf [ 0] registry-redirector-7f6bd4c5df-w9bpw Pending (Pending) 2025-12-09T17:02:57.866542Z info tf Checking pods ready... 2025-12-09T17:02:57.866638Z info tf Checking pods ready... 2025-12-09T17:02:57.874249Z info tf [ 0] registry-redirector-7f6bd4c5df-w9bpw Running (container not ready: 'registry-redirector') 2025-12-09T17:02:59.475162Z info tf Checking pods ready... 2025-12-09T17:02:59.475257Z info tf Checking pods ready... 2025-12-09T17:02:59.484185Z info tf [ 0] registry-redirector-7f6bd4c5df-w9bpw Running (Ready) 2025-12-09T17:02:59.495061Z info klog Warning: v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 2025-12-09T17:02:59.495149Z info tf registry redirector server address: 172.30.253.159:1338 2025-12-09T17:02:59.546677Z info tf === SUCCEEDED: Deploy registry redirector server === 2025-12-09T17:02:59.546890Z info tf === BEGIN: Create namespace external === 2025-12-09T17:02:59.546890Z info tf === BEGIN: Create namespace echo === 2025-12-09T17:02:59.560433Z info tf === SUCCEEDED: Create namespace external in 13.540207ms === 2025-12-09T17:02:59.565975Z info tf === SUCCEEDED: Create namespace echo in 19.085272ms === 2025-12-09T17:02:59.673986Z info tf === BEGIN: Deploy echo instances === 2025-12-09T17:02:59.693666Z error tf === FAILED: Deploy echo instances === 2025-12-09T17:02:59.693740Z error tf failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-25f4f81e120647088/_suite_context/Service.3529295926.yaml] to ns external in cluster cluster-0: apply: patch: services "external" is forbidden: unable to create new content in namespace external because it is being terminated 2025-12-09T17:02:59.693780Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-12-09T17:02:59.693788Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-12-09T17:02:59.693814Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-12-09T17:02:59.693859Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-12-09T17:02:59.801910Z error tf Test setup error: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-25f4f81e120647088/_suite_context/Service.3529295926.yaml] to ns external in cluster cluster-0: apply: patch: services "external" is forbidden: unable to create new content in namespace external because it is being terminated 2025-12-09T17:02:59.801970Z info tf === FAILED: Setup: 'telemetry_api' (failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-25f4f81e120647088/_suite_context/Service.3529295926.yaml] to ns external in cluster cluster-0: apply: patch: services "external" is forbidden: unable to create new content in namespace external because it is being terminated) === 2025-12-09T17:02:59.801979Z error tf Exiting due to setup failure: failed applying YAML files [/home/jenkins/workspace/sail/istio-integration-tests-suites/telemetry-api/artifacts/telemetry-api-25f4f81e120647088/_suite_context/Service.3529295926.yaml] to ns external in cluster cluster-0: apply: patch: services "external" is forbidden: unable to create new content in namespace external because it is being terminated 2025-12-09T17:02:59.802018Z error tf === Dumping Namespace echo State for [suite(telemetry_api)]... 2025-12-09T17:02:59.802023Z error tf === Dumping Namespace registry-redirector State for [suite(telemetry_api)]... 2025-12-09T17:02:59.802047Z error tf === Dumping Istio Deployment State for [suite(telemetry_api)]... 2025-12-09T17:02:59.802070Z error tf === Dumping Namespace external State for [suite(telemetry_api)]... 2025-12-09T17:02:59.864318Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:58Z/11516c3ddfc-9083-42a2-9ec5-d3052ff5a166 2025-12-09T17:02:59.864471Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1703s 2025-12-09T17:02:59.865429Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:03:00.043466Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:58Z/11549d38992-47fe-4e78-9584-2a838dc15b43 2025-12-09T17:03:00.043631Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3494s 2025-12-09T17:03:00.044632Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:03:00.057592Z info tf === BEGIN: Cleanup Istio [Suite=telemetry_api] === 2025-12-09T17:03:00.057641Z info tf === SUCCEEDED: Cleanup Istio in 40.512µs [Suite=telemetry_api] === FAIL tests/integration/telemetry/api === Failed === FAIL: tests/integration/telemetry/api TestStatsFilter/additional-labels (81.58s) 2025-12-09T16:43:15.597918Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 9 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 11 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [source_version upstream_custom_label] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 12 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 5 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 6 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: Series 7 (source: istio-system/istio-ingressgateway-5754586674-wzcdj) stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 8 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 9 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: missing labels: [upstream_custom_label source_version] stats_test.go:142: Series 10 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "418" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 11 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "source_workload", wanted "a-v1" but got "istio-ingressgateway" stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: for label "source_app", wanted "a" but got "istio-ingressgateway" stats_test.go:142: for label "source_workload_namespace", wanted "echo" but got "istio-system" stats_test.go:142: for label "source_version", wanted "v1" but got "unknown" stats_test.go:142: for label "response_code", wanted "200" but got "520" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: query "istio_requests_total" returned 5 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-12-09T16:44:35.614667Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.016745189s)' === 2025-12-09T16:44:35.614749Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.614749Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.614803Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.614795Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-12-09T16:44:35.797890Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:43:15Z/29393b9b15-0b84-4f62-9568-a9c1daec3c12 2025-12-09T16:44:35.798141Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1831s 2025-12-09T16:44:35.798959Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:35.889166Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:43:15Z/2911f95d32-a6bf-4ae6-92b6-8971ac337cc0 2025-12-09T16:44:35.889276Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2742s 2025-12-09T16:44:35.889924Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:35.987595Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:43:15Z/2973f57d51-e95f-440b-8b48-e5eaf872ea3e 2025-12-09T16:44:35.987714Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3726s 2025-12-09T16:44:35.988483Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsFilter (85.49s) 2025-12-09T16:43:13.495018Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-12-09T16:44:37.247888Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m23.752867376s)' === 2025-12-09T16:44:37.247949Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-12-09T16:44:37.247961Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-12-09T16:44:37.247975Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-12-09T16:44:37.248010Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-12-09T16:44:37.429705Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:37Z/303c2b93e1-cff5-4203-82c8-5c02a9d1ec72 2025-12-09T16:44:37.430025Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1817s 2025-12-09T16:44:37.430711Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:37.540796Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:37Z/301302f5f6-deea-42d4-83ff-8fedad1bac28 2025-12-09T16:44:37.540874Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2925s 2025-12-09T16:44:37.541866Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:44:37.641815Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:37Z/3097c0b9ef-73f0-4082-b2b5-3776247b1c2d 2025-12-09T16:44:37.641988Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3936s 2025-12-09T16:44:37.642804Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter/additional-labels (81.77s) 2025-12-09T16:44:43.054436Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 5 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 4 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "source_app", wanted "a" but got "unknown" stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: for label "destination_service_name", wanted "b" but got "InboundPassthroughCluster" stats_test.go:214: for label "source_version", wanted "v1" but got "unknown" stats_test.go:214: for label "source_cluster", wanted "Kubernetes" but got "unknown" stats_test.go:214: for label "source_workload", wanted "a-v1" but got "unknown" stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 3 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/b-v1-67cc9c4fff-m9sj7) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/a-v1-5b559bd498-bk2s8) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-12-09T16:46:03.065261Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.010816268s)' === 2025-12-09T16:46:03.065421Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.065429Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.065474Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.065459Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-12-09T16:46:03.251583Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:43Z/31c3483293-2677-4b88-b14b-3de54faa4ff9 2025-12-09T16:46:03.251724Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.1854s 2025-12-09T16:46:03.252717Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:03.351312Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:43Z/3115cf439b-9f5a-47a8-9cbf-aa84d9ae9864 2025-12-09T16:46:03.351575Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2852s 2025-12-09T16:46:03.352257Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:03.439039Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:44:43Z/31b04e27e6-bee6-4ab2-9ea1-bbfb93295b4e 2025-12-09T16:46:03.439123Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3727s 2025-12-09T16:46:03.440160Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter (87.75s) 2025-12-09T16:44:38.987769Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-12-09T16:46:04.827347Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m25.839570821s)' === 2025-12-09T16:46:04.827479Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-12-09T16:46:04.827503Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-12-09T16:46:04.827522Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-12-09T16:46:04.827490Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-12-09T16:46:04.994448Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:04Z/32261ef53d-0e00-4e2b-86d1-2f87f9004527 2025-12-09T16:46:04.994742Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1670s 2025-12-09T16:46:04.995719Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:05.098758Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:04Z/32468f3851-f4ee-473d-880f-4ab8b7048891 2025-12-09T16:46:05.098950Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2712s 2025-12-09T16:46:05.099794Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:46:05.212126Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:46:04Z/325e3ab13c-2982-4032-b703-d7c611dcd971 2025-12-09T16:46:05.212309Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3846s 2025-12-09T16:46:05.213455Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsFilter/additional-labels (re-run 1) (81.18s) 2025-12-09T16:57:08.397366Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5558d6d789-p4l6w) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/b-v1-597d68fd78-4rtbt) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-12-09T16:58:28.422774Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.0254034s)' === 2025-12-09T16:58:28.422886Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.422889Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.422951Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.422957Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-12-09T16:58:28.681325Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:57:08Z/58669fc8df-1cb7-499a-a857-d92fd2317877 2025-12-09T16:58:28.681579Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2580s 2025-12-09T16:58:28.682356Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:28.808583Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:57:08Z/58e42d44de-288e-40ee-91a4-4143795505fa 2025-12-09T16:58:28.808708Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3851s 2025-12-09T16:58:28.809568Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:28.912494Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:57:08Z/584b94c17d-ea47-4951-adf3-8a4e4d0ed33e 2025-12-09T16:58:28.912654Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.4890s 2025-12-09T16:58:28.913399Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsFilter (re-run 1) (82.39s) 2025-12-09T16:57:08.397130Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-12-09T16:58:29.573276Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m21.176139062s)' === 2025-12-09T16:58:29.573471Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-12-09T16:58:29.573471Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-12-09T16:58:29.573478Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-12-09T16:58:29.573478Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-12-09T16:58:29.764191Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/596c5936da-c8d2-421a-95b7-cf9d13fbe6a1 2025-12-09T16:58:29.764399Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1906s 2025-12-09T16:58:29.764944Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:29.861439Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/599f1a828b-b36c-4381-8651-b6dd9c0f05b4 2025-12-09T16:58:29.861557Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2877s 2025-12-09T16:58:29.862515Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T16:58:29.963068Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:58:29Z/59298444e7-0c76-432b-a7e1-2dd53c8d700e 2025-12-09T16:58:29.963236Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3893s 2025-12-09T16:58:29.964082Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter/additional-labels (re-run 1) (80.68s) 2025-12-09T16:59:16.139390Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter/additional-labels]' === stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: no metrics found for istio_tcp_connections_opened_total{} stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 4 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 1 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 2 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: Series 3 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:214: query "istio_tcp_connections_opened_total" returned 2 series, but none matched our query exactly. stats_test.go:214: Original query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",} stats_test.go:214: Series 0 (source: echo/a-v1-7bf5bb9475-2gch6) stats_test.go:214: for label "downstream_custom_label", wanted "a" but got "unknown" stats_test.go:214: Series 1 (source: echo/b-v1-5f5b658f7b-48v4n) stats_test.go:214: for label "reporter", wanted "destination" but got "source" stats_test.go:214: missing labels: [downstream_custom_label] stats_test.go:227: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_tcp_connections_opened_total{destination_app="b",destination_canonical_revision="v1",destination_canonical_service="b",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",downstream_custom_label="a",reporter="destination",request_protocol="tcp",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",})) 2025-12-09T17:00:36.152634Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter/additional-labels] (1m20.013238516s)' === 2025-12-09T17:00:36.152772Z error tf === Dumping Namespace echo State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.152796Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.152808Z error tf === Dumping Namespace external State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.152833Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter/additional-labels... 2025-12-09T17:00:36.341127Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:59:16Z/806e85e992-1294-42e1-8570-f2f048d69666 2025-12-09T17:00:36.341335Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1793s 2025-12-09T17:00:36.342262Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:36.435711Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:59:16Z/80e33c1dd5-3010-4a4b-aa73-45053fb05761 2025-12-09T17:00:36.435820Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2738s 2025-12-09T17:00:36.436731Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:36.530699Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T16:59:16Z/80b66f56f3-e16a-49f4-8c30-a9e4193cde89 2025-12-09T17:00:36.530780Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3687s 2025-12-09T17:00:36.531521Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsTCPFilter (re-run 1) (81.33s) 2025-12-09T16:59:16.139147Z info tf === BEGIN: Test: 'telemetry_api[TestStatsTCPFilter]' === 2025-12-09T17:00:36.820960Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsTCPFilter] (1m20.681805759s)' === 2025-12-09T17:00:36.821066Z error tf === Dumping Namespace echo State for TestStatsTCPFilter... 2025-12-09T17:00:36.821087Z error tf === Dumping Istio Deployment State for TestStatsTCPFilter... 2025-12-09T17:00:36.821072Z error tf === Dumping Namespace external State for TestStatsTCPFilter... 2025-12-09T17:00:36.821107Z error tf === Dumping Namespace registry-redirector State for TestStatsTCPFilter... 2025-12-09T17:00:37.030720Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/813fe188b8-5d2b-48d6-97a2-bd919d004772 2025-12-09T17:00:37.031051Z info tf istioctl ([x internal-debug --all configz]): completed after 0.2097s 2025-12-09T17:00:37.031916Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:37.153226Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/815516ce58-2fa2-4280-a3b5-e37e61d3ffde 2025-12-09T17:00:37.153346Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.3319s 2025-12-09T17:00:37.154206Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:00:37.257601Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:00:36Z/8104bceb60-b4d7-4b9c-ad38-7a9e2ff56ca9 2025-12-09T17:00:37.257765Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4363s 2025-12-09T17:00:37.258374Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsFilter/additional-labels (re-run 2) (80.89s) 2025-12-09T17:01:22.181815Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter/additional-labels]' === stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: no metrics found for istio_requests_total{} stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 6 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 1 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 3 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 4 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 5 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: query "istio_requests_total" returned 3 series, but none matched our query exactly. stats_test.go:142: Original query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",} stats_test.go:142: Series 0 (source: echo/b-v1-6855685c8f-fq25z) stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: Series 1 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "reporter", wanted "source" but got "destination" stats_test.go:142: missing labels: [upstream_custom_label] stats_test.go:142: Series 2 (source: echo/a-v1-5d7d795b7b-b28hf) stats_test.go:142: for label "destination_version", wanted "v1" but got "unknown" stats_test.go:142: for label "destination_service", wanted "b.echo.svc.cluster.local" but got "naked.echo.svc.cluster.local" stats_test.go:142: for label "destination_service_name", wanted "b" but got "naked" stats_test.go:142: for label "upstream_custom_label", wanted "b" but got "unknown" stats_test.go:142: for label "destination_app", wanted "b" but got "unknown" stats_test.go:175: test failed: timeout while waiting after 40 attempts (last error: value not found (query: istio_requests_total{destination_app="b",destination_service="b.echo.svc.cluster.local",destination_service_name="b",destination_service_namespace="echo",destination_version="v1",destination_workload_namespace="echo",reporter="source",request_protocol="http",response_code="200",source_app="a",source_cluster="Kubernetes",source_version="v1",source_workload="a-v1",source_workload_namespace="echo",upstream_custom_label="b",})) 2025-12-09T17:02:42.203906Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter/additional-labels] (1m20.022084878s)' === 2025-12-09T17:02:42.204073Z error tf === Dumping Namespace echo State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.204090Z error tf === Dumping Namespace external State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.204117Z error tf === Dumping Istio Deployment State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.204087Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter/additional-labels... 2025-12-09T17:02:42.391913Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:01:22Z/103c5d3bd35-177a-4599-bc2e-ceff6390269c 2025-12-09T17:02:42.392183Z info tf istioctl ([x internal-debug --all configz]): completed after 0.1877s 2025-12-09T17:02:42.393072Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:42.497430Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:01:22Z/103c681b4fe-bb04-455c-aede-cb2cd66118c5 2025-12-09T17:02:42.497558Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2931s 2025-12-09T17:02:42.498218Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:42.598743Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:01:22Z/103f04994a6-b778-440d-a4bd-eb56e25f474a 2025-12-09T17:02:42.598843Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.3944s 2025-12-09T17:02:42.599731Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF === FAIL: tests/integration/telemetry/api TestStatsFilter (re-run 2) (81.81s) 2025-12-09T17:01:22.181142Z info tf === BEGIN: Test: 'telemetry_api[TestStatsFilter]' === 2025-12-09T17:02:43.072011Z info tf === DONE (failed): Test: 'telemetry_api[TestStatsFilter] (1m20.890858382s)' === 2025-12-09T17:02:43.072146Z error tf === Dumping Namespace echo State for TestStatsFilter... 2025-12-09T17:02:43.072157Z error tf === Dumping Namespace registry-redirector State for TestStatsFilter... 2025-12-09T17:02:43.072196Z error tf === Dumping Namespace external State for TestStatsFilter... 2025-12-09T17:02:43.072220Z error tf === Dumping Istio Deployment State for TestStatsFilter... 2025-12-09T17:02:43.295411Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/10497a30cef-9591-4a3c-91c1-511c8b4f350a 2025-12-09T17:02:43.295576Z info tf istioctl ([x internal-debug --all mcsz]): completed after 0.2230s 2025-12-09T17:02:43.296269Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:43.400817Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/104d75d9da0-fdfa-4d8e-ac99-66b2fe929ced 2025-12-09T17:02:43.401198Z info tf istioctl ([x internal-debug --all configz]): completed after 0.3286s 2025-12-09T17:02:43.401935Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF 2025-12-09T17:02:43.504533Z info adsc Received type=istio.io/debug count=1 nonce=2025-12-09T17:02:43Z/104beeac049-3529-4dfc-84dd-076f55a99dc8 2025-12-09T17:02:43.504693Z info tf istioctl ([x internal-debug --all clusterz]): completed after 0.4321s 2025-12-09T17:02:43.505691Z info adsc connection closed with err: rpc error: code = Unavailable desc = error reading from server: EOF DONE 3 runs, 43 tests, 10 failures in 1450.196s exec: go version