Major While executing Istio Integration tests (sidecar), it is observed that the istio-proxy container is throwing some error related to certificate signing in tproxy and not getting into ready state. It looks like it is not able to reach istiod and connection is timing out. Log from istio-proxy is below. Seeing this in both power and s390x.
Here is the log.
2025-11-10T13:18:20.016850Z info FLAG: --concurrency="0" 2025-11-10T13:18:20.016872Z info FLAG: --domain="echo.svc.cluster.local" 2025-11-10T13:18:20.016879Z info FLAG: --help="false" 2025-11-10T13:18:20.016884Z info FLAG: --log_as_json="false" 2025-11-10T13:18:20.016888Z info FLAG: --log_caller="" 2025-11-10T13:18:20.016893Z info FLAG: --log_output_level="default:info" 2025-11-10T13:18:20.016897Z info FLAG: --log_stacktrace_level="default:none" 2025-11-10T13:18:20.016906Z info FLAG: --log_target="[stdout]" 2025-11-10T13:18:20.016910Z info FLAG: --meshConfig="./etc/istio/config/mesh" 2025-11-10T13:18:20.016914Z info FLAG: --outlierLogPath="" 2025-11-10T13:18:20.016919Z info FLAG: --profiling="true" 2025-11-10T13:18:20.016923Z info FLAG: --proxyComponentLogLevel="misc:error" 2025-11-10T13:18:20.016927Z info FLAG: --proxyLogLevel="warning" 2025-11-10T13:18:20.016931Z info FLAG: --serviceCluster="istio-proxy" 2025-11-10T13:18:20.016936Z info FLAG: --stsPort="0" 2025-11-10T13:18:20.016941Z info FLAG: --templateFile="" 2025-11-10T13:18:20.016946Z info FLAG: --tokenManagerPlugin="" 2025-11-10T13:18:20.016951Z info FLAG: --vklog="0" 2025-11-10T13:18:20.016957Z info Version 1.27.3-f6a985379fd8fe1e5362851fe5c24e3730f643f9-Clean 2025-11-10T13:18:20.017199Z info Proxy role ips=[172.21.3.218] type=sidecar id=tproxy-v1-847b88f5c7-7kg2s.echo domain=echo.svc.cluster.local 2025-11-10T13:18:20.017295Z info Apply proxy config from env {"proxyMetadata":{"ISTIO_META_DNS_CAPTURE":"true","WASM_INSECURE_REGISTRIES":"172.30.79.23:1338"}} 2025-11-10T13:18:20.019041Z info Apply proxy config from annotation proxyMetadata: WASM_INSECURE_REGISTRIES: "172.30.79.23:1338" 2025-11-10T13:18:20.019152Z info cpu limit detected as 2, setting concurrency 2025-11-10T13:18:20.019451Z info Effective config: binaryPath: /usr/local/bin/envoy concurrency: 2 configPath: ./etc/istio/proxy controlPlaneAuthPolicy: MUTUAL_TLS discoveryAddress: istiod.istio-system.svc:15012 drainDuration: 45s proxyAdminPort: 15000 proxyMetadata: ISTIO_META_DNS_CAPTURE: "true" WASM_INSECURE_REGISTRIES: 172.30.79.23:1338 serviceCluster: istio-proxy statNameLength: 189 statusPort: 15020 terminationDrainDuration: 5s 2025-11-10T13:18:20.019462Z info JWT policy is third-party-jwt 2025-11-10T13:18:20.019468Z info using credential fetcher of JWT type in cluster.local trust domain 2025-11-10T13:18:20.025907Z info Prometheus scraping configuration: {true 15014} 2025-11-10T13:18:20.026182Z info Opening status port 15020 2025-11-10T13:18:20.026266Z info dns Starting local udp DNS server on 127.0.0.1:15053 2025-11-10T13:18:20.026514Z info dns Starting local tcp DNS server on 127.0.0.1:15053 2025-11-10T13:18:20.026554Z info Starting default Istio SDS Server 2025-11-10T13:18:20.026580Z info CA Endpoint istiod.istio-system.svc:15012, provider Citadel 2025-11-10T13:18:20.026626Z info Using CA istiod.istio-system.svc:15012 cert with certs: var/run/secrets/istio/root-cert.pem 2025-11-10T13:18:20.027899Z info xdsproxy Initializing with upstream address "istiod.istio-system.svc:15012" and cluster "Kubernetes" 2025-11-10T13:18:20.029595Z info sds Starting SDS grpc server 2025-11-10T13:18:20.029612Z info sds Starting SDS server for workload certificates, will listen on "var/run/secrets/workload-spiffe-uds/socket" 2025-11-10T13:18:20.030254Z info Pilot SAN: [istiod.istio-system.svc] 2025-11-10T13:18:20.031535Z info Starting proxy agent 2025-11-10T13:18:20.031575Z info Envoy command: [-c etc/istio/proxy/envoy-rev.json --drain-time-s 45 --drain-strategy immediate --local-address-ip-version v4 --file-flush-interval-msec 1000 --disable-hot-restart --allow-unknown-static-fields -l warning --component-log-level misc:error --skip-deprecated-logs --concurrency 2] 2025-11-10T13:18:20.128074Z warning envoy main external/envoy/source/server/server.cc:901 Usage of the deprecated runtime key overload.global_downstream_max_connections, consider switching to `envoy.resource_monitors.global_downstream_max_connections` instead.This runtime key will be removed in future. thread=9 2025-11-10T13:18:20.128503Z warning envoy main external/envoy/source/server/server.cc:997 There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor. thread=9 2025-11-10T13:18:40.028847Z warn ca ca request failed, starting attempt 1 in 95.289011ms 2025-11-10T13:18:40.125090Z warn ca ca request failed, starting attempt 2 in 218.61909ms 2025-11-10T13:18:40.344577Z warn ca ca request failed, starting attempt 3 in 438.777222ms 2025-11-10T13:18:40.784293Z warn ca ca request failed, starting attempt 4 in 745.791142ms 2025-11-10T13:18:41.530448Z error citadelclient failed to sign CSR: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 172.30.196.135:15012: i/o timeout" 2025-11-10T13:18:41.530930Z info citadelclient recreated connection 2025-11-10T13:18:41.531008Z error cache resource:default failed to sign: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 172.30.196.135:15012: i/o timeout" 2025-11-10T13:18:41.531047Z warn sds failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 172.30.196.135:15012: i/o timeout"