Loading...

XML

Word

Printable

Type: Bug
Resolution: Cannot Reproduce
Priority: Undefined
Fix Version/s: None
Affects Version/s: 4.18
Component/s: Networking / router
Labels:
- gateway-api
- ne-triaged

Activity Type:
Quality / Stability / Reliability
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Story Points:
1
Severity:
Important
Regression:
No

Target Backport Versions:
None
Target Version:
None
Release Blocker:
Rejected
Sprint:
NI&D Sprint 269
sprint_count:
1

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Release Note Status:
None
Release Note Type:
None
Release Note Text:
None

Escape Reason:
None
Escape Impact:
None
Corrective Measures:
None
SDLC stage when should've been found:
None

Description of problem:

    gateway pod CrashLoopBackOff on AWS STS cluster

Version-Release number of selected component (if applicable):

    4.18.0-0.nightly-2024-12-12-133926

How reproducible:

    100%

Steps to Reproduce:

    1. setup AWS STS cluster
    2. enable GatewayAPI featuregate
    3. create gatewaycalss named "openshift-default"
    4. create gateway named "gateway"

Actual results:

    the gateway pod CrashLoopBackOff, here is log

$ oc -n openshift-ingress get pod
NAME                                        READY   STATUS             RESTARTS     AGE
gateway-openshift-default-f696467fb-fz7sq   0/1     CrashLoopBackOff   6 (3s ago)   5m10s
istiod-openshift-gateway-dcb6884c4-rgjmt    1/1     Running            0            18m
router-default-bbd94ff68-9q8vm              1/1     Running            0            13m
router-default-bbd94ff68-r5hjt              1/1     Running            0            17m

$ oc -n openshift-ingress logs gateway-openshift-default-f696467fb-fz7sq
2024-12-13T02:19:34.893007Z    info    ads    All caches have been synced up in 15.524203ms, marking server ready
2024-12-13T02:19:34.893187Z    info    xdsproxy    Initializing with upstream address "istiod-openshift-gateway.openshift-ingress.svc:15012" and cluster "Kubernetes"
2024-12-13T02:19:34.905662Z    info    sds    Starting SDS grpc server
2024-12-13T02:19:34.942442Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:34.973069Z    warn    xdsproxy    upstream [1] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:34.973469Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:35.079379Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:35.083111Z    warn    xdsproxy    upstream [2] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:35.083346Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:35.121907Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:35.233431Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:35.238755Z    warn    xdsproxy    upstream [3] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:35.239101Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:35.496103Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:35.507929Z    warn    xdsproxy    upstream [4] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:35.508311Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:35.677280Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:35.685154Z    warn    xdsproxy    upstream [5] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:35.685419Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:35.974009Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:36.526307Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:37.235893Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:39.648521Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:40.352818Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:40.356463Z    warn    xdsproxy    upstream [6] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:40.356716Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:42.482365Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:19:42.485573Z    warn    xdsproxy    upstream [7] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:19:42.485847Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:19:43.518967Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:45.565388Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:49.591251Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:19:58.782499Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:20:00.795407Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:20:00.800548Z    warn    xdsproxy    upstream [8] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:20:00.800816Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:20:02.182271Z    info    xdsproxy    connected to upstream XDS server: istiod-openshift-gateway.openshift-ingress.svc:15012
2024-12-13T02:20:02.185753Z    warn    xdsproxy    upstream [9] terminated with unexpected error rpc error: code = Unauthenticated desc = authentication failure
2024-12-13T02:20:02.186004Z    warning    envoy config external/envoy/source/extensions/config_subscription/grpc/grpc_stream.h:152    StreamAggregatedResources gRPC config stream to xds-grpc closed: 16, authentication failure    thread=9
2024-12-13T02:20:04.587589Z    error    accept tcp [::]:15020: use of closed network connection
2024-12-13T02:20:05.430634Z    warn    sds    failed to warm certificate: failed to generate workload certificate: create certificate: rpc error: code = Unauthenticated desc = request authenticate failure
2024-12-13T02:20:09.591051Z    warn    Aborting proxy
2024-12-13T02:20:09.591187Z    warn    Aborted proxy instance
2024-12-13T02:20:09.591373Z    info    sds    SDS server for workload certificates started, listening on "./var/run/secrets/workload-spiffe-uds/socket"

Expected results:

    gateway pod should be ready on AWS STS cluster

Additional info:

    same issue in ROSA STS cluster

Assignee:: Miciah Masters

Reporter:: Hongan Li

Need Info From:: None

Contributors:: None

QA Contact:: Melvin Joseph

Doc Contact:: None

Votes:: 1 Vote for this issue

Watchers:: 5 Start watching this issue

Created:: 2024/12/13 2:25 AM

Updated:: 2025/07/17 1:33 PM

Resolved:: 2025/05/15 1:55 AM

Details

Description

Attachments

Easy Agile Planning Poker

Activity

People

Dates