-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.17.z, 4.16.z, 4.18.z
-
Moderate
-
None
-
Installer Sprint 267
-
1
-
False
-
Description of problem:
In some cases, installer may need to call ReplaceRoute action, but with minimum permission, this is not allowed: ... time="2025-02-22T06:44:35Z" level=debug msg="E0222 06:44:35.976720 218 awscluster_controller.go:319] \"failed to reconcile network\" err=<" time="2025-02-22T06:44:35Z" level=debug msg="\tfailed to replace outdated route on route table \"rtb-0f3322786d2a7a9fc\": UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:iam::301721915996:user/ci-op-n3z38rfl-21543-minimal-perm-installer is not authorized to perform: ec2:ReplaceRoute on resource: arn:aws:ec2:us-east-1:301721915996:route-table/rtb-0f3322786d2a7a9fc because no identity-based policy allows the ec2:ReplaceRoute action. Encoded authorization failure message: HIDDEN" time="2025-02-22T06:44:35Z" level=debug msg="\t\tstatus code: 403, request id: 405cded7-daae-49b6-aa38-ed2d2fbceb75" time="2025-02-22T06:44:35Z" level=debug msg=" > controller=\"awscluster\" controllerGroup=\"infrastructure.cluster.x-k8s.io\" controllerKind=\"AWSCluster\" AWSCluster=\"openshift-cluster-api-guests/ci-op-n3z38rfl-21543-6h797\" namespace=\"openshift-cluster-api-guests\" name=\"ci-op-n3z38rfl-21543-6h797\" reconcileID=\"24aae75e-bd3e-4705-a88a-e69bfa0b4974\" cluster=\"openshift-cluster-api-guests/ci-op-n3z38rfl-21543-6h797\"" time="2025-02-22T06:44:35Z" level=debug msg="I0222 06:44:35.976749 218 recorder.go:104] \"Operation ReplaceRoute failed with a credentials or permission issue\" logger=\"events\" type=\"Warning\" object={\"kind\":\"AWSCluster\",\"namespace\":\"openshift-cluster-api-guests\",\"name\":\"ci-op-n3z38rfl-21543-6h797\",\"uid\":\"dfdcd50e-f0d5-4456-b5d6-c26de8f2c2ce\",\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta2\",\"resourceVersion\":\"448\"} reason=\"UnauthorizedOperation\"" time="2025-02-22T06:44:35Z" level=debug msg="I0222 06:44:35.976773 218 recorder.go:104] \"Failed to replace outdated route on managed RouteTable \\\"rtb-0f3322786d2a7a9fc\\\": UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:iam::301721915996:user/ci-op-n3z38rfl-21543-minimal-perm-installer is not authorized to perform: ec2:ReplaceRoute on resource: arn:aws:ec2:us-east-1:301721915996:route-table/rtb-0f3322786d2a7a9fc because no identity-based policy allows the ec2:ReplaceRoute action. Encoded authorization failure message: [HIDDEN]\\n\\tstatus code: 403, request id: 405cded7-daae-49b6-aa38-ed2d2fbceb75\" logger=\"events\" type=\"Warning\" object={\"kind\":\"AWSCluster\",\"namespace\":\"openshift-cluster-api-guests\",\"name\":\"ci-op-n3z38rfl-21543-6h797\",\"uid\":\"dfdcd50e-f0d5-4456-b5d6-c26de8f2c2ce\",\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta2\",\"resourceVersion\":\"448\"} reason=\"FailedReplaceRoute\"" time="2025-02-22T06:44:36Z" level=debug msg="E0222 06:44:36.035951 218 controller.go:324] \"Reconciler error\" err=<" ...
Version-Release number of selected component (if applicable):
4.18.1
How reproducible:
Occasionally
Steps to Reproduce:
it's not always reproducible, in this case, the install-config is like: ... fips: true controlPlane: platform: aws: zones: - us-east-1c - us-east-1b type: m6i.xlarge architecture: amd64 name: master replicas: 3 compute: - platform: aws: zones: - us-east-1c - us-east-1b type: m5.xlarge architecture: amd64 name: worker replicas: 3 - name: edge architecture: amd64 hyperthreading: Enabled replicas: 1 platform: aws: zones: [us-east-1-atl-1a] baseDomain: qe.devcluster.openshift.com platform: aws: region: us-east-1 ...
Actual results:
Install failed.
Expected results:
Install succeeds.
Additional info:
It looks like the issue comes from the upstream CAPA [1], so all CAPI installs (4.16+) might be affected. [1] https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/4e912b4e4d1f855abf9b5194acaf9f31b5763c57/pkg/cloud/services/network/routetables.go#L160