Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-51171

[AWS] Performing ReplaceRoute action is not allowed with minimum permission policy

XMLWordPrintable

    • Moderate
    • None
    • Installer Sprint 267
    • 1
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      
      In some cases, installer may need to call ReplaceRoute action, but with minimum permission, this is not allowed:
      
      ...
      time="2025-02-22T06:44:35Z" level=debug msg="E0222 06:44:35.976720 	218 awscluster_controller.go:319] \"failed to reconcile network\" err=<"
      time="2025-02-22T06:44:35Z" level=debug msg="\tfailed to replace outdated route on route table \"rtb-0f3322786d2a7a9fc\": UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:iam::301721915996:user/ci-op-n3z38rfl-21543-minimal-perm-installer is not authorized to perform: ec2:ReplaceRoute on resource: arn:aws:ec2:us-east-1:301721915996:route-table/rtb-0f3322786d2a7a9fc because no identity-based policy allows the ec2:ReplaceRoute action. Encoded authorization failure message: HIDDEN"
      time="2025-02-22T06:44:35Z" level=debug msg="\t\tstatus code: 403, request id: 405cded7-daae-49b6-aa38-ed2d2fbceb75"
      time="2025-02-22T06:44:35Z" level=debug msg=" > controller=\"awscluster\" controllerGroup=\"infrastructure.cluster.x-k8s.io\" controllerKind=\"AWSCluster\" AWSCluster=\"openshift-cluster-api-guests/ci-op-n3z38rfl-21543-6h797\" namespace=\"openshift-cluster-api-guests\" name=\"ci-op-n3z38rfl-21543-6h797\" reconcileID=\"24aae75e-bd3e-4705-a88a-e69bfa0b4974\" cluster=\"openshift-cluster-api-guests/ci-op-n3z38rfl-21543-6h797\""
      time="2025-02-22T06:44:35Z" level=debug msg="I0222 06:44:35.976749 	218 recorder.go:104] \"Operation ReplaceRoute failed with a credentials or permission issue\" logger=\"events\" type=\"Warning\" object={\"kind\":\"AWSCluster\",\"namespace\":\"openshift-cluster-api-guests\",\"name\":\"ci-op-n3z38rfl-21543-6h797\",\"uid\":\"dfdcd50e-f0d5-4456-b5d6-c26de8f2c2ce\",\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta2\",\"resourceVersion\":\"448\"} reason=\"UnauthorizedOperation\""
      time="2025-02-22T06:44:35Z" level=debug msg="I0222 06:44:35.976773 	218 recorder.go:104] \"Failed to replace outdated route on managed RouteTable \\\"rtb-0f3322786d2a7a9fc\\\": UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:iam::301721915996:user/ci-op-n3z38rfl-21543-minimal-perm-installer is not authorized to perform: ec2:ReplaceRoute on resource: arn:aws:ec2:us-east-1:301721915996:route-table/rtb-0f3322786d2a7a9fc because no identity-based policy allows the ec2:ReplaceRoute action. Encoded authorization failure message: [HIDDEN]\\n\\tstatus code: 403, request id: 405cded7-daae-49b6-aa38-ed2d2fbceb75\" logger=\"events\" type=\"Warning\" object={\"kind\":\"AWSCluster\",\"namespace\":\"openshift-cluster-api-guests\",\"name\":\"ci-op-n3z38rfl-21543-6h797\",\"uid\":\"dfdcd50e-f0d5-4456-b5d6-c26de8f2c2ce\",\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta2\",\"resourceVersion\":\"448\"} reason=\"FailedReplaceRoute\""
      time="2025-02-22T06:44:36Z" level=debug msg="E0222 06:44:36.035951 	218 controller.go:324] \"Reconciler error\" err=<"
      ...
      
      
          

      Version-Release number of selected component (if applicable):

      4.18.1
          

      How reproducible:

      Occasionally
          

      Steps to Reproduce:

      it's not always reproducible, in this case, the install-config is like:
      
      ...
      fips: true
      controlPlane:
        platform:
      	aws:
        	zones:
        	- us-east-1c
        	- us-east-1b
        	type: m6i.xlarge
        architecture: amd64
        name: master
        replicas: 3
      compute:
      - platform:
      	aws:
        	zones:
        	- us-east-1c
        	- us-east-1b
        	type: m5.xlarge
        architecture: amd64
        name: worker
        replicas: 3
      - name: edge
        architecture: amd64
        hyperthreading: Enabled
        replicas: 1
        platform:
      	aws:
        	zones: [us-east-1-atl-1a]
      baseDomain: qe.devcluster.openshift.com
      platform:
        aws:
      	region: us-east-1
      ...
      
          

      Actual results:

      Install failed.
          

      Expected results:

         Install succeeds.
          

      Additional info:

        It looks like the issue comes from the upstream CAPA [1], so all CAPI installs (4.16+) might be affected.
      
      [1] https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/4e912b4e4d1f855abf9b5194acaf9f31b5763c57/pkg/cloud/services/network/routetables.go#L160
      
          

              rh-ee-thvo Thuan Vo
              yunjiang-1 Yunfei Jiang
              Yunfei Jiang Yunfei Jiang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: