Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-23039

AWS/CI: Local Zones provisioning is failing in us-west-2-las-1b


      Description of problem:

      Installations on AWS with Local Zone subnets or zones are failing on CI when the zone selected is us-west-2-las-1b (new zone in Las Vegas/US).
      The CI selects randomly one zone[1], when that specific zone is selected and added to the install-config.yaml[2] the installer tries to discover the instance type offered in the zone[3], but the EC2 API is returning empty values[4] - which means no EC2 available in that zone, consequently the installer will fill the InstanceType in the MachineSet with empty value, leading to fail when MAPI tries to provision the instance[5].
      Additionally, the AWS Local Zones feature page[6] does not show the zone us-west-2-las-1b listed.
      [1] https://github.com/openshift/release/blob/f36aa8f608a786674c27db163c30f7d88a0f64e9/ci-operator/step-registry/ipi/aws/pre/local-zones/opt-in/ipi-aws-pre-local-zones-opt-in-commands.sh#L15-L16
      [2] https://github.com/openshift/release/blob/f36aa8f608a786674c27db163c30f7d88a0f64e9/ci-operator/step-registry/ipi/conf/aws/ipi-conf-aws-commands.sh#L324-L336
      [3] https://github.com/openshift/installer/blob/2596e5d46c7b8f2c4907c7c6d80beaf768ff9953/pkg/asset/machines/worker.go#L206-L220
      [4] No Instances in us-west-2-las-1b
      $ aws ec2 describe-instance-type-offerings --region us-west-2 --location-type availability-zone --filters Name=location,Values=us-west-2-las-1b
          "InstanceTypeOfferings": []
      [5] Machine status
          - lastTransitionTime: "2023-11-07T19:32:52Z"
            message: 'error launching instance: Invalid value '''' for InstanceType.'
            reason: MachineCreationFailed
            status: "False"
            type: MachineCreation
      [6] https://aws.amazon.com/about-aws/global-infrastructure/localzones/features/?nc=sn&loc=2

      Version-Release number of selected component (if applicable):


      How reproducible:


      Steps to Reproduce:

      1. create the install-config, setting edge compute pool with zone us-west-2-las-1b
      2. generate the manifests or install the cluster

      Actual results:

      Empty Instance Type, MAPi fails to provision the cluster, CI job does not progress as it requires to have the number of nodes equals the machines

      Expected results:

      - installer never progress when there are no EC2 available in the zone
      - CI step check if there is available EC2 offerings in the zone, otherwise, select another zone (preventing to lose the job run)

      Additional info:

      [1] Example of job runs:
      - https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_release/45158/rehearse-45158-pull-ci-openshift-installer-release-4.14-e2e-aws-ovn-shared-vpc-localzones/1721937572780838912
      - https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_installer/7676/pull-ci-openshift-installer-master-e2e-aws-ovn-localzones/1721327505312321536
      - https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-nightly-4.15-e2e-aws-ovn-shared-vpc-localzones/1720954245286465536

            rhn-support-mrbraga Marco Braga
            rhn-support-mrbraga Marco Braga
            Yunfei Jiang Yunfei Jiang
            0 Vote for this issue
            4 Start watching this issue