Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-36670

[CAPI Azure] Gen2 image definition missed security features enabled when configuring securitytype in install-config

XMLWordPrintable

    • Critical
    • No
    • Installer (PB) Sprint 259, Installer Sprint 260, Installer Sprint 261
    • 3
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • In Progress

      Description of problem:

      Using payload built with https://github.com/openshift/installer/pull/8666/ so that master instances can be provisioned from gen2 image, which is required when configuring security type in install-config.
      
      Enable TrustedLaunch security type in install-config:
      ==================
      controlPlane:
        architecture: amd64
        hyperthreading: Enabled
        name: master
        platform:
          azure: 
            encryptionAtHost: true
            settings:
              securityType: TrustedLaunch
              trustedLaunch:
                uefiSettings:
                  secureBoot: Enabled
                  virtualizedTrustedPlatformModule: Enabled
      
      Launch capi-based installation, installer failed after waiting 15min for machines to provision...
      INFO GalleryImage.ID=/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima08conf01-9vgq5-rg/providers/Microsoft.Compute/galleries/gallery_jima08conf01_9vgq5/images/jima08conf01-9vgq5 
      INFO GalleryImage.ID=/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima08conf01-9vgq5-rg/providers/Microsoft.Compute/galleries/gallery_jima08conf01_9vgq5/images/jima08conf01-9vgq5-gen2 
      INFO Created manifest *v1beta1.AzureMachine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-bootstrap 
      INFO Created manifest *v1beta1.AzureMachine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master-0 
      INFO Created manifest *v1beta1.AzureMachine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master-1 
      INFO Created manifest *v1beta1.AzureMachine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master-2 
      INFO Created manifest *v1beta1.Machine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-bootstrap 
      INFO Created manifest *v1beta1.Machine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master-0 
      INFO Created manifest *v1beta1.Machine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master-1 
      INFO Created manifest *v1beta1.Machine, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master-2 
      INFO Created manifest *v1.Secret, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-bootstrap 
      INFO Created manifest *v1.Secret, namespace=openshift-cluster-api-guests name=jima08conf01-9vgq5-master 
      INFO Waiting up to 15m0s (until 6:26AM UTC) for machines [jima08conf01-9vgq5-bootstrap jima08conf01-9vgq5-master-0 jima08conf01-9vgq5-master-1 jima08conf01-9vgq5-master-2] to provision... 
      ERROR failed to fetch Cluster: failed to generate asset "Cluster": failed to create cluster: control-plane machines were not provisioned within 15m0s: client rate limiter Wait returned an error: context deadline exceeded 
      INFO Shutting down local Cluster API control plane... 
      INFO Stopped controller: Cluster API              
      INFO Stopped controller: azure infrastructure provider 
      INFO Stopped controller: azureaso infrastructure provider 
      INFO Local Cluster API system has completed operations 
      
      In openshift-install.log,
      time="2024-07-08T06:25:49Z" level=debug msg="\tfailed to reconcile AzureMachine: failed to reconcile AzureMachine service virtualmachine: failed to create or update resource jima08conf01-9vgq5-rg/jima08conf01-9vgq5-bootstrap (service: virtualmachine): PUT https://management.azure.com/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima08conf01-9vgq5-rg/providers/Microsoft.Compute/virtualMachines/jima08conf01-9vgq5-bootstrap"
      time="2024-07-08T06:25:49Z" level=debug msg="\t--------------------------------------------------------------------------------"
      time="2024-07-08T06:25:49Z" level=debug msg="\tRESPONSE 400: 400 Bad Request"
      time="2024-07-08T06:25:49Z" level=debug msg="\tERROR CODE: BadRequest"
      time="2024-07-08T06:25:49Z" level=debug msg="\t--------------------------------------------------------------------------------"
      time="2024-07-08T06:25:49Z" level=debug msg="\t{"
      time="2024-07-08T06:25:49Z" level=debug msg="\t  \"error\": {"
      time="2024-07-08T06:25:49Z" level=debug msg="\t    \"code\": \"BadRequest\","
      time="2024-07-08T06:25:49Z" level=debug msg="\t    \"message\": \"Use of TrustedLaunch setting is not supported for the provided image. Please select Trusted Launch Supported Gen2 OS Image. For more information, see https://aka.ms/TrustedLaunch-FAQ.\""
      time="2024-07-08T06:25:49Z" level=debug msg="\t  }"
      time="2024-07-08T06:25:49Z" level=debug msg="\t}"
      time="2024-07-08T06:25:49Z" level=debug msg="\t--------------------------------------------------------------------------------"
      time="2024-07-08T06:25:49Z" level=debug msg=" > controller=\"azuremachine\" controllerGroup=\"infrastructure.cluster.x-k8s.io\" controllerKind=\"AzureMachine\" AzureMachine=\"openshift-cluster-api-guests/jima08conf01-9vgq5-bootstrap\" namespace=\"openshift-cluster-api-guests\" name=\"jima08conf01-9vgq5-bootstrap\" reconcileID=\"bee8a459-c3c8-4295-ba4a-f3d560d6a68b\""
      
      Looks like that capi-based installer missed to enable security features during creating gen2 image, which can be found in terraform code.
      https://github.com/openshift/installer/blob/master/data/data/azure/vnet/main.tf#L166-L169
      
      Gen2 image definition created by terraform:
      $ az sig image-definition show --gallery-image-definition jima08conf02-4mrnz-gen2 -r gallery_jima08conf02_4mrnz -g jima08conf02-4mrnz-rg --query 'features'
      [
        {
          "name": "SecurityType",
          "value": "TrustedLaunch"
        }
      ]
      It's empty when querying from gen2 image created by using CAPI.
      $ az sig image-definition show --gallery-image-definition jima08conf01-9vgq5-gen2 -r gallery_jima08conf01_9vgq5 -g jima08conf01-9vgq5-rg --query 'features'
      $ 

      Version-Release number of selected component (if applicable):

      4.17 payload built from cluster-bot with PR https://github.com/openshift/installer/pull/8666/

      How reproducible:

      Always

      Steps to Reproduce:

          1. Enable security type in install-config
          2. Create cluster by using CAPI
          3. 
          

      Actual results:

          Install failed.

      Expected results:

          Install succeeded.

      Additional info:

         It impacts installation with security type ConfidentialVM or TrustedLaunch enabled.  

       

            sdasu@redhat.com Sandhya Dasu
            jinyunma Jinyun Ma
            Jinyun Ma Jinyun Ma
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated: