Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-57603

Azure: failed to create cluster with disk encryption set created in different subscription

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.17, 4.18, 4.19, 4.20
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Create disk encryption set in different subscription than the cluster, and specify this disk encryption set in install-config, 
      ========
      platform:
        azure:
          baseDomainResourceGroupName: os4-common
          cloudName: AzurePublicCloud
          outboundType: Loadbalancer
          region: eastus
          defaultMachinePlatform:
            identity:
              type: None
            encryptionAtHost: true
            osDisk:
              diskEncryptionSet:
                resourceGroup: jima-test-rg
                name: jima-des
                subscriptionId: 8cbff7ff-5103-4cc2-b691-abbee101e1d0
      
      It's no problem to create manifests, and I checked that infrastructure machine manifests, and diskEncryptionSet on osDisk is set correctly.
      $ cat 10_machine_jima-des-1-8nvjl-bootstrap.yaml 
      apiVersion: cluster.x-k8s.io/v1beta1
      kind: Machine
      ......
      spec:
        additionalCapabilities:
          ultraSSDEnabled: false
      ......
        osDisk:
          cachingType: ReadWrite
          diskSizeGB: 1024
          managedDisk:
            diskEncryptionSet:
              id: /subscriptions/8cbff7ff-5103-4cc2-b691-abbee101e1d0/resourceGroups/jima-test-rg/providers/Microsoft.Compute/diskEncryptionSets/jima-des
            storageAccountType: Premium_LRS
      
      But continued to create cluster, and failed. 
      
      time="2025-06-18T09:02:36Z" level=debug msg="I0618 09:02:36.745663  333295 recorder.go:104] \"failed to reconcile AzureMachine: failed to reconcile AzureMachine service virtualmachine: failed to create or update resource jima-des-1-5x8g7-rg/jima-des-1-5x8g7-bootstrap (service: virtualmachine): PUT https://management.azure.com/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima-des-1-5x8g7-rg/providers/Microsoft.Compute/virtualMachines/jima-des-1-5x8g7-bootstrap\\n--------------------------------------------------------------------------------\\nRESPONSE 400: 400 Bad Request\\nERROR CODE: BadRequest\\n--------------------------------------------------------------------------------\\n{\\n  \\\"error\\\": {\\n    \\\"code\\\": \\\"BadRequest\\\",\\n    \\\"message\\\": \\\"DiskEncryptionSet '/subscriptions/8cbff7ff-5103-4cc2-b691-abbee101e1d0/resourceGroups/jima-test-rg/providers/Microsoft.Compute/diskEncryptionSets/jima-des' was not found.\\\",\\n    \\\"target\\\": \\\"/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima-des-1-5x8g7-rg/providers/Microsoft.Compute/disks/jima-des-1-5x8g7-bootstrap_OSDisk\\\"\\n  }\\n}\\n--------------------------------------------------------------------------------\\n\" logger=\"events\" type=\"Warning\" object={\"kind\":\"AzureMachine\",\"namespace\":\"openshift-cluster-api-guests\",\"name\":\"jima-des-1-5x8g7-bootstrap\",\"uid\":\"56768e5a-cf83-45de-8f06-4366c1ca3a2e\",\"apiVersion\":\"infrastructure.cluster.x-k8s.io/v1beta1\",\"resourceVersion\":\"1717\"} reason=\"ReconcileError\""
      time="2025-06-18T09:02:36Z" level=debug msg="E0618 09:02:36.746092  333295 controller.go:316] \"Reconciler error\" err=<"
      time="2025-06-18T09:02:36Z" level=debug msg="\tfailed to reconcile AzureMachine: failed to reconcile AzureMachine service virtualmachine: failed to create or update resource jima-des-1-5x8g7-rg/jima-des-1-5x8g7-bootstrap (service: virtualmachine): PUT https://management.azure.com/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima-des-1-5x8g7-rg/providers/Microsoft.Compute/virtualMachines/jima-des-1-5x8g7-bootstrap"
      time="2025-06-18T09:02:36Z" level=debug msg="\t--------------------------------------------------------------------------------"
      time="2025-06-18T09:02:36Z" level=debug msg="\tRESPONSE 400: 400 Bad Request"
      time="2025-06-18T09:02:36Z" level=debug msg="\tERROR CODE: BadRequest"
      time="2025-06-18T09:02:36Z" level=debug msg="\t--------------------------------------------------------------------------------"
      time="2025-06-18T09:02:36Z" level=debug msg="\t{"
      time="2025-06-18T09:02:36Z" level=debug msg="\t  \"error\": {"
      time="2025-06-18T09:02:36Z" level=debug msg="\t    \"code\": \"BadRequest\","
      time="2025-06-18T09:02:36Z" level=debug msg="\t    \"message\": \"DiskEncryptionSet '/subscriptions/8cbff7ff-5103-4cc2-b691-abbee101e1d0/resourceGroups/jima-test-rg/providers/Microsoft.Compute/diskEncryptionSets/jima-des' was not found.\","
      time="2025-06-18T09:02:36Z" level=debug msg="\t    \"target\": \"/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/jima-des-1-5x8g7-rg/providers/Microsoft.Compute/disks/jima-des-1-5x8g7-bootstrap_OSDisk\""
      time="2025-06-18T09:02:36Z" level=debug msg="\t  }"
      time="2025-06-18T09:02:36Z" level=debug msg="\t}"
      time="2025-06-18T09:02:36Z" level=debug msg="\t--------------------------------------------------------------------------------"
      time="2025-06-18T09:02:36Z" level=debug msg=" > controller=\"azuremachine\" controllerGroup=\"infrastructure.cluster.x-k8s.io\" controllerKind=\"AzureMachine\" AzureMachine=\"openshift-cluster-api-guests/jima-des-1-5x8g7-bootstrap\" namespace=\"openshift-cluster-api-guests\" name=\"jima-des-1-5x8g7-bootstrap\" reconcileID=\"7b7729dd-5c28-4b5c-9808-8b1878f5a56a\""

      Version-Release number of selected component (if applicable):

          4.20 nightly build

      How reproducible:

          Always

      Steps to Reproduce:

          1. Create disk encryption set in different subscription than cluster
          2. Specify the disk encryption set in install-config
          3. Install cluster
          

      Actual results:

          Fail to create cluster

      Expected results:

          Cluster creation is successful.

      Additional info:

          

              Unassigned Unassigned
              jinyunma Jinyun Ma
              None
              None
              Jinyun Ma Jinyun Ma
              None
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: