Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-52455

Sometime azure machine recreate with error "Disk already exists"

    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      In a CI job, during cluster installation we met an error when machine ProvisioningState is failed, we deleted it and recreated it, but disk exists, then failed to create.    

      Version-Release number of selected component (if applicable):

       4.19   

      How reproducible:

      Met one time    

      Steps to Reproduce:

          1. Install an azure cluster
          2.
          3.
          

      Actual results:

      One master was created failed, cluster install Failed
      
          - lastTransitionTime: "2025-03-04T16:53:08Z"
            message: Instance has not been created
            reason: InstanceNotCreated
            severity: Warning
            status: "False"
            type: InstanceExists
          - lastTransitionTime: "2025-03-04T16:53:08Z"
            status: "True"
            type: Terminable
        errorMessage: "failed to reconcile machine \"ci-op-8ql2zrii-0cfeb-nxztf-master-2\": failed to create vm ci-op-8ql2zrii-0cfeb-nxztf-master-2: failure sending request for machine ci-op-8ql2zrii-0cfeb-nxztf-master-2: cannot create vm: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status= Code=\"ConflictingUserInput\" Message=\"Disk ci-op-8ql2zrii-0cfeb-nxztf-master-2_OSDisk already exists in resource group CI-OP-8QL2ZRII-0CFEB-NXZTF-RG. Only CreateOption.Attach is supported.\" Target=\"/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ci-op-8ql2zrii-0cfeb-nxztf-rg/providers/Microsoft.Compute/disks/ci-op-8ql2zrii-0cfeb-nxztf-master-2_OSDisk\""
        errorReason: InvalidConfiguration
        lastUpdated: "2025-03-04T16:53:52Z"
        phase: Failed
        providerStatus:
          conditions:
            - lastTransitionTime: "2025-03-04T16:53:16Z"
              message: "failed to create vm ci-op-8ql2zrii-0cfeb-nxztf-master-2: failure sending request for machine ci-op-8ql2zrii-0cfeb-nxztf-master-2: cannot create vm: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status= Code=\"ConflictingUserInput\" Message=\"Disk ci-op-8ql2zrii-0cfeb-nxztf-master-2_OSDisk already exists in resource group CI-OP-8QL2ZRII-0CFEB-NXZTF-RG. Only CreateOption.Attach is supported.\" Target=\"/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ci-op-8ql2zrii-0cfeb-nxztf-rg/providers/Microsoft.Compute/disks/ci-op-8ql2zrii-0cfeb-nxztf-master-2_OSDisk\""
              reason: MachineCreationFailed
              status: "False"
              type: MachineCreated
          metadata: {}
          vmId: /subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ci-op-8ql2zrii-0cfeb-nxztf-rg/providers/Microsoft.Compute/virtualMachines/ci-op-8ql2zrii-0cfeb-nxztf-master-2
          vmState: Unknown
      
      I0304 16:53:08.119786       1 reconciler.go:456] vm for machine ci-op-8ql2zrii-0cfeb-nxztf-master-2 has unexpected 'Failed' provisioning state
      I0304 16:53:08.119834       1 controller.go:392] ci-op-8ql2zrii-0cfeb-nxztf-master-2: setting phase to Provisioning and requeuing
      I0304 16:53:08.119840       1 controller.go:454] ci-op-8ql2zrii-0cfeb-nxztf-master-2: going into phase "Provisioning"
      I0304 16:53:08.130403       1 controller.go:173] ci-op-8ql2zrii-0cfeb-nxztf-master-2: reconciling Machine
      I0304 16:53:08.130498       1 actuator.go:221] ci-op-8ql2zrii-0cfeb-nxztf-master-2: actuator checking if machine exists
      I0304 16:53:08.501818       1 reconciler.go:456] vm for machine ci-op-8ql2zrii-0cfeb-nxztf-master-2 has unexpected 'Failed' provisioning state
      ...
      E0304 16:53:26.733644       1 actuator.go:84] Machine error: failed to reconcile machine "ci-op-8ql2zrii-0cfeb-nxztf-master-2"s: failed to create vm ci-op-8ql2zrii-0cfeb-nxztf-master-2: failed to delete machine: failed to delete machine: compute.VirtualMachinesDeleteFuture: asynchronous operation has not completed
      W0304 16:53:26.733746       1 controller.go:401] ci-op-8ql2zrii-0cfeb-nxztf-master-2: failed to create machine: requeue in: 20s
      I0304 16:53:26.733823       1 controller.go:431] Actuator returned requeue-after error: requeue in: 20s
      I0304 16:53:26.733761       1 recorder.go:104] "msg"="CreateError: failed to reconcile machine \"ci-op-8ql2zrii-0cfeb-nxztf-master-2\"s: failed to create vm ci-op-8ql2zrii-0cfeb-nxztf-master-2: failed to delete machine: failed to delete machine: compute.VirtualMachinesDeleteFuture: asynchronous operation has not completed" "logger"="events" "object"={"kind":"Machine","namespace":"openshift-machine-api","name":"ci-op-8ql2zrii-0cfeb-nxztf-master-2","uid":"61bdd40a-f67b-4337-8574-57cd570cfa6d","apiVersion":"machine.openshift.io/v1beta1","resourceVersion":"10396"} "reason"="FailedCreate" "type"="Warning"
      ...
      E0304 16:53:42.796972       1 actuator.go:84] Machine error: failed to reconcile machine "ci-op-8ql2zrii-0cfeb-nxztf-master-2": failed to create vm ci-op-8ql2zrii-0cfeb-nxztf-master-2: failure sending request for machine ci-op-8ql2zrii-0cfeb-nxztf-master-2: cannot create vm: compute.VirtualMachinesClient#CreateOrUpdate: Failure sending request: StatusCode=0 -- Original Error: autorest/azure: Service returned an error. Status=<nil> Code="ConflictingUserInput" Message="Disk ci-op-8ql2zrii-0cfeb-nxztf-master-2_OSDisk already exists in resource group CI-OP-8QL2ZRII-0CFEB-NXZTF-RG. Only CreateOption.Attach is supported." Target="/subscriptions/53b8f551-f0fc-4bea-8cba-6d1fefd54c8a/resourceGroups/ci-op-8ql2zrii-0cfeb-nxztf-rg/providers/Microsoft.Compute/disks/ci-op-8ql2zrii-0cfeb-nxztf-master-2_OSDisk"

      Expected results:

      Create cluster succeed.    

      Additional info:

      job link: https://qe-private-deck-ci.apps.ci.l2s4.p1.openshiftapps.com/view/gs/qe-private-deck/logs/periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-azure-ipi-confidentialvm-vmgueststateonly-f28-destructive/1896956373040631808 

      must-gather: https://drive.google.com/file/d/1ekQa-q2zTZg-lLK66FZUK28lIaFFqdUR/view?usp=sharing 

              rh-ee-tbarberb Theo Barber-Bany
              rhn-support-zhsun Zhaohua Sun
              Zhaohua Sun Zhaohua Sun
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: