Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-73821

[AWS][CAPI] some dedicated host behavior is not consistent between MAPI and CAPI which cause conversion failed

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

          some AWS dedicated host behavior is not consistent between MAPI and CAPI which cause conversion failed

      Version-Release number of selected component (if applicable):

          4.22

      How reproducible:

          always

      Steps to Reproduce:

          1.Install an AWS techpreview cluster
      
          2.Create awsmachinetemplate and CAPI machine set, in awsmachinetemplate, only set hostAffinity: default, the CAPI machine get Running
      
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms2.yaml 
      awsmachinetemplate.infrastructure.cluster.x-k8s.io/huliu-aws0115c-78tq4-worker-us-east-2aa created
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms3.yaml 
      machineset.cluster.x-k8s.io/huliu-aws0115c-78tq4-worker-us-east-2aa created
      liuhuali@Lius-MacBook-Pro huali-test % oc get machine.c
      NAME                                            CLUSTER                NODE NAME                                   READY   AVAILABLE   UP-TO-DATE   PHASE     AGE     VERSION
      huliu-aws0115c-78tq4-worker-us-east-2a-sxhr6    huliu-aws0115c-78tq4   ip-10-0-19-113.us-east-2.compute.internal   True    True                     Running   64m     
      huliu-aws0115c-78tq4-worker-us-east-2aa-cqfz7   huliu-aws0115c-78tq4   ip-10-0-6-24.us-east-2.compute.internal     True    True                     Running   9m14s   
      huliu-aws0115c-78tq4-worker-us-east-2b-5bhns    huliu-aws0115c-78tq4   ip-10-0-48-226.us-east-2.compute.internal   True    True                     Running   64m     
      huliu-aws0115c-78tq4-worker-us-east-2c-9qvqf    huliu-aws0115c-78tq4   ip-10-0-81-177.us-east-2.compute.internal   True    True                     Running   64m     
      
      
      liuhuali@Lius-MacBook-Pro huali-test % cat ms2.yaml 
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
      kind: AWSMachineTemplate
      metadata:
        name: huliu-aws0115c-78tq4-worker-us-east-2aa
        namespace: openshift-cluster-api
      spec:
        template:
          metadata: {}
          spec:
            additionalSecurityGroups:
            - filters:
              - name: tag:Name
                values:
                - huliu-aws0115c-78tq4-node
            - filters:
              - name: tag:Name
                values:
                - huliu-aws0115c-78tq4-lb
            additionalTags:
              kubernetes.io/cluster/huliu-aws0115c-78tq4: owned
            ami:
              id: ami-0bc8dda494f111572
            cloudInit: {}
            hostAffinity: default
            iamInstanceProfile: huliu-aws0115c-78tq4-worker-profile
            ignition:
              storageType: UnencryptedUserData
            instanceMetadataOptions:
              httpEndpoint: enabled
              httpPutResponseHopLimit: 1
              httpTokens: optional
              instanceMetadataTags: disabled
            instanceType: m6i.xlarge
            rootVolume:
              encrypted: true
              size: 120
              type: gp3
            subnet:
              filters:
              - name: tag:Name
                values:
                - huliu-aws0115c-78tq4-subnet-private-us-east-2a
      liuhuali@Lius-MacBook-Pro huali-test % cat ms3.yaml 
      apiVersion: cluster.x-k8s.io/v1beta2
      kind: MachineSet
      metadata:
        name: huliu-aws0115c-78tq4-worker-us-east-2aa
        namespace: openshift-cluster-api
      spec:
        clusterName: huliu-aws0115c-78tq4
        deletion:
          order: Random
        replicas: 1
        selector:
          matchLabels:
            machine.openshift.io/cluster-api-cluster: huliu-aws0115c-78tq4
            machine.openshift.io/cluster-api-machineset: huliu-aws0115c-78tq4-worker-us-east-2aa
        template:
          metadata:
            labels:
              cluster.x-k8s.io/cluster-name: huliu-aws0115c-78tq4
              machine.openshift.io/cluster-api-cluster: huliu-aws0115c-78tq4
              machine.openshift.io/cluster-api-machineset: huliu-aws0115c-78tq4-worker-us-east-2aa
              node-role.kubernetes.io/worker: ""
          spec:
            bootstrap:
              dataSecretName: worker-user-data
            clusterName: huliu-aws0115c-78tq4
            deletion:
              nodeDeletionTimeoutSeconds: 10
            failureDomain: us-east-2a
            infrastructureRef:
              apiGroup: infrastructure.cluster.x-k8s.io
              kind: AWSMachineTemplate
              name: huliu-aws0115c-78tq4-worker-us-east-2aa
      liuhuali@Lius-MacBook-Pro huali-test % 
      
      While the same configuration doesn't work for MAPI
       only set 
                  host:
                    affinity: AnyAvailable    
      in MAPI machineset, webhook prompt
      
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms1.yaml 
      Error from server (Forbidden): error when creating "ms1.yaml": admission webhook "validation.machineset.machine.openshift.io" denied the request: spec.placement.host: Forbidden: host may only be specified when tenancy is 'host'
      
      liuhuali@Lius-MacBook-Pro huali-test % cat ms1.yaml 
      apiVersion: machine.openshift.io/v1beta1
      kind: MachineSet
      metadata:
        name: huliu-aws0115c-78tq4-worker-us-east-2a1
        namespace: openshift-machine-api
      spec:
        authoritativeAPI: MachineAPI
        replicas: 1
        selector:
          matchLabels:
            machine.openshift.io/cluster-api-cluster: huliu-aws0115c-78tq4
            machine.openshift.io/cluster-api-machineset: huliu-aws0115c-78tq4-worker-us-east-2a1
        template:
          metadata:
            labels:
              machine.openshift.io/cluster-api-cluster: huliu-aws0115c-78tq4
              machine.openshift.io/cluster-api-machine-role: worker
              machine.openshift.io/cluster-api-machine-type: worker
              machine.openshift.io/cluster-api-machineset: huliu-aws0115c-78tq4-worker-us-east-2a1
          spec:
            authoritativeAPI: MachineAPI
            lifecycleHooks: {}
            metadata: {}
            providerSpec:
              value:
                ami:
                  id: ami-0bc8dda494f111572
                apiVersion: machine.openshift.io/v1beta1
                blockDevices:
                - ebs:
                    encrypted: true
                    iops: 0
                    kmsKey:
                      arn: ""
                    volumeSize: 120
                    volumeType: gp3
                capacityReservationId: ""
                credentialsSecret:
                  name: aws-cloud-credentials
                deviceIndex: 0
                iamInstanceProfile:
                  id: huliu-aws0115c-78tq4-worker-profile
                instanceType: m6i.xlarge
                kind: AWSMachineProviderConfig
                metadata: {}
                metadataServiceOptions: {}
                placement:
                  availabilityZone: us-east-2a
                  region: us-east-2
                  host:
                    affinity: AnyAvailable
                securityGroups:
                - filters:
                  - name: tag:Name
                    values:
                    - huliu-aws0115c-78tq4-node
                - filters:
                  - name: tag:Name
                    values:
                    - huliu-aws0115c-78tq4-lb
                subnet:
                  filters:
                  - name: tag:Name
                    values:
                    - huliu-aws0115c-78tq4-subnet-private-us-east-2a
                tags:
                - name: kubernetes.io/cluster/huliu-aws0115c-78tq4
                  value: owned
                userDataSecret:
                  name: worker-user-data
      liuhuali@Lius-MacBook-Pro huali-test % 
      
      
       3.Create the same name MAPI machineset with authoritativeAPI: ClusterAPI. The inconsistent cause CAPI2MAPI conversion failed
      
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms4.yaml 
      machineset.machine.openshift.io/huliu-aws0115c-78tq4-worker-us-east-2aa created
      liuhuali@Lius-MacBook-Pro huali-test % oc get machineset -n openshift-machine-api
      NAME                                      DESIRED   CURRENT   READY   AVAILABLE   AGE
      huliu-aws0115c-78tq4-worker-us-east-2a    1         1         1       1           113m
      huliu-aws0115c-78tq4-worker-us-east-2aa   1                                       64s
      huliu-aws0115c-78tq4-worker-us-east-2b    1         1         1       1           113m
      huliu-aws0115c-78tq4-worker-us-east-2c    1         1         1       1           113m
      liuhuali@Lius-MacBook-Pro huali-test % oc get machineset huliu-aws0115c-78tq4-worker-us-east-2aa -n openshift-machine-api -oyaml
      ...
      status:
        authoritativeAPI: ClusterAPI
        conditions:
        - lastTransitionTime: "2026-01-15T04:23:31Z"
          message: The AuthoritativeAPI status is set to 'ClusterAPI'
          reason: AuthoritativeAPINotMachineAPI
          status: "True"
          type: Paused
        - lastTransitionTime: "2026-01-15T04:23:32Z"
          message: 'failed to update MAPI machine set: admission webhook "validation.machineset.machine.openshift.io"
            denied the request: spec.placement.host: Forbidden: host may only be specified
            when tenancy is ''host'''
          reason: FailedToUpdateMAPIMachineSet
          severity: Error
          status: "False"
          type: Synchronized
        observedGeneration: 1
        synchronizedGeneration: 0
      
      liuhuali@Lius-MacBook-Pro huali-test % cat ms4.yaml 
      apiVersion: machine.openshift.io/v1beta1
      kind: MachineSet
      metadata:
        name: huliu-aws0115c-78tq4-worker-us-east-2aa
        namespace: openshift-machine-api
      spec:
        authoritativeAPI: ClusterAPI
        replicas: 1
        selector:
          matchLabels:
            machine.openshift.io/cluster-api-cluster: huliu-aws0115c-78tq4
            machine.openshift.io/cluster-api-machineset: huliu-aws0115c-78tq4-worker-us-east-2aa
        template:
          metadata:
            labels:
              machine.openshift.io/cluster-api-cluster: huliu-aws0115c-78tq4
              machine.openshift.io/cluster-api-machine-role: worker
              machine.openshift.io/cluster-api-machine-type: worker
              machine.openshift.io/cluster-api-machineset: huliu-aws0115c-78tq4-worker-us-east-2aa
          spec:
            authoritativeAPI: ClusterAPI
            lifecycleHooks: {}
            metadata: {}
            providerSpec:
              value:
                ami:
                  id: ami-0bc8dda494f111572
                apiVersion: machine.openshift.io/v1beta1
                blockDevices:
                - ebs:
                    encrypted: true
                    iops: 0
                    kmsKey:
                      arn: ""
                    volumeSize: 120
                    volumeType: gp3
                capacityReservationId: ""
                credentialsSecret:
                  name: aws-cloud-credentials
                deviceIndex: 0
                iamInstanceProfile:
                  id: huliu-aws0115c-78tq4-worker-profile
                instanceType: m6i.xlarge
                kind: AWSMachineProviderConfig
                metadata: {}
                metadataServiceOptions: {}
                placement:
                  availabilityZone: us-east-2a
                  region: us-east-2
                securityGroups:
                - filters:
                  - name: tag:Name
                    values:
                    - huliu-aws0115c-78tq4-node
                - filters:
                  - name: tag:Name
                    values:
                    - huliu-aws0115c-78tq4-lb
                subnet:
                  filters:
                  - name: tag:Name
                    values:
                    - huliu-aws0115c-78tq4-subnet-private-us-east-2a
                tags:
                - name: kubernetes.io/cluster/huliu-aws0115c-78tq4
                  value: owned
                userDataSecret:
                  name: worker-user-data
      
      4.Another case is when CAPI set
      
            hostAffinity: host
            tenancy: host
      
      works, but MAPI set
      
                  tenancy: host
                  host:
                    affinity: DedicatedHost
      
      doesn't work
      
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms1.yaml 
      Error from server (Forbidden): error when creating "ms1.yaml": admission webhook "validation.machineset.machine.openshift.io" denied the request: spec.placement.host.dedicatedHost: Required value: dedicatedHost is required when hostAffinity is DedicatedHost, and optional otherwise
      
      which cause CAPI2MAPI conversion failed
      
      status:
        authoritativeAPI: ClusterAPI
        conditions:
        - lastTransitionTime: "2026-01-15T04:38:11Z"
          message: The AuthoritativeAPI status is set to 'ClusterAPI'
          reason: AuthoritativeAPINotMachineAPI
          status: "True"
          type: Paused
        - lastTransitionTime: "2026-01-15T04:38:11Z"
          message: 'failed to convert CAPI machine set to MAPI machine set: spec.dedicatedHost.id:
            Required value: id is required and must start with ''h-'' followed by 8 or 17
            lowercase hexadecimal characters (0-9 and a-f)'
          reason: FailedToConvertCAPIMachineSetToMAPI
          severity: Error
          status: "False"
          type: Synchronized
        observedGeneration: 1
        synchronizedGeneration: 0

      Actual results:

          some AWS dedicated host behavior is not consistent between MAPI and CAPI which cause conversion failed

      Expected results:

      MAPI and CAPI should be feature parity and behave the same to ensure conversion succeed

      Additional info:

      must-gather:  https://drive.google.com/file/d/1Mid6tvlKx-tHTWxak9OzkLus3O8NgUIH/view?usp=sharing

              rhn-support-ngirard Neil Girard
              huliu@redhat.com Huali Liu
              None
              None
              Huali Liu Huali Liu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: