Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-63318

[AWS][CAPI] the newly created CAPI master machine isn't associated to the loadbalancers

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      The newly created CAPI master machine isn't associated to the loadbalancers defined in the AWSCluster.   
      
      I also tried newly created a MAPI master machine and it's associated to the loadbalancers.
      https://drive.google.com/file/d/1BIS_XoJSrueB6fZlip3ssXfce8DhVPtt/view?usp=sharing
      https://drive.google.com/file/d/17G_alfuBWLh3rYNapNoqYP-X5enSFHtg/view?usp=sharing
      https://drive.google.com/file/d/1_XH9ADuO1RRROU-UWyiO2_EfQmmdlKn0/view?usp=sharing

      Version-Release number of selected component (if applicable):

          4.21.0-0.nightly-2025-10-19-181151

      How reproducible:

          always

      Steps to Reproduce:

          1.Install an AWS techpreview cluster, we can see there are loadbalancers in the AWSCluster
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get clusterversion
      NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
      version   4.21.0-0.nightly-2025-10-19-181151   True        False         40m     Cluster version is 4.21.0-0.nightly-2025-10-19-181151
      liuhuali@Lius-MacBook-Pro huali-test % oc get awscluster -n openshift-cluster-api
      NAME                   CLUSTER                READY   VPC   BASTION IP
      huliu-aws1020b-rsrcn   huliu-aws1020b-rsrcn   true          
      liuhuali@Lius-MacBook-Pro huali-test % oc get awscluster huliu-aws1020b-rsrcn -n openshift-cluster-api -oyaml
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
      kind: AWSCluster
      metadata:
        annotations:
          cluster.x-k8s.io/managed-by: cluster-capi-operator-infracluster-controller
        creationTimestamp: "2025-10-20T03:41:22Z"
        generation: 1
        labels:
          cluster.x-k8s.io/cluster-name: huliu-aws1020b-rsrcn
        name: huliu-aws1020b-rsrcn
        namespace: openshift-cluster-api
        ownerReferences:
        - apiVersion: cluster.x-k8s.io/v1beta1
          blockOwnerDeletion: true
          controller: true
          kind: Cluster
          name: huliu-aws1020b-rsrcn
          uid: fbb38b3c-3cad-402b-9163-149f6092af17
        resourceVersion: "20146"
        uid: 4030074c-39d4-4e20-a2f9-615e60449ba4
      spec:
        bastion:
          enabled: false
        controlPlaneEndpoint:
          host: api-int.huliu-aws1020b.qe.devcluster.openshift.com
          port: 6443
        controlPlaneLoadBalancer:
          crossZoneLoadBalancing: false
          loadBalancerType: nlb
          name: huliu-aws1020b-rsrcn-int
          scheme: internet-facing
        identityRef:
          kind: AWSClusterControllerIdentity
          name: default
        network:
          vpc:
            availabilityZoneSelection: Ordered
            availabilityZoneUsageLimit: 3
            subnetSchema: PreferPrivate
        region: us-east-2
        secondaryControlPlaneLoadBalancer:
          crossZoneLoadBalancing: false
          loadBalancerType: nlb
          name: huliu-aws1020b-rsrcn-ext
          scheme: internet-facing
      status:
        ready: true
      
          2.Delete controlplanemachineset, then we can see master machines are sync to CAPI
      
      liuhuali@Lius-MacBook-Pro huali-test % oc delete controlplanemachineset cluster
      controlplanemachineset.machine.openshift.io "cluster" deleted
      liuhuali@Lius-MacBook-Pro huali-test % oc get machine.c -n openshift-cluster-api
      NAME                                           CLUSTER                NODENAME                                    PROVIDERID                              PHASE     AGE     VERSION
      huliu-aws1020b-rsrcn-master-0                  huliu-aws1020b-rsrcn   ip-10-0-92-153.us-east-2.compute.internal   aws:///us-east-2c/i-0fa92d671ac915000   Running   29s     
      huliu-aws1020b-rsrcn-master-1                  huliu-aws1020b-rsrcn   ip-10-0-30-170.us-east-2.compute.internal   aws:///us-east-2a/i-01f680f3429d29905   Running   29s     
      huliu-aws1020b-rsrcn-master-2                  huliu-aws1020b-rsrcn   ip-10-0-44-101.us-east-2.compute.internal   aws:///us-east-2b/i-03991c5f2f7c5b71c   Running   29s     
      huliu-aws1020b-rsrcn-worker-us-east-2a-nvmkq   huliu-aws1020b-rsrcn   ip-10-0-1-75.us-east-2.compute.internal     aws:///us-east-2a/i-0ddb4b632cac3ce52   Running   48m     
      huliu-aws1020b-rsrcn-worker-us-east-2b-ncxqh   huliu-aws1020b-rsrcn   ip-10-0-59-127.us-east-2.compute.internal   aws:///us-east-2b/i-0ccb851b773bee346   Running   48m     
      huliu-aws1020b-rsrcn-worker-us-east-2c-fd9gn   huliu-aws1020b-rsrcn   ip-10-0-95-168.us-east-2.compute.internal   aws:///us-east-2c/i-07fe7af0a98b5b739   Running   48m     
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get awsmachine -n openshift-cluster-api
      NAME                                           CLUSTER                STATE     READY   INSTANCEID                              MACHINE
      huliu-aws1020b-rsrcn-master-0                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2c/i-0fa92d671ac915000   huliu-aws1020b-rsrcn-master-0
      huliu-aws1020b-rsrcn-master-1                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2a/i-01f680f3429d29905   huliu-aws1020b-rsrcn-master-1
      huliu-aws1020b-rsrcn-master-2                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2b/i-03991c5f2f7c5b71c   huliu-aws1020b-rsrcn-master-2
      huliu-aws1020b-rsrcn-worker-us-east-2a-nvmkq   huliu-aws1020b-rsrcn   running   true    aws:///us-east-2a/i-0ddb4b632cac3ce52   huliu-aws1020b-rsrcn-worker-us-east-2a-nvmkq
      huliu-aws1020b-rsrcn-worker-us-east-2b-ncxqh   huliu-aws1020b-rsrcn   running   true    aws:///us-east-2b/i-0ccb851b773bee346   huliu-aws1020b-rsrcn-worker-us-east-2b-ncxqh
      huliu-aws1020b-rsrcn-worker-us-east-2c-fd9gn   huliu-aws1020b-rsrcn   running   true    aws:///us-east-2c/i-07fe7af0a98b5b739   huliu-aws1020b-rsrcn-worker-us-east-2c-fd9gn
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get machine -oyaml|grep -B7 "type: Synchronized"
            status: "False"
            type: Paused
          - lastTransitionTime: "2025-10-20T04:49:19Z"
            message: Successfully synchronized MAPI Machine to CAPI
            reason: ResourceSynchronized
            severity: ""
            status: "True"
            type: Synchronized
      --
            status: "False"
            type: Paused
          - lastTransitionTime: "2025-10-20T04:49:19Z"
            message: Successfully synchronized MAPI Machine to CAPI
            reason: ResourceSynchronized
            severity: ""
            status: "True"
            type: Synchronized
      --
            status: "False"
            type: Paused
          - lastTransitionTime: "2025-10-20T04:49:19Z"
            message: Successfully synchronized MAPI Machine to CAPI
            reason: ResourceSynchronized
            severity: ""
            status: "True"
            type: Synchronized
      --
            status: "False"
            type: Paused
          - lastTransitionTime: "2025-10-20T04:00:50Z"
            message: Successfully synchronized MAPI Machine to CAPI
            reason: ResourceSynchronized
            severity: ""
            status: "True"
            type: Synchronized
      --
            status: "False"
            type: Paused
          - lastTransitionTime: "2025-10-20T04:00:50Z"
            message: Successfully synchronized MAPI Machine to CAPI
            reason: ResourceSynchronized
            severity: ""
            status: "True"
            type: Synchronized
      --
            status: "False"
            type: Paused
          - lastTransitionTime: "2025-10-20T04:00:50Z"
            message: Successfully synchronized MAPI Machine to CAPI
            reason: ResourceSynchronized
            severity: ""
            status: "True"
            type: Synchronized
      
      
           3.Create master-user-data in openshift-cluster-api namespace
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get secret worker-user-data -oyaml>s1.yaml
      liuhuali@Lius-MacBook-Pro huali-test % vim s1.yaml 
      
      change name from worker-user-data to master-user-data
      
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f s1.yaml 
      secret/master-user-data created
      liuhuali@Lius-MacBook-Pro huali-test % oc get secret
      NAME                                         TYPE                                  DATA   AGE
      builder-dockercfg-42ph7                      kubernetes.io/dockercfg               1      61m
      capa-controller-manager-dockercfg-49xq4      kubernetes.io/dockercfg               1      61m
      capa-manager-bootstrap-credentials           Opaque                                3      74m
      capa-webhook-service-cert                    kubernetes.io/tls                     2      71m
      capi-manager-dockercfg-v2s5t                 kubernetes.io/dockercfg               1      61m
      capi-webhook-service-cert                    kubernetes.io/tls                     2      71m
      cluster-capi-operator-dockercfg-htm28        kubernetes.io/dockercfg               1      61m
      cluster-capi-operator-secret                 kubernetes.io/service-account-token   4      4m8s
      cluster-capi-operator-webhook-service-cert   kubernetes.io/tls                     2      72m
      default-dockercfg-vldj7                      kubernetes.io/dockercfg               1      61m
      deployer-dockercfg-gxbn6                     kubernetes.io/dockercfg               1      61m
      huliu-aws1020b-rsrcn-kubeconfig              cluster.x-k8s.io/secret               1      72m
      master-user-data                             Opaque                                2      8s
      worker-user-data                             Opaque                                2      72m
      
      4.Create CAPI master machine and awsmachine
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get machine.c huliu-aws1020b-rsrcn-master-0 -oyaml>ms1.yaml 
      liuhuali@Lius-MacBook-Pro huali-test % vim ms1.yaml 
      liuhuali@Lius-MacBook-Pro huali-test % oc get awsmachine huliu-aws1020b-rsrcn-master-0 -oyaml>ms2.yaml
      liuhuali@Lius-MacBook-Pro huali-test % vim ms2.yaml 
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms1.yaml 
      machine.cluster.x-k8s.io/huliu-aws1020b-rsrcn-master-3 created
      liuhuali@Lius-MacBook-Pro huali-test % oc create -f ms2.yaml 
      awsmachine.infrastructure.cluster.x-k8s.io/huliu-aws1020b-rsrcn-master-3 created
      
      liuhuali@Lius-MacBook-Pro huali-test % cat ms1.yaml 
      apiVersion: cluster.x-k8s.io/v1beta1
      kind: Machine
      metadata:
        labels:
          cluster.x-k8s.io/cluster-name: huliu-aws1020b-rsrcn
          machine.openshift.io/cluster-api-cluster: huliu-aws1020b-rsrcn
          machine.openshift.io/instance-type: m6i.xlarge
          machine.openshift.io/region: us-east-2
          machine.openshift.io/zone: us-east-2c
          node-role.kubernetes.io/master: ""
        name: huliu-aws1020b-rsrcn-master-3
        namespace: openshift-cluster-api
      spec:
        bootstrap:
          dataSecretName: master-user-data
        clusterName: huliu-aws1020b-rsrcn
        failureDomain: us-east-2b
        infrastructureRef:
          apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
          kind: AWSMachine
          name: huliu-aws1020b-rsrcn-master-3
          namespace: openshift-cluster-api
        nodeDeletionTimeout: 10s
      liuhuali@Lius-MacBook-Pro huali-test % cat ms2.yaml 
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
      kind: AWSMachine
      metadata:
        labels:
          cluster.x-k8s.io/cluster-name: huliu-aws1020b-rsrcn
          machine.openshift.io/cluster-api-cluster: huliu-aws1020b-rsrcn
          machine.openshift.io/instance-type: m6i.xlarge
          machine.openshift.io/region: us-east-2
          machine.openshift.io/zone: us-east-2c
          node-role.kubernetes.io/master: ""
        name: huliu-aws1020b-rsrcn-master-3
        namespace: openshift-cluster-api
      spec:
        additionalSecurityGroups:
        - filters:
          - name: tag:Name
            values:
            - huliu-aws1020b-rsrcn-node
        - filters:
          - name: tag:Name
            values:
            - huliu-aws1020b-rsrcn-lb
        - filters:
          - name: tag:Name
            values:
            - huliu-aws1020b-rsrcn-controlplane
        additionalTags:
          kubernetes.io/cluster/huliu-aws1020b-rsrcn: owned
        ami:
          id: ami-034f9c3fdfc80a314
        cloudInit: {}
        iamInstanceProfile: huliu-aws1020b-rsrcn-master-profile
        ignition:
          storageType: UnencryptedUserData
        instanceMetadataOptions:
          httpEndpoint: enabled
          httpPutResponseHopLimit: 1
          httpTokens: optional
          instanceMetadataTags: disabled
        instanceType: m6i.xlarge
        rootVolume:
          encrypted: true
          size: 120
          type: gp3
        subnet:
          filters:
          - name: tag:Name
            values:
            - huliu-aws1020b-rsrcn-subnet-private-us-east-2b
      liuhuali@Lius-MacBook-Pro huali-test % 
      
      5. The CAPI master machine get Running, but check on AWS console, the master machine is not associated to the loadbalancers defined in AWSCluster.
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get machine.c 
      NAME                                           CLUSTER                NODENAME                                    PROVIDERID                              PHASE     AGE   VERSION
      huliu-aws1020b-rsrcn-master-0                  huliu-aws1020b-rsrcn   ip-10-0-92-153.us-east-2.compute.internal   aws:///us-east-2c/i-0fa92d671ac915000   Running   20m   
      huliu-aws1020b-rsrcn-master-1                  huliu-aws1020b-rsrcn   ip-10-0-30-170.us-east-2.compute.internal   aws:///us-east-2a/i-01f680f3429d29905   Running   20m   
      huliu-aws1020b-rsrcn-master-2                  huliu-aws1020b-rsrcn   ip-10-0-44-101.us-east-2.compute.internal   aws:///us-east-2b/i-03991c5f2f7c5b71c   Running   20m   
      huliu-aws1020b-rsrcn-master-3                  huliu-aws1020b-rsrcn   ip-10-0-45-95.us-east-2.compute.internal    aws:///us-east-2b/i-09d7524bc421d141d   Running   11m   
      huliu-aws1020b-rsrcn-worker-us-east-2a-nvmkq   huliu-aws1020b-rsrcn   ip-10-0-1-75.us-east-2.compute.internal     aws:///us-east-2a/i-0ddb4b632cac3ce52   Running   69m   
      huliu-aws1020b-rsrcn-worker-us-east-2b-ncxqh   huliu-aws1020b-rsrcn   ip-10-0-59-127.us-east-2.compute.internal   aws:///us-east-2b/i-0ccb851b773bee346   Running   69m   
      huliu-aws1020b-rsrcn-worker-us-east-2c-fd9gn   huliu-aws1020b-rsrcn   ip-10-0-95-168.us-east-2.compute.internal   aws:///us-east-2c/i-07fe7af0a98b5b739   Running   69m   
      
      
      liuhuali@Lius-MacBook-Pro huali-test % oc get awsmachine 
      NAME                                           CLUSTER                STATE     READY   INSTANCEID                              MACHINE
      huliu-aws1020b-rsrcn-master-0                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2c/i-0fa92d671ac915000   huliu-aws1020b-rsrcn-master-0
      huliu-aws1020b-rsrcn-master-1                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2a/i-01f680f3429d29905   huliu-aws1020b-rsrcn-master-1
      huliu-aws1020b-rsrcn-master-2                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2b/i-03991c5f2f7c5b71c   huliu-aws1020b-rsrcn-master-2
      huliu-aws1020b-rsrcn-master-3                  huliu-aws1020b-rsrcn   running   true    aws:///us-east-2b/i-09d7524bc421d141d   huliu-aws1020b-rsrcn-master-3
      huliu-aws1020b-rsrcn-worker-us-east-2a-nvmkq   huliu-aws1020b-rsrcn   running   true    aws:///us-east-2a/i-0ddb4b632cac3ce52   huliu-aws1020b-rsrcn-worker-us-east-2a-nvmkq
      huliu-aws1020b-rsrcn-worker-us-east-2b-ncxqh   huliu-aws1020b-rsrcn   running   true    aws:///us-east-2b/i-0ccb851b773bee346   huliu-aws1020b-rsrcn-worker-us-east-2b-ncxqh
      huliu-aws1020b-rsrcn-worker-us-east-2c-fd9gn   huliu-aws1020b-rsrcn   running   true    aws:///us-east-2c/i-07fe7af0a98b5b739   huliu-aws1020b-rsrcn-worker-us-east-2c-fd9gn
      
      loadbalancer Resource map on AWS console
      https://drive.google.com/file/d/1j9hpTE8QN1P7SHlsfyy0XYjPwW_5lBtM/view?usp=sharing
      https://drive.google.com/file/d/1WdjEPDMXdkj67w0nQGBMMXnDHdneWh9J/view?usp=sharing
      
           

      Actual results:

          The newly created CAPI master machine isn't associated to the loadbalancers defined in the AWSCluster.

      Expected results:

          The newly created CAPI master machine should be associated to the loadbalancers defined in the AWSCluster.

      Additional info:

          must-gather: https://drive.google.com/file/d/1l8oJKc-nbpV_qmCCLUmHrkOrMBEPIAPi/view?usp=sharing
      
      new feature testing for https://issues.redhat.com/browse/OCPCLOUD-2709

              rmanak@redhat.com Radek Manak
              huliu@redhat.com Huali Liu
              None
              None
              Huali Liu Huali Liu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: