-
Bug
-
Resolution: Done
-
Major
-
Submariner 0.12.2, ACM 2.5.5
Description of problem:
Installing Submariner 0.12.2 within ACM 2.5.5 failed to create the Gateway on AWS:
Error updating load balancer with new hosts map
Version-Release number of selected component (if applicable):
ACM 2.5.5
Submariner 0.12.2
Cloud platform: Amazon
OCP version: 4.10.44
kubernetes 1.23.12
### lighthouse-agent-rhel8:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:33 name=rhacm2/lighthouse-agent-rhel8 release=10 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/lighthouse-agent-rhel8/images/v0.12.2-10 version=v0.12.2 ### lighthouse-coredns-rhel8:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:22 name=rhacm2/lighthouse-coredns-rhel8 release=10 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/lighthouse-coredns-rhel8/images/v0.12.2-10 version=v0.12.2 ### submariner-gateway-rhel8:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:32 name=rhacm2/submariner-gateway-rhel8 release=9 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/submariner-gateway-rhel8/images/v0.12.2-9 version=v0.12.2 ### submariner-globalnet-rhel8:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:29 name=rhacm2/submariner-globalnet-rhel8 release=9 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/submariner-globalnet-rhel8/images/v0.12.2-9 version=v0.12.2 ### submariner-networkplugin-syncer-rhel8:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:26 name=rhacm2/submariner-networkplugin-syncer-rhel8 release=9 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/submariner-networkplugin-syncer-rhel8/images/v0.12.2-9 version=v0.12.2 ### submariner-operator-bundle:v0.12 Image-Stream tag ### build-date=2022-12-02T07:34:59 name=rhacm2/submariner-operator-bundle release=9 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/submariner-operator-bundle/images/v0.12.2-9 version=v0.12.2 ### submariner-rhel8-operator:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:40 name=rhacm2/submariner-rhel8-operator release=8 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/submariner-rhel8-operator/images/v0.12.2-8 version=v0.12.2 ### submariner-route-agent-rhel8:v0.12 Image-Stream tag ### build-date=2022-12-02T07:08:26 name=rhacm2/submariner-route-agent-rhel8 release=9 url =https://access.redhat.com/containers/#/registry.access.redhat.com/rhacm2/submariner-route-agent-rhel8/images/v0.12.2-9 version=v0.12.2
How reproducible:
?
Steps to Reproduce:
Install ACM 2.5.5 with Submariner 0.12.2 on AWS:
Actual results:
Name: submariner Namespace: acm-aws-nmanos-a2 Labels: <none> Annotations: <none> API Version: submarineraddon.open-cluster-management.io/v1alpha1 Kind: SubmarinerConfig Metadata: Creation Timestamp: 2022-12-06T06:03:55Z Finalizers: submarineraddon.open-cluster-management.io/config-cleanup Generation: 2 Managed Fields: API Version: submarineraddon.open-cluster-management.io/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:annotations: .: f:kubectl.kubernetes.io/last-applied-configuration: f:spec: .: f:IPSecIKEPort: f:IPSecNATTPort: f:NATTDiscoveryPort: f:NATTEnable: f:cableDriver: f:credentialsSecret: .: f:name: f:gatewayConfig: .: f:aws: .: f:instanceType: f:gateways: f:imagePullSpecs: f:loadBalancerEnable: f:subscriptionConfig: .: f:channel: f:source: f:sourceNamespace: f:startingCSV: Manager: kubectl-client-side-apply Operation: Update Time: 2022-12-06T06:03:55Z API Version: submarineraddon.open-cluster-management.io/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:metadata: f:finalizers: .: v:"submarineraddon.open-cluster-management.io/config-cleanup": f:spec: f:gatewayConfig: f:azure: .: f:instanceType: f:gcp: .: f:instanceType: f:rhos: .: f:instanceType: Manager: submariner Operation: Update Time: 2022-12-06T06:03:55Z API Version: submarineraddon.open-cluster-management.io/v1alpha1 Fields Type: FieldsV1 fieldsV1: f:status: .: f:conditions: f:managedClusterInfo: .: f:clusterName: f:infraId: f:platform: f:region: f:vendor: Manager: submariner Operation: Update Subresource: status Time: 2022-12-06T06:04:00Z Resource Version: 112556 UID: 80e4f944-e3ef-45a1-ab5c-37a37dc4d72c Spec: IP Sec IKE Port: 502 IP Sec NATT Port: 4502 NATT Discovery Port: 4900 NATT Enable: true Cable Driver: libreswan Credentials Secret: Name: acm-aws-nmanos-a2-aws-creds Gateway Config: Aws: Instance Type: c5d.large Azure: Instance Type: Standard_D4s_v3 Gateways: 1 Gcp: Instance Type: n1-standard-4 Rhos: Instance Type: PnTAE.CPU_16_Memory_32768_Disk_80 Image Pull Specs: Load Balancer Enable: false Subscription Config: Channel: stable-0.12 Source: submariner-stable-0-12-catalog Source Namespace: submariner-operator Starting CSV: submariner.v0.12 Status: Conditions: Last Transition Time: 2022-12-06T06:04:00Z Message: Submariner cluster environment was prepared Reason: SubmarinerClusterEnvPrepared Status: True Type: SubmarinerClusterEnvironmentPrepared Last Transition Time: 2022-12-06T06:04:01Z Message: SubmarinerConfig was applied Reason: SubmarinerConfigApplied Status: True Type: SubmarinerConfigApplied Last Transition Time: 2022-12-06T06:08:02Z Message: 1 node(s) ("ip-10-166-10-129.us-west-1.compute.internal") are labeled as gateways Reason: Success Status: True Type: SubmarinerGatewaysLabeled Managed Cluster Info: Cluster Name: acm-aws-nmanos-a2 Infra Id: aws-nmanos-a2-h79vd Platform: AWS Region: us-west-1 Vendor: OpenShift Events: <none>
$ oc get nodes -l node-role.kubernetes.io/worker -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME ip-10-166-10-129.us-west-1.compute.internal Ready worker 21m v1.23.12+8a6bfe4 10.166.10.129 18.144.167.96 Red Hat Enterprise Linux CoreOS 410.84.202211282118-0 (Ootpa) 4.18.0-305.65.1.el8_4.x86_64 cri-o://1.23.3-20.rhaos4.10.git89344de.el8 ip-10-166-154-70.us-west-1.compute.internal Ready worker 161m v1.23.12+8a6bfe4 10.166.154.70 <none> Red Hat Enterprise Linux CoreOS 410.84.202211282118-0 (Ootpa) 4.18.0-305.65.1.el8_4.x86_64 cri-o://1.23.3-20.rhaos4.10.git89344de.el8 ip-10-166-180-183.us-west-1.compute.internal Ready worker 164m v1.23.12+8a6bfe4 10.166.180.183 <none> Red Hat Enterprise Linux CoreOS 410.84.202211282118-0 (Ootpa) 4.18.0-305.65.1.el8_4.x86_64 cri-o://1.23.3-20.rhaos4.10.git89344de.el8 ip-10-166-255-43.us-west-1.compute.internal Ready worker 161m v1.23.12+8a6bfe4 10.166.255.43 <none> Red Hat Enterprise Linux CoreOS 410.84.202211282118-0 (Ootpa) 4.18.0-305.65.1.el8_4.x86_64 cri-o://1.23.3-20.rhaos4.10.git89344de.el8
$ oc describe Gateway -n submariner-operator
No resources found in submariner-operator namespace.
OC logs shows:
00:33:56 2022-12-06T06:08:39Z 2022-12-06T06:08:39Z 1 machineconfigdaemon Normal NodeDone Setting node ip-10-166-10-129.us-west-1.compute.internal, currentConfig rendered-worker-f46d0369e1183ea8248aec9bf8950b43 to Done 00:33:56 2022-12-06T06:08:43Z 2022-12-06T06:08:43Z 1 kubelet Normal NodeReady Node ip-10-166-10-129.us-west-1.compute.internal status is now: NodeReady 00:33:56 2022-12-06T06:08:44Z 2022-12-06T06:33:32Z 16 service-controller Warning
UpdateLoadBalancerFailed Error updating load balancer with new hosts map[ip-10-166-10-129.us-west-1.compute.internal:{} ip-10-166-129-205.us-west-1.compute.internal:{} ip-10-166-150-70.us-west-1.compute.internal:{} ip-10-166-154-70.us-west-1.compute.internal:{} ip-10-166-180-183.us-west-1.compute.internal:{} ip-10-166-198-122.us-west-1.compute.internal:{} ip-10-166-255-43.us-west-1.compute.internal:{}]: Multiple tagged security groups found for instance i-0e722eeedf0d69eeb; ensure only the k8s security group is tagged; the tagged groups were sg-07f460ad239364d9b(aws-nmanos-a2-h79vd-submariner-gw-sg) sg-0f8d89b4202126713(terraform-20221206033036145600000002)
Expected results:
Gateway should be created on the AWS node with the external IP.
Additional info:
More logs including Submariner gather logs in build artifacts.