W0404 13:26:44.830457 1 cmd.go:245] Using insecure, self-signed certificates I0404 13:26:45.302329 1 leaderelection.go:122] The leader election gives 4 retries and allows for 30s of clock skew. The kube-apiserver downtime tolerance is 78s. Worst non-graceful lease acquisition is 2m43s. Worst graceful lease acquisition is {26s}. I0404 13:26:45.302631 1 observer_polling.go:159] Starting file observer I0404 13:26:45.337508 1 builder.go:299] submariner-agent version v0.0.0-unknown-- I0404 13:26:46.229508 1 secure_serving.go:57] Forcing use of http/1.1 only W0404 13:26:46.229530 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256' detected. W0404 13:26:46.229535 1 secure_serving.go:69] Use of insecure cipher 'TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256' detected. I0404 13:26:46.233338 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController I0404 13:26:46.233366 1 shared_informer.go:311] Waiting for caches to sync for RequestHeaderAuthRequestController I0404 13:26:46.233360 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::client-ca-file" I0404 13:26:46.233379 1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0404 13:26:46.233392 1 configmap_cafile_content.go:202] "Starting controller" name="client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file" I0404 13:26:46.233401 1 shared_informer.go:311] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0404 13:26:46.233924 1 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/tmp/serving-cert-2561030296/tls.crt::/tmp/serving-cert-2561030296/tls.key" I0404 13:26:46.234077 1 secure_serving.go:213] Serving securely on [::]:8443 I0404 13:26:46.234117 1 tlsconfig.go:240] "Starting DynamicServingCertificateController" I0404 13:26:46.235361 1 leaderelection.go:250] attempting to acquire leader lease submariner-operator/submariner-agent-lock... I0404 13:26:46.253066 1 leaderelection.go:260] successfully acquired lease submariner-operator/submariner-agent-lock I0404 13:26:46.253191 1 event.go:364] Event(v1.ObjectReference{Kind:"Lease", Namespace:"submariner-operator", Name:"submariner-agent-lock", UID:"29e32574-8b88-4112-9a55-f214f499f69a", APIVersion:"coordination.k8s.io/v1", ResourceVersion:"2559205", FieldPath:""}): type: 'Normal' reason: 'LeaderElection' submariner-addon-6859f676c4-rlsh5_7992ebee-886d-4e67-baab-e66f10c6741e became leader I0404 13:26:46.269677 1 base_controller.go:67] Waiting for caches to sync for SubmarinerConfigController I0404 13:26:46.269778 1 base_controller.go:67] Waiting for caches to sync for DeploymentStatusController I0404 13:26:46.269811 1 base_controller.go:67] Waiting for caches to sync for GatewaysStatusController I0404 13:26:46.269846 1 base_controller.go:67] Waiting for caches to sync for ConnectionsStatusController W0404 13:26:46.273548 1 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource E0404 13:26:46.273606 1 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch submariner.io/v1alpha1, Resource=submariners: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource I0404 13:26:46.334202 1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file I0404 13:26:46.334343 1 shared_informer.go:318] Caches are synced for RequestHeaderAuthRequestController I0404 13:26:46.334450 1 shared_informer.go:318] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file I0404 13:26:46.369990 1 base_controller.go:73] Caches are synced for SubmarinerConfigController I0404 13:26:46.370110 1 base_controller.go:110] Starting #1 worker of SubmarinerConfigController controller ... I0404 13:26:46.370021 1 base_controller.go:73] Caches are synced for GatewaysStatusController I0404 13:26:46.370246 1 base_controller.go:110] Starting #1 worker of GatewaysStatusController controller ... 2024-04-04T13:26:46.403Z INF ..eways_controller.go:100 ..ysStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerGatewayNodesLabeled", "status": "False", "lastTransitionTime": null, "reason": "SubmarinerGatewayNodesUnlabeled", "message": "There are no nodes with label \"submariner.io/gateway\"" } I0404 13:26:46.403939 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"Unknown", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 37, 0, time.Local), Reason:"ManagedClusterAddOnLeaseNotFound", Message:"The status of submariner add-on is unknown."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}} 2024-04-04T13:26:46.421Z INF ..onfig_controller.go:654 ..erConfigController SubmarinerConfig network type was set to "OVNKubernetes" for managed cluster "rdr-c1" I0404 13:26:46.421701 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SubmarinerConfigNetworkTypeSet' SubmarinerConfig network type was set to "OVNKubernetes" for managed cluster "rdr-c1" I0404 13:26:46.612674 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic W0404 13:26:47.745489 1 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource E0404 13:26:47.745520 1 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch submariner.io/v1alpha1, Resource=submariners: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource W0404 13:26:50.323830 1 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource E0404 13:26:50.323865 1 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch submariner.io/v1alpha1, Resource=submariners: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource I0404 13:26:51.083921 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:26:53.605369 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:26:53.614870 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:26:53.658229 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:26:53.658383 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SubmarinerClusterEnvPrepared' submariner cluster environment was prepared for managed cluster rdr-c1 2024-04-04T13:26:53.697Z INF ..onfig_controller.go:654 ..erConfigController SubmarinerConfig network type was set to "OVNKubernetes" for managed cluster "rdr-c1" I0404 13:26:53.698288 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SubmarinerConfigNetworkTypeSet' SubmarinerConfig network type was set to "OVNKubernetes" for managed cluster "rdr-c1" I0404 13:26:53.750889 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:26:56.103484 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes W0404 13:26:56.465080 1 reflector.go:539] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource E0404 13:26:56.465133 1 reflector.go:147] k8s.io/client-go@v0.29.2/tools/cache/reflector.go:229: Failed to watch submariner.io/v1alpha1, Resource=submariners: failed to list submariner.io/v1alpha1, Resource=submariners: the server could not find the requested resource I0404 13:26:58.995641 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:26:59.023728 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:26:59.032917 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" 2024-04-04T13:26:59.083Z INF ..onfig_controller.go:654 ..erConfigController SubmarinerConfig network type was set to "OVNKubernetes" for managed cluster "rdr-c1" I0404 13:26:59.083307 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SubmarinerConfigNetworkTypeSet' SubmarinerConfig network type was set to "OVNKubernetes" for managed cluster "rdr-c1" I0404 13:26:59.139869 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:01.428394 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:03.805343 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:03.818888 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:03.831276 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:03.907933 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:06.989424 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:09.070518 1 base_controller.go:73] Caches are synced for ConnectionsStatusController I0404 13:27:09.070634 1 base_controller.go:110] Starting #1 worker of ConnectionsStatusController controller ... I0404 13:27:09.070564 1 base_controller.go:73] Caches are synced for DeploymentStatusController I0404 13:27:09.070681 1 base_controller.go:110] Starting #1 worker of DeploymentStatusController controller ... 2024-04-04T13:27:09.106Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoGatewayDaemonSet,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet", "message": "The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThe gateway daemon set does not exist\nThe route agents are not found\nThe metrics proxy daemon set does not exist" } I0404 13:27:09.107216 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoGatewayDaemonSet,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet", Message:"The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThe gateway daemon set does not exist\nThe route agents are not found\nThe metrics proxy daemon set does not exist"}} I0404 13:27:09.357007 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:09.365200 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:09.373836 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:09.417790 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:12.298251 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:14.661347 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:14.692110 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:14.707004 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:14.778591 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic 2024-04-04T13:27:17.356Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoGatewayDaemonSet,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", "message": "The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThe gateway daemon set does not exist\nThe route agents are not found\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist" } I0404 13:27:17.357052 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoGatewayDaemonSet,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", Message:"The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThe gateway daemon set does not exist\nThe route agents are not found\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist"}} 2024-04-04T13:27:17.468Z INF ..ctions_controller.go:90 ..nsStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerConnectionDegraded", "status": "True", "lastTransitionTime": null, "reason": "ConnectionsNotEstablished", "message": "There are no connections on gateways" } I0404 13:27:17.468280 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoGatewayDaemonSet,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", Message:"The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThe gateway daemon set does not exist\nThe route agents are not found\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:27:17.509323 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes 2024-04-04T13:27:17.609Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoScheduledGateways,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", "message": "The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThe route agents are not found\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist" } I0404 13:27:17.610263 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoScheduledGateways,NoRouteAgentDaemonSet,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", Message:"The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThe route agents are not found\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:18.147Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoScheduledGateways,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", "message": "The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist" } I0404 13:27:18.147576 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoScheduledGateways,NoMetricsProxyDaemonSet,NoGlobalnetDaemonSet", Message:"The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThe metrics proxy daemon set does not exist\nThe globalnet daemon set does not exist"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:18.544Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:18.544997 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentDeployment,NoLighthouseCoreDNSDeployment,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"The submariner lighthouse agent deployment does not exist\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:18.966Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentAvailable,NoLighthouseCoreDNSDeployment,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no submariner lighthouse agent replica available\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:18.966627 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentAvailable,NoLighthouseCoreDNSDeployment,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no submariner lighthouse agent replica available\nThe submariner lighthouse coredns deployment does not exist\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:19.542Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentAvailable,NoLighthouseCoreDNSAvailable,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no submariner lighthouse agent replica available\nThere are no submariner lighthouse coredns replica available\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:19.544360 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentAvailable,NoLighthouseCoreDNSAvailable,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no submariner lighthouse agent replica available\nThere are no submariner lighthouse coredns replica available\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:27:20.384809 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:20.400391 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:20.427630 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:20.491358 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:23.114500 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes 2024-04-04T13:27:24.840Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoLighthouseAgentAvailable,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no submariner lighthouse agent replica available\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:24.840561 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoLighthouseAgentAvailable,NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no submariner lighthouse agent replica available\nThere are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:24.914Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:24.914853 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are 6 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:27:25.484083 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:25.498714 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:25.507269 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:25.562296 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic 2024-04-04T13:27:25.942Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are 5 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:25.942576 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are 5 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:25.982Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are 4 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:25.982504 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are 4 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:27.469Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are 3 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:27.469519 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are 3 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:27:28.262512 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes 2024-04-04T13:27:28.482Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are 2 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:28.483067 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are 2 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:27:29.485Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are 1 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:29.485643 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,RouteAgentsUnavailable,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are 1 unavailable route agents\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:27:30.619048 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:30.629877 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:30.637328 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:30.698671 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic 2024-04-04T13:27:32.294Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,NoScheduledMetricsProxy,NoScheduledGlobalnet", "message": "There are no nodes to run the gateways\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods" } I0404 13:27:32.294568 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 46, 0, time.Local), Reason:"SubmarinerGatewayNodesUnlabeled", Message:"There are no nodes with label \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:27:32.864077 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:35.167444 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:35.174424 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:35.182419 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:35.226701 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:38.201519 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:41.179675 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:41.187289 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:41.239029 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:41.293392 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:44.068077 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:46.103201 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:46.110453 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:46.123691 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:46.180250 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:49.270011 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:52.523722 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:52.529900 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:52.638846 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:52.790340 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:27:55.187798 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:27:57.313222 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:27:57.320652 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:27:57.327150 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:27:57.434972 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:28:00.455917 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:28:02.526576 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:28:02.533205 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:28:02.547755 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:28:02.949388 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:28:05.897729 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:28:08.372200 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:28:08.380084 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:28:08.387144 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:28:08.454267 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:28:11.407389 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:28:13.932028 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:28:13.942819 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:28:13.958266 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:28:22.499477 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:28:25.408091 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:28:28.183089 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:28:28.194512 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:28:28.204285 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:29:10.453909 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:29:13.587838 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:29:15.953884 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:29:15.961168 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:29:15.968123 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:29:30.423114 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:29:33.711139 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:29:35.889972 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:29:35.897623 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:29:35.908349 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:29:51.527500 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:29:54.514027 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:29:56.679543 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:29:56.690809 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:29:56.709193 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:29:56.789677 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:29:59.623593 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:30:03.030080 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:30:03.036791 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:30:03.066677 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:30:15.065788 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:30:25.600459 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:30:27.674859 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:30:27.682775 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:30:27.712189 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:30:57.031455 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:31:00.381335 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:31:03.716307 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:31:03.754660 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:31:03.762154 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:31:06.380032 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:31:08.928482 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:31:11.860555 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:31:11.869230 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:31:11.880456 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:31:23.599551 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:31:26.016505 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:31:28.527333 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:31:28.539831 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:31:28.548854 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:31:30.120765 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:31:33.193193 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:31:35.239344 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:31:35.261497 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:31:35.269703 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:31:35.589722 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:31:38.109092 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:31:40.386376 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:31:40.396557 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:31:40.411265 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" 2024-04-04T13:32:21.153Z INF ..eways_controller.go:100 ..ysStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerGatewayNodesLabeled", "status": "True", "lastTransitionTime": null, "reason": "SubmarinerGatewayNodesLabeled", "message": "The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\"" } I0404 13:32:21.153254 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,NoScheduledMetricsProxy,NoScheduledGlobalnet", Message:"There are no nodes to run the gateways\nThere are no nodes to run the metrics proxy\nThere are no nodes to run the globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:32:21.216337 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic 2024-04-04T13:32:24.898Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "NoScheduledGateways,MetricsProxyUnavailable,GlobalnetUnavailable", "message": "There are no nodes to run the gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods" } I0404 13:32:24.898312 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"NoScheduledGateways,MetricsProxyUnavailable,GlobalnetUnavailable", Message:"There are no nodes to run the gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:32:25.000Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable,MetricsProxyUnavailable,GlobalnetUnavailable", "message": "There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods" } I0404 13:32:25.000753 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"GatewaysUnavailable,MetricsProxyUnavailable,GlobalnetUnavailable", Message:"There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:32:25.657487 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:32:28.243847 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:32:28.251765 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:32:28.268115 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:32:28.312828 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:32:31.102471 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes 2024-04-04T13:32:32.097Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "MetricsProxyUnavailable,GlobalnetUnavailable", "message": "There are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods" } I0404 13:32:32.097484 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"MetricsProxyUnavailable,GlobalnetUnavailable", Message:"There are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:32:32.172Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "MetricsProxyUnavailable", "message": "There are 1 unavailable metrics proxy pods" } I0404 13:32:32.172847 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 9, 0, time.Local), Reason:"MetricsProxyUnavailable", Message:"There are 1 unavailable metrics proxy pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:32:33.132Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "False", "lastTransitionTime": null, "reason": "SubmarinerAgentDeployed", "message": "Submariner (submariner.v0.17.0) is deployed on managed cluster." } I0404 13:32:33.132721 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 33, 0, time.Local), Reason:"SubmarinerAgentDeployed", Message:"Submariner (submariner.v0.17.0) is deployed on managed cluster."}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:32:34.311334 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:32:34.323300 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:32:34.338313 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:32:34.385766 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:32:37.695456 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:32:40.539671 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:32:40.545874 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:32:40.552509 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:32:40.603100 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:32:45.912171 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:32:48.331430 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:32:48.339300 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:32:48.347183 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" 2024-04-04T13:32:58.968Z INF ..eways_controller.go:100 ..ysStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerGatewayNodesLabeled", "status": "True", "lastTransitionTime": null, "reason": "SubmarinerGatewayNodesLabeled", "message": "The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\"" } I0404 13:32:58.968935 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 33, 0, time.Local), Reason:"SubmarinerAgentDeployed", Message:"Submariner (submariner.v0.17.0) is deployed on managed cluster."}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:32:59.050Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "MetricsProxyUnavailable", "message": "There are 1 unavailable metrics proxy pods" } I0404 13:32:59.050638 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 59, 0, time.Local), Reason:"MetricsProxyUnavailable", Message:"There are 1 unavailable metrics proxy pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:32:59.096Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable,MetricsProxyUnavailable", "message": "There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods" } I0404 13:32:59.096264 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 59, 0, time.Local), Reason:"GatewaysUnavailable,MetricsProxyUnavailable", Message:"There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:32:59.149Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable,MetricsProxyUnavailable,GlobalnetUnavailable", "message": "There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods" } I0404 13:32:59.149619 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 59, 0, time.Local), Reason:"GatewaysUnavailable,MetricsProxyUnavailable,GlobalnetUnavailable", Message:"There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:32:59.223785 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic 2024-04-04T13:33:03.764Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable,MetricsProxyUnavailable", "message": "There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods" } I0404 13:33:03.764660 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 59, 0, time.Local), Reason:"GatewaysUnavailable,MetricsProxyUnavailable", Message:"There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:33:04.596010 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes 2024-04-04T13:33:05.862Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "MetricsProxyUnavailable", "message": "There are 1 unavailable metrics proxy pods" } I0404 13:33:05.862983 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 59, 0, time.Local), Reason:"MetricsProxyUnavailable", Message:"There are 1 unavailable metrics proxy pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:05.949Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "False", "lastTransitionTime": null, "reason": "SubmarinerAgentDeployed", "message": "Submariner (submariner.v0.17.0) is deployed on managed cluster." } I0404 13:33:05.949624 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 5, 0, time.Local), Reason:"SubmarinerAgentDeployed", Message:"Submariner (submariner.v0.17.0) is deployed on managed cluster."}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:33:07.147750 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:33:07.157533 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:33:07.164302 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:33:07.215561 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic I0404 13:33:10.891077 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:33:13.776335 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Deploying dedicated Submariner gateway node rdr-c1-x8qgz-submariner-gw0 I0404 13:33:13.784182 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Warning' reason: 'RHOSCloudProvider' Unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" E0404 13:33:13.791063 1 base_controller.go:268] SubmarinerConfigController reconciliation failed: unable to deploy gateway: error getting the worker image: could not retrieve the image of one of the worker nodes from the infra "rdr-c1-x8qgz" I0404 13:33:28.155409 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Configuring the required firewall rules for inter-cluster traffic 2024-04-04T13:33:28.183Z INF ..eways_controller.go:100 ..ysStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerGatewayNodesLabeled", "status": "True", "lastTransitionTime": null, "reason": "SubmarinerGatewayNodesLabeled", "message": "The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\"" } I0404 13:33:28.183952 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 5, 0, time.Local), Reason:"SubmarinerAgentDeployed", Message:"Submariner (submariner.v0.17.0) is deployed on managed cluster."}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:28.258Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GlobalnetUnavailable", "message": "There are 1 unavailable globalnet pods" } I0404 13:33:28.258381 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 28, 0, time.Local), Reason:"GlobalnetUnavailable", Message:"There are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:28.288Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "MetricsProxyUnavailable,GlobalnetUnavailable", "message": "There are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods" } I0404 13:33:28.288646 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 28, 0, time.Local), Reason:"MetricsProxyUnavailable,GlobalnetUnavailable", Message:"There are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:28.379Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable,MetricsProxyUnavailable,GlobalnetUnavailable", "message": "There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods" } I0404 13:33:28.379526 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 28, 0, time.Local), Reason:"GatewaysUnavailable,MetricsProxyUnavailable,GlobalnetUnavailable", Message:"There are 1 unavailable gateways\nThere are 1 unavailable metrics proxy pods\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:33.526Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable,GlobalnetUnavailable", "message": "There are 1 unavailable gateways\nThere are 1 unavailable globalnet pods" } I0404 13:33:33.526873 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 28, 0, time.Local), Reason:"GatewaysUnavailable,GlobalnetUnavailable", Message:"There are 1 unavailable gateways\nThere are 1 unavailable globalnet pods"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:33.568Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "True", "lastTransitionTime": null, "reason": "GatewaysUnavailable", "message": "There are 1 unavailable gateways" } I0404 13:33:33.569026 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 28, 0, time.Local), Reason:"GatewaysUnavailable", Message:"There are 1 unavailable gateways"}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} 2024-04-04T13:33:33.636Z INF ..yment_controller.go:151 ..ntStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerAgentDegraded", "status": "False", "lastTransitionTime": null, "reason": "SubmarinerAgentDeployed", "message": "Submariner (submariner.v0.17.0) is deployed on managed cluster." } I0404 13:33:33.636394 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 33, 0, time.Local), Reason:"SubmarinerAgentDeployed", Message:"Submariner (submariner.v0.17.0) is deployed on managed cluster."}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsNotEstablished", Message:"There are no connections on gateways"}} I0404 13:33:33.724770 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Opened external ports "4500/udp, 4900/udp, 0/esp, 0/ah" in security group "rdr-c1-x8qgz-submariner-gw-sg" on RHOS for existing g/w nodes I0404 13:33:33.724801 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' Current Submariner gateways match the required number of Submariner gateways I0404 13:33:33.724811 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'RHOSCloudProvider' The Submariner cluster environment has been set up on RHOS I0404 13:33:33.752699 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SubmarinerClusterEnvPrepared' submariner cluster environment was prepared for managed cluster rdr-c1 2024-04-04T13:33:33.752Z INF ..onfig_controller.go:283 ..erConfigController Submariner environment was prepared for cluster "rdr-c1": v1alpha1.ManagedClusterInfo{ClusterName:"rdr-c1", Vendor:"OpenShift", Platform:"OpenStack", Region:"", InfraID:"rdr-c1-x8qgz", VendorVersion:"4.15.6", NetworkType:"OVNKubernetes"} 2024-04-04T13:33:33.773Z INF ..onfig_controller.go:326 ..erConfigController Updated SubmarinerConfig status condition: { "type": "SubmarinerGatewaysLabeled", "status": "True", "lastTransitionTime": null, "reason": "Success", "message": "3 node(s) (\"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\") are labeled as gateways" } I0404 13:33:33.773483 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'SubmarinerConfigStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SubmarinerConfigApplied", Message:"SubmarinerConfig was applied"}, v1.Condition{Type:"SubmarinerClusterEnvironmentPrepared", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 33, 0, time.Local), Reason:"SubmarinerClusterEnvPrepared", Message:"Submariner cluster environment was prepared"}, v1.Condition{Type:"SubmarinerGatewaysLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 33, 0, time.Local), Reason:"Success", Message:"3 node(s) (\"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\") are labeled as gateways"}} 2024-04-04T13:33:36.040Z INF ..ctions_controller.go:90 ..nsStatusController Updated submariner ManagedClusterAddOn status condition: { "type": "SubmarinerConnectionDegraded", "status": "True", "lastTransitionTime": null, "reason": "ConnectionsDegraded", "message": "The connection between clusters \"rdr-c1\" and \"rdr-c2\" is not established (status=connecting)" } I0404 13:33:36.040849 1 event.go:364] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"submariner-operator", Name:"submariner-addon", UID:"d6a62cca-4e57-40aa-902e-f19d4647b995", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'ManagedClusterAddOnStatusUpdated' Updated status conditions: []v1.Condition{v1.Condition{Type:"SubmarinerBrokerConfigApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"BrokerConfigApplied", Message:""}, v1.Condition{Type:"RegistrationApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 34, 0, time.Local), Reason:"SetPermissionApplied", Message:"Registration of the addon agent is configured"}, v1.Condition{Type:"ClusterCertificateRotated", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 35, 0, time.Local), Reason:"ClientCertificateUpdated", Message:"client certificate rotated starting from 2024-04-04 13:21:34 +0000 UTC to 2024-04-21 05:03:18 +0000 UTC"}, v1.Condition{Type:"ManifestApplied", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 26, 36, 0, time.Local), Reason:"AddonManifestApplied", Message:"manifests of addon are applied successfully"}, v1.Condition{Type:"Available", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 7, 0, time.Local), Reason:"ManagedClusterAddOnLeaseUpdated", Message:"submariner add-on is available."}, v1.Condition{Type:"SubmarinerGatewayNodesLabeled", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 32, 21, 0, time.Local), Reason:"SubmarinerGatewayNodesLabeled", Message:"The nodes \"rdr-c1-x8qgz-worker-0-gjkz6,rdr-c1-x8qgz-worker-0-pw5tb,rdr-c1-x8qgz-worker-0-zqv6r\" are labeled with \"submariner.io/gateway\""}, v1.Condition{Type:"SubmarinerAgentDegraded", Status:"False", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 33, 33, 0, time.Local), Reason:"SubmarinerAgentDeployed", Message:"Submariner (submariner.v0.17.0) is deployed on managed cluster."}, v1.Condition{Type:"SubmarinerConnectionDegraded", Status:"True", ObservedGeneration:0, LastTransitionTime:time.Date(2024, time.April, 4, 13, 27, 17, 0, time.Local), Reason:"ConnectionsDegraded", Message:"The connection between clusters \"rdr-c1\" and \"rdr-c2\" is not established (status=connecting)"}}