Feature Overview (Goal Summary)

This feature explores the effects of enabling User Defined Networking (UDN) on the OpenShift hosting cluster (management cluster), where control planes are managed by the HyperShift operator. It examines how UDN impacts control-plane deployments, including network isolation per control plane, and specifies the necessary APIs to expose and configurations to support isolated hosted control planes using UDNs.

Goals (Expected User Outcomes)

• Understand the Impact of UDN on Hosted Control Planes:
  • Assess how enabling UDN on the management cluster affects control-plane pods for multiple clusters.
  • Determine any changes required for control-plane pod networking and connectivity when UDN is configured.

• Enable and Configure Isolation for Hosted Control Planes Using UDN:
  • Provide cluster service providers with the ability to isolate control planes corresponding to each cluster within the same management cluster.
  • Identify which APIs need to be exposed to configure UDN on hosted clusters, allowing for per-cluster control-plane network isolation.

Requirements (Acceptance Criteria)

• When UDN is enabled on the management cluster, each control-plane should have the ability to operate within its isolated UDN.
• HyperShift APIs APIs should allow specification of control-plane network settings (e.g., primary/secondary UDNs, network isolation parameters).

Impact of UDN on Control-Plane Deployments

Isolation of Control Planes:

• When UDN is configured on the management cluster, control-plane pods for each cluster are provisioned into isolated network namespaces based on cluster-specific UDNs.
• Example: If a management cluster has UDN enabled, control-plane pods for cluster-1 and cluster-2 will each be assigned to separate UDNs, preventing inter-cluster network traffic.

Changes to Networking and Connectivity:

• Control planes in UDN will have segregated network routes, IP addresses, and potentially overlapping IP ranges depending on configuration.
• Configurations like BGP and EVPN will need adjustments to work within isolated UDN environments, with each control plane utilizing network-specific settings for external routing.

Configuration and API Implications:

• Networking API Exposure: cluster service providers should be able to configure UDNs for control planes via HyperShift APIs: spec.networking.userDefinedNetworking allowing per-cluster UDN specifications.
• Network Attachment Definitions (NADs): The HyperShift operator will need to dynamically create NADs corresponding to each control plane’s UDN requirements based on the cluster’s needs.

Example Configurations

Specifying Cluster-Specific UDNs in HyperShift Control Plane Configurations

The control planes for each cluster can specify unique network configurations through HyperShift:

 
apiVersion: hypershift.openshift.io/v1alpha1
kind: HostedCluster
metadata:
  name: cluster-1-control-plane
  namespace: cluster-1
spec:
  networking:
    userDefinedNetworking:
      controlPlaneNetworkName: "cluster-1-udn"
      secondaryNetworkName: "cluster-1-secondary-udn"

apiVersion: hypershift.openshift.io/v1alpha1
kind: HostedCluster
metadata:
  name: cluster-2-control-plane
  namespace: cluster-2
spec:
  networking:
    userDefinedNetworking:
      controlPlaneNetworkName: "cluster-2-udn"
      secondaryNetworkName: "cluster-2-secondary-udn"

Each control plane is associated with a distinct UDN, isolating traffic and enabling advanced routing within each UDN.

Network Configurations for Isolated UDNs

When control-plane UDNs require connectivity with external networks or other clusters, configure BGP and EVPN per UDN:

apiVersion: k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
  name: cluster-1-control-plane-bgp
  namespace: cluster-1
spec:
  config: |-
    {
        "cniVersion": "0.3.1",
        "name": "cluster-1-control-plane-bgp",
        "type": "ovn-k8s-cni-overlay",
        "topology": "layer3",
        "subnets": "10.1.2.0/24",
        "enableBGP": true,
        "bgpRouterID": "10.0.0.1",
        "bgpPeers": ["10.0.0.2", "10.0.0.3"],
        "evpn": true
    }

 

API Exposure and Configuration for Isolation

• Expose Control-Plane Networking API: Allow administrators to set spec.networking.userNetworking within the HyperShift configuration for each control plane, specifying UDNs at the cluster level.
• Automated NAD Management: Code the HyperShift operator to automatically create and manage NADs based on UDN specifications, ensuring each control plane operates within its designated network.
• Namespace Scoped UDNs: Leverage namespace-scoped UDNs to align with the control planes, isolating networking based on namespaces representing each cluster (e.g., cluster-1, cluster-2).

Questions to Answer

• How will the HyperShift operator adapt to dynamically provision and manage NADs for each control plane under UDN?
• What specific UDN configurations are needed to maintain compatibility between UDN and existing HyperShift control plane deployments?