Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-34303

[TechPreview Feature]EgressFirewall doesn't work in hypershift hosted cluster

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • 4.16
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Important
    • No
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      EgressFirewall doesn't work in hypershift hosted cluster
          

      Version-Release number of selected component (if applicable):

      4.16.0-0.nightly-2024-05-21-221942 
          

      How reproducible:

      Always
      
          

      Steps to Reproduce:

          1. Created a hypershift cluster and enable techprivew on hosted cluster
      % oc get featuregate/cluster -o yaml
      apiVersion: config.openshift.io/v1
      kind: FeatureGate
      metadata:
        annotations:
          include.release.openshift.io/ibm-cloud-managed: "true"
          kubectl.kubernetes.io/last-applied-configuration: |
            {"apiVersion":"config.openshift.io/v1","kind":"FeatureGate","metadata":{"annotations":{"include.release.openshift.io/ibm-cloud-managed":"true"},"creationTimestamp":null,"name":"cluster"},"spec":{"featureSet":"TechPreviewNoUpgrade"},"status":{"featureGates":[{"disabled":[{"name":"ClusterAPIInstall"},{"name":"ClusterAPIInstallAzure"},{"name":"ClusterAPIInstallIBMCloud"},{"name":"EventedPLEG"},{"name":"GatewayAPI"},{"name":"MachineAPIOperatorDisableMachineHealthCheckController"}],"enabled":[{"name":"AdminNetworkPolicy"},{"name":"AlibabaPlatform"},{"name":"AutomatedEtcdBackup"},{"name":"AzureWorkloadIdentity"},{"name":"BareMetalLoadBalancer"},{"name":"BuildCSIVolumes"},{"name":"CSIDriverSharedResource"},{"name":"ChunkSizeMiB"},{"name":"CloudDualStackNodeIPs"},{"name":"ClusterAPIInstallAWS"},{"name":"ClusterAPIInstallGCP"},{"name":"ClusterAPIInstallNutanix"},{"name":"ClusterAPIInstallOpenStack"},{"name":"ClusterAPIInstallPowerVS"},{"name":"ClusterAPIInstallVSphere"},{"name":"DNSNameResolver"},{"name":"DisableKubeletCloudCredentialProviders"},{"name":"DynamicResourceAllocation"},{"name":"EtcdBackendQuota"},{"name":"Example"},{"name":"ExternalCloudProvider"},{"name":"ExternalCloudProviderAzure"},{"name":"ExternalCloudProviderExternal"},{"name":"ExternalCloudProviderGCP"},{"name":"ExternalOIDC"},{"name":"ExternalRouteCertificate"},{"name":"GCPClusterHostedDNS"},{"name":"GCPLabelsTags"},{"name":"HardwareSpeed"},{"name":"ImagePolicy"},{"name":"InsightsConfig"},{"name":"InsightsConfigAPI"},{"name":"InsightsOnDemandDataGather"},{"name":"InstallAlternateInfrastructureAWS"},{"name":"KMSv1"},{"name":"MachineAPIProviderOpenStack"},{"name":"MachineConfigNodes"},{"name":"ManagedBootImages"},{"name":"MaxUnavailableStatefulSet"},{"name":"MetricsCollectionProfiles"},{"name":"MetricsServer"},{"name":"MixedCPUsAllocation"},{"name":"NetworkDiagnosticsConfig"},{"name":"NetworkLiveMigration"},{"name":"NewOLM"},{"name":"NodeDisruptionPolicy"},{"name":"NodeSwap"},{"name":"OnClusterBuild"},{"name":"OpenShiftPodSecurityAdmission"},{"name":"PinnedImages"},{"name":"PlatformOperators"},{"name":"PrivateHostedZoneAWS"},{"name":"RouteExternalCertificate"},{"name":"ServiceAccountTokenNodeBinding"},{"name":"ServiceAccountTokenNodeBindingValidation"},{"name":"ServiceAccountTokenPodNodeInfo"},{"name":"SignatureStores"},{"name":"SigstoreImageVerification"},{"name":"TranslateStreamCloseWebsocketRequests"},{"name":"UpgradeStatus"},{"name":"VSphereControlPlaneMachineSet"},{"name":"VSphereDriverConfiguration"},{"name":"VSphereMultiVCenters"},{"name":"VSphereStaticIPs"},{"name":"ValidatingAdmissionPolicy"},{"name":"VolumeGroupSnapshot"}],"version":"4.16.0-0.nightly-2024-05-21-221942"}]}}
        creationTimestamp: "2024-05-23T03:16:51Z"
        generation: 2
        name: cluster
        resourceVersion: "9143"
        uid: 5f84bde8-261e-4d5d-8c86-be67e7c656de
      spec:
        featureSet: TechPreviewNoUpgrade
      status:
        featureGates:
        - disabled:
          - name: ClusterAPIInstall
          - name: ClusterAPIInstallAzure
          - name: ClusterAPIInstallIBMCloud
          - name: EventedPLEG
          - name: GatewayAPI
          - name: MachineAPIOperatorDisableMachineHealthCheckController
          enabled:
          - name: AdminNetworkPolicy
          - name: AlibabaPlatform
          - name: AutomatedEtcdBackup
          - name: AzureWorkloadIdentity
          - name: BareMetalLoadBalancer
          - name: BuildCSIVolumes
          - name: CSIDriverSharedResource
          - name: ChunkSizeMiB
          - name: CloudDualStackNodeIPs
          - name: ClusterAPIInstallAWS
          - name: ClusterAPIInstallGCP
          - name: ClusterAPIInstallNutanix
          - name: ClusterAPIInstallOpenStack
          - name: ClusterAPIInstallPowerVS
          - name: ClusterAPIInstallVSphere
          - name: DNSNameResolver
          - name: DisableKubeletCloudCredentialProviders
          - name: DynamicResourceAllocation
          - name: EtcdBackendQuota
          - name: Example
          - name: ExternalCloudProvider
          - name: ExternalCloudProviderAzure
          - name: ExternalCloudProviderExternal
          - name: ExternalCloudProviderGCP
          - name: ExternalOIDC
          - name: ExternalRouteCertificate
          - name: GCPClusterHostedDNS
          - name: GCPLabelsTags
          - name: HardwareSpeed
          - name: ImagePolicy
          - name: InsightsConfig
          - name: InsightsConfigAPI
          - name: InsightsOnDemandDataGather
          - name: InstallAlternateInfrastructureAWS
          - name: KMSv1
          - name: MachineAPIProviderOpenStack
          - name: MachineConfigNodes
          - name: ManagedBootImages
          - name: MaxUnavailableStatefulSet
          - name: MetricsCollectionProfiles
          - name: MetricsServer
          - name: MixedCPUsAllocation
          - name: NetworkDiagnosticsConfig
          - name: NetworkLiveMigration
          - name: NewOLM
          - name: NodeDisruptionPolicy
          - name: NodeSwap
          - name: OnClusterBuild
          - name: OpenShiftPodSecurityAdmission
          - name: PinnedImages
          - name: PlatformOperators
          - name: PrivateHostedZoneAWS
          - name: RouteExternalCertificate
          - name: ServiceAccountTokenNodeBinding
          - name: ServiceAccountTokenNodeBindingValidation
          - name: ServiceAccountTokenPodNodeInfo
          - name: SignatureStores
          - name: SigstoreImageVerification
          - name: TranslateStreamCloseWebsocketRequests
          - name: UpgradeStatus
          - name: VSphereControlPlaneMachineSet
          - name: VSphereDriverConfiguration
          - name: VSphereMultiVCenters
          - name: VSphereStaticIPs
          - name: ValidatingAdmissionPolicy
          - name: VolumeGroupSnapshot
          version: 4.16.0-0.nightly-2024-05-21-221942
          2. Create a test namespace and test pod
      % oc get pods -n test -o wide
      NAME            READY   STATUS    RESTARTS   AGE    IP            NODE                                         NOMINATED NODE   READINESS GATES
      test-rc-4657j   1/1     Running   0          8m9s   10.132.0.32   ip-10-0-133-120.us-east-2.compute.internal   <none>           <none>
      test-rc-nftlf   1/1     Running   0          8m9s   10.132.0.28   ip-10-0-133-120.us-east-2.compute.internal   <none>           <none>
          3. Created an egressfirwall
      % oc get egressfirewall -n test -o yaml
      apiVersion: v1
      items:
      - apiVersion: k8s.ovn.org/v1
        kind: EgressFirewall
        metadata:
          creationTimestamp: "2024-05-23T03:40:34Z"
          generation: 1
          name: default
          namespace: test
          resourceVersion: "15224"
          uid: 7aab9667-745c-4404-a77d-6aa7e8efdddc
        spec:
          egress:
          - to:
              dnsName: registry-1.docker.io
            type: Allow
          - ports:
            - port: 80
              protocol: TCP
            to:
              dnsName: www.facebook.com
            type: Allow
          - to:
              cidrSelector: 0.0.0.0/0
            type: Deny
        status:
          messages:
          - 'ip-10-0-133-120.us-east-2.compute.internal: EgressFirewall Rules applied'
          status: EgressFirewall Rules applied
      kind: List
      metadata:
        resourceVersion: ""
      
      4. From test pod to access the allowed rules
      
          

      Actual results:

      The destination cannot be accessed.
      % oc rsh -n test test-rc-4657j         
      ~ $ curl registry-1.docker.io
      
      ^C
      ~ $ curl registry-1.docker.io --connect-timeout 5
      curl: (28) Failed to connect to registry-1.docker.io port 80 after 4426 ms: Operation timed out
      ~ $ curl www.facebook.com --connect-timeout 5
      curl: (28) Failed to connect to www.facebook.com port 80 after 2706 ms: Operation timed out
      
      There is no dnsnameresolver.
      % oc get dnsnameresolver   -n openshift-ovn-kubernetes             
      No resources found in openshift-ovn-kubernetes namespace.
      
      After deleting the egressfiewall, the destination can be accessed.
      % oc rsh -n test test-rc-4657j          
      ~ $ curl  www.facebook.com 
      ~ $ curl  www.facebook.com  -I
      HTTP/1.1 301 Moved Permanently
      Location: https://www.facebook.com/
      Content-Type: text/plain
      Server: proxygen-bolt
      Date: Thu, 23 May 2024 03:39:54 GMT
      Connection: keep-alive
      Content-Length: 0
      
      
          

      Expected results:

      The allowed rules should take effect.
          

      Additional info:

      
          

              rh-ee-arsen Arkadeep Sen (Aurko)
              huirwang Huiran Wang
              None
              None
              Huiran Wang Huiran Wang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: