Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-9037

Configuring mTLS on default Ingress breaks ingress canary check & console health checks

XMLWordPrintable

    • Important
    • None
    • Sprint 238, Sprint 239, Sprint 240, Sprint 241, Sprint 242, Sprint 243, Sprint 244, Sprint 245, Sprint 246, Sprint 247, Sprint 248, Sprint 249, Sprint 250, Sprint 251, Sprint 252, Sprint 253, Sprint 254, NE Sprint 255, NE Sprint 256
    • 19
    • Rejected
    • Unspecified
    • Hide
      When the default IngressController is configured to require mTLS (Client TLS), the canary would report failing health checks regardless of if the router was actually healthy or not. With this change, the canary pod handles its own TLS connection via a passthrough route, allowing it to determine the health of the router without needing specific mTLS configuration.
      Show
      When the default IngressController is configured to require mTLS (Client TLS), the canary would report failing health checks regardless of if the router was actually healthy or not. With this change, the canary pod handles its own TLS connection via a passthrough route, allowing it to determine the health of the router without needing specific mTLS configuration.
    • Bug Fix
    • Proposed

      Description of problem:
      Configuring mTLS on default IngressController breaks ingress canary check & console health checks which in turn makes the ingress and console cluster operators into a degraded state.

      OpenShift release version:
      OCP-4.9.5

      Cluster Platform:
      UPI on Baremetal (Disconnected cluster)

      How reproducible:
      Configure mutual TLS/mTLS using default IngressController as described in the doc(https://docs.openshift.com/container-platform/4.9/networking/ingress-operator.html#nw-mutual-tls-auth_configuring-ingress)

      Steps to Reproduce (in detail):
      1. Create a config map that is in the openshift-config namespace.
      2. Edit the IngressController resource in the openshift-ingress-operator project
      3.Add the spec.clientTLS field and subfields to configure mutual TLS:
      ~~~
      apiVersion: operator.openshift.io/v1
      kind: IngressController
      metadata:
      name: default
      namespace: openshift-ingress-operator
      spec:
      clientTLS:
      clientCertificatePolicy: Required
      clientCA:
      name: router-ca-certs-default
      allowedSubjectPatterns:

      • "^/CN=example.com/ST=NC/C=US/O=Security/OU=OpenShift$"
        ~~~
        Actual results:
        setting up mTLS using documented steps breaks canary and console health checks as clientCertificatePolicy is set as Required these health checks are looking for the client Certs and hence failing and in turn Ingress and Console operators are in a degraded state.

      Expected results:
      mTLS setup should work properly without degrading the Ingress and Console operators.

      Impact of the problem:
      Instable cluster with Ingress and Console operators into Degraded state.

      Additional info:
      The following is the Error message for your reference:
      The "default" ingress controller reports Degraded=True: DegradedConditions: One or more other status conditions indicate a degraded state: CanaryChecksSucceeding=False (CanaryChecksRepetitiveFailures: Canary route checks for the default ingress controller are failing)

      // Canary checks looking for required tls certificate.
      2021-11-19T17:17:58.237Z ERROR operator.canary_controller wait/wait.go:155 error performing canary route check

      {"error": "error sending canary HTTP request to \"canary-openshift-ingress-canary.apps.bruce.openshift.local\": Get \"https://canary-openshift-ingress-canary.apps.bruce.openshift.local\": remote error: tls: certificate required"}

      // Console operator:
      RouteHealthDegraded: failed to GET route (https://console-openshift-console.apps.bruce.openshift.local): Get "https://console-openshift-console.apps.bruce.openshift.local": remote error: tls: certificate required

        • Please do not disregard the report template; filling the template out as much as possible will allow us to help you. Please consider attaching a must-gather archive (via `oc adm must-gather`). Please review must-gather contents for sensitive information before attaching any must-gathers to a bugzilla report. You may also mark the bug private if you wish.

            rfredett@redhat.com Ryan Fredette
            rhn-support-nkashyap Nirupma Nirupma
            Hongan Li Hongan Li
            Red Hat Employee
            Votes:
            2 Vote for this issue
            Watchers:
            16 Start watching this issue

              Created:
              Updated: