Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-8000

openshift-ingress-operator is failing to update router-certs because "Too long: must have at most 1048576 bytes" message

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • 4.10
    • Networking / router
    • None
    • Critical
    • 2
    • Sprint 232, Sprint 233, Sprint 234, Sprint 235
    • 4
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      Previously, the Ingress Operator published certificates and keys of all Ingress Controllers to the `router-certs` secret in the `openshift-config-managed` namespace. The Authentication Operator is the only operator that uses the `router-certs` secret to access the cluster Ingress domain. Publishing unnecessary certificates and keys can impact the generation of the secret, because the secret exceeds the maximum resource size limit.

      For the {product-title} {product-version} release, the Ingress Operator publishes certificates and keys for any Ingress Controllers that relate to the cluster Ingress domain. This change ensures that the Ingress Operator can generate a `router-certs` secret that the Authentication Operator can access for OAuth authentication purposes.

      (link:https://issues.redhat.com/browse/OCPBUGS-853[*OCPBUGS-853*]
      Show
      Previously, the Ingress Operator published certificates and keys of all Ingress Controllers to the `router-certs` secret in the `openshift-config-managed` namespace. The Authentication Operator is the only operator that uses the `router-certs` secret to access the cluster Ingress domain. Publishing unnecessary certificates and keys can impact the generation of the secret, because the secret exceeds the maximum resource size limit. For the {product-title} {product-version} release, the Ingress Operator publishes certificates and keys for any Ingress Controllers that relate to the cluster Ingress domain. This change ensures that the Ingress Operator can generate a `router-certs` secret that the Authentication Operator can access for OAuth authentication purposes. (link: https://issues.redhat.com/browse/OCPBUGS-853 [* OCPBUGS-853 *]
    • Bug Fix
    • Done

      This is a clone of OCPBUGS-853.

      Description of problem:

      Large OpenShift Container Platform 4.10.24 - Cluster is failing to update router-certs secret in openshift-config-managed namespace as the given secret is too big.
      
      2022-09-01T06:24:15.157333294Z 2022-09-01T06:24:15.157Z ERROR operator.init.controller.certificate_publisher_controller controller/controller.go:266  Reconciler error  {"name": "foo-bar", "namespace": "openshift-ingress-operator", "error": "failed to ensure global secret: failed to update published router certificates secret: Secret \"router-certs\" is invalid: data: Too long: must have at most 1048576 bytes"}
      
      The OpenShift Container Platform 4 - Cluster has 180 IngressController configured with endpointPublishingStrategy set to private.
      
      Now the default certificate needs to be replaced but is not properly replicated to openshift-authentication namespace and potentially other location because of the problem mentioned (since the required secret can not be updated)

      Version-Release number of selected component (if applicable):

      OpenShift Container Platform 4.10.24

      How reproducible:

      Always

      Steps to Reproduce:

      1. Install OpenShift Container Platform 4.10
      2. Create 180 IngressController with specific certificates
      3. Check openshift-ingress-operator logs to see how it fails to update/create the necessary secret in openshift-config-managed
      

      Actual results:

      2022-09-01T06:24:15.157333294Z 2022-09-01T06:24:15.157Z ERROR operator.init.controller.certificate_publisher_controller controller/controller.go:266  Reconciler error  {"name": "foo-bar", "namespace": "openshift-ingress-operator", "error": "failed to ensure global secret: failed to update published router certificates secret: Secret \"router-certs\" is invalid: data: Too long: must have at most 1048576 bytes"}

      Expected results:

      No matter how many IngressController is created, secret management taken care by Operators need to work, even if data exceed 1 MB size limitation. In that case an approach needs to exist to split data into multiple secrets or handle it otherwise.

      Additional info:

       

            mmasters1@redhat.com Miciah Masters
            rhn-support-sreber Simon Reber
            Shudi Li Shudi Li
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: