Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-32937

[IBU] postpivot failure: failed to run once recert for post pivot: failed recert full flow: failed to run recert tool container: 2024-04-24 20:22:49 - INFO - src/cluster_crypto/crypto_utils.rs:246: using openssl: OpenSSL 3.0.7

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • 4.16.0
    • LCA operator
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • No
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      The setup is configured with ipv6 and uses proxy.
      Attempted to upgrade from 4.15.10 to 4.16.0-ec.5

      oc get ibu
      NAME      AGE   DESIRED STAGE   STATE    DETAILS
      upgrade   74m   Upgrade         Failed   Rollback due to postpivot failure: failed to run once recert for post pivot: failed recert full flow: failed to run recert tool container: 2024-04-24 20:22:49 - INFO - src/cluster_crypto/crypto_utils.rs:246: using openssl: OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)...
      
      oc get ibu upgrade -o yaml
      apiVersion: lca.openshift.io/v1alpha1
      kind: ImageBasedUpgrade
      metadata:
        creationTimestamp: "2024-04-24T20:26:26Z"
        generation: 1
        name: upgrade
        resourceVersion: "36800"
        uid: b04df1fe-f800-4422-af43-4dc57fa29796
      spec:
        autoRollbackOnFailure:
          initMonitorTimeoutSeconds: 1801
        extraManifests:
        - name: lca-catalog-source-configmap
          namespace: openshift-marketplace
        oadpContent:
        - name: oadp-cm
          namespace: openshift-adp
        seedImageRef:
          image: quay.io/ocp-edge-qe/ib-seedimage-public:ci-ipv6-proxy
          version: 4.16.0-ec.5
        stage: Upgrade
      status:
        conditions:
        - lastTransitionTime: "2024-04-24T20:09:26Z"
          message: In progress
          observedGeneration: 9
          reason: InProgress
          status: "False"
          type: Idle
        - lastTransitionTime: "2024-04-24T20:21:04Z"
          message: Prep completed
          observedGeneration: 9
          reason: Completed
          status: "False"
          type: PrepInProgress
        - lastTransitionTime: "2024-04-24T20:21:04Z"
          message: Prep completed successfully
          observedGeneration: 9
          reason: Completed
          status: "True"
          type: PrepCompleted
        - lastTransitionTime: "2024-04-24T20:21:29Z"
          message: Upgrade failed
          observedGeneration: 1
          reason: Failed
          status: "False"
          type: UpgradeCompleted
        - lastTransitionTime: "2024-04-24T20:21:29Z"
          message: |-
            Rollback due to postpivot failure: failed to run once recert for post pivot: failed recert full flow: failed to run recert tool container: 2024-04-24 20:22:49 - INFO - src/cluster_crypto/crypto_utils.rs:246: using openssl: OpenSSL 3.0.7 1 Nov 2022 (Library: OpenSSL 3.0.7 1 Nov 2022)
      
            2024-04-24 20:22:58 - WARN - src/cluster_crypto/crypto_objects.rs:81: ignoring error from processing pem-looking text at location k8s:ConfigMap/kube-system:cluster-config-v1:/data/install-config, without encoding, unknown: processing pem bundle
            2024-04-24 20:23:24 - WARN - src/cluster_crypto/crypto_objects.rs:81: ignoring error from processing pem-looking text at location k8s:ConfigMap/openshift-etcd:cluster-config-v1:/data/install-config, without encoding, unknown: processing pem bundle
            2024-04-24 20:24:25 - INFO - src/cluster_crypto/cert_key_pair.rs:173: Using custom private key for CN kube-apiserver-lb-signer
            2024-04-24 20:24:26 - INFO - src/cluster_crypto/cert_key_pair.rs:173: Using custom private key for CN kube-apiserver-localhost-signer
            2024-04-24 20:24:27 - INFO - src/cluster_crypto/cert_key_pair.rs:173: Using custom private key for CN kube-apiserver-service-network-signer
            Error: finalizing
      
            Caused by:
                0: performing ocp specific post-processing
                1: renaming hostname
                2: renaming all
                3: renaming etcd resources
                4: fixing etcd-all-certs
                5: could not replace managed fields keys
                6: Failed to replace keys
                7: key not found
                8: could not remove key: f:etcd-peer-seed-0-0.crt
            : exit status 1
          observedGeneration: 1
          reason: Failed
          status: "False"
          type: UpgradeInProgress
        observedGeneration: 1
        validNextStages:
        - Idle
      
      

              jche@redhat.com Jun Chen
              achuzhoy@redhat.com Alexander Chuzhoy
              None
              None
              Yang Liu Yang Liu
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: