Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-60376

AutoNode karpenter stops working after a token rotation

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 3
    • None
    • None
    • None
    • None
    • AUTOSCALE - Sprint 276
    • 1
    • None
    • Release Note Not Required
    • None
    • None
    • None
    • None
    • None

      Related to https://issues.redhat.com//browse/AUTOSCALE-293

      Token rotations no longer drift nodes, UNLESS the EC2NodeClass controller calls Reconcile which only happens if the guest cluster CRDs get modified, since the controller does not watch secrets in the management cluster and cannot react to the token being rotated.

      We actually probably want the ec2nodeclass controller to watch secrets (or at least get triggered when it rotates), because if not and the token gets rotated anyways, then whenever a new Karpenter wants to create a new node, it cannot because the token is outdated when sending the ignition payload to the ignition-server pod. This is currently what happens if 5.5 hours pass and the token for the hypershift nodepool that's tied to karpenter gets rotated --> new nodeclaims will fail to bring up new nodes after this. 

              rh-ee-macao Max Cao
              rh-ee-macao Max Cao
              None
              None
              Paul Rozehnal Paul Rozehnal
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: