Uploaded image for project: 'OpenShift Request For Enhancement'
  1. OpenShift Request For Enhancement
  2. RFE-8466

improvement: Limit bootstrap pod list when building k8s client

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • Network - Core
    • None
    • None
    • Product / Portfolio Work
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Each multus pod is listing every pod on initial startup. We noticed in our Kubernetes API server audit logs that the first time a Multus pod starts on a node that it makes an request to the API server at /api/v1/pods (list all pods, across all namesapces). 
      
      I tracked this down to https://github.com/openshift/multus-cni/blob/release-4.18/pkg/k8sclient/kubeconfig.go#L99 and as far as I can tell there's no reason to specifically make this call rather than something much less expensive (like a self subject access review).
      
      This is especially problematic in our environments because we may at any given time be running 20k pods and we run all of our compute / worker machines ephemerally meaning we spin them up and down on demand and often in chunks (i.e. we'll launch 20+ new machines at the same time), these nodes all come up and start making the calls to the API server triggering pressure on ETCD and the API server because of the amount of data processed and returned in response.

      Version-Release number of selected component (if applicable):

          

      How reproducible:

          

      Steps to Reproduce:

      This is especially problematic in our environments because we may at any given time be running 20k pods and we run all of our compute / worker machines ephemerally meaning we spin them up and down on demand and often in chunks (i.e. we'll launch 20+ new machines at the same time), these nodes all come up and start making the calls to the API server triggering pressure on ETCD and the API server because of the amount of data processed and returned in response. 
      
      PR Filed : https://github.com/k8snetworkplumbingwg/multus-cni/pull/1434

              mcurry@redhat.com Marc Curry
              rhn-support-nchoudhu Novonil Choudhuri
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                None
                None