Uploaded image for project: 'OpenShift Hive'
  1. OpenShift Hive
  2. HIVE-1862

[Verizon] split control plane investigation and scoping

    XMLWordPrintable

Details

    • Story
    • Resolution: Unresolved
    • Critical
    • None
    • None
    • False
    • None
    • False
    • OCPSTRAT-218 - Hive in split control plane for etcd scaling

    Description

      Copyover from epic:

      Customer wants to create/manage O(100k) spoke clusters from a single OCP cluster. Due to the number of objects each requires, etcd on one ACM+hive can handle O(1k). ACM proposes an architecture where:

      • ACM/MCE/Hive stacks are running in O(100) separate namespaces on a single OCP cluster. We'll call these "control planes".
      • Each control plane talks to a corresponding "data plane", a virtualized kube API server running in the same namespace, importantly with its own independent etcd.
      • A single "hub of hubs" ACM front end aggregates everything for management and observability. (Does this front end run in the big OCP cluster or elsewhere?) (Hive doesn't know/care about this layer.)

      In this model, all the "runnables" (containers in pods from deployments/statefulsets including controllers; containers in pods created from jobs for e.g. provision/deprovision; maybe other things??) need to be created/reconciled in the control planes; but all the other objects – hive CRs, the Secrets hive consumes, etc. – need to be created/reconciled in the data plane corresponding to that control plane.

      This means we need to teach hive to understand two API servers and manage the right objects in the right one.
      From a hive API perspective, this is probably just one field in HiveConfig. Internally, we use different clients for the control plane and data plane objects.
      The hard part is going to be figuring out which code paths need to use which clients, and validating that we didn't miss any

      Attachments

        Activity

          People

            Unassigned Unassigned
            mworthin@redhat.com Mike Worthington
            Mingxia Huang Mingxia Huang
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated: