Feature overview

Allow customer to define distribution of costs of nodes and clusters into projects to fully reflect the costs as expected by the customer and provide coherence between views.

Goals

• Allow the customer to get a clear understanding of how costs are impacting their applications by providing a way to distribute costs of cluster and nodes into pods and projects.
• Allow the customer to see all costs in a way that makes sense, independently of whether they are looking at cluster costs, project costs, node costs or tag costs.
• Allow the customer to define if they want to distribute costs on CPU (Current) or Memory, per cluster.
• Allow the customer to define if idle resources are taking into account (i.e. the report show idle resources with a price associated) or not (the report distributes costs into existing workloads)
• Allow the customer to define if Kubernetes internal projects are reported or not (i.e. define which parts of the standard installation is reported as resources or not)

Background

There are some costs that are associated to a level in the hierarchy and thus are not easily shown. For instance, you can associate a cost for a cluster, or to a node. As a customer, I expect to see all those costs integrated into my costs to provide the same number whether I am looking at the project, the tag of the cluster.

Going up is easy:

1. Project are part of a single cluster -> project costs can be shown at the cluster level adding them to other costs
2. Nodes are part of a single cluster -> node costs can be shown at the cluster level adding them to other costs
3. Pods run on nodes, and are part of a single project -> Pod costs can be associated to the project, to the node, and thus to the cluster.

The tricky part is how to distribute cluster and node costs to be shown at the project, label and node level.

1. Cluster costs can be shown at the node level distributing evenly the total CPU or memory of the cluster and then dividing it for the node capability -> 6 nodes with 6,6,6,12,12,12 CPU means that a cost associated to the cluster can be distributed into 6/54 splits (or 12/54 depending on the type of node). Distribution could also be done based on CPU capacity per node. As OpenShift has two types of nodes, one of them without cost, this could lead to free nodes having costs distributed.
2. Node costs are independent of applications running on them. They could show node only costs (i.e only those costs that are directly associated to them and those related to cluster, but not pod costs). A node would then show costs per node and distributed costs per cluster. Either we show this very clear in the interface or we allow the customer to switch the pod costs into nodes.
3. Costs of running a project should take into account all the costs of running it, including node costs and cluster costs, to take into account any  costs defined by the customer. For doing that, we need to be able define what is being distributed and how. Costs should be distributed into nodes, and nodes distribute costs into pods, that in time will distribute them to projects.
4. Costs can be shared depending on a label. For instance, the cost of projects with label "backup=yes" would share the costs associated to the cluster / node, with label "support=backup". It should be possible to define both the label used to define the cost, and those elements that gets the costs being distributed. If this is defined, other projects/pods/clusters without the label, won't be considered when calculating the distribution
    

Cluster: Monthly costs should be distributed into pods costs by CPU or memory. Costs per pod = CPU used (with floor on requested) by pod  / capacity of cluster. As request is reserved, when usage is lower than request, the CPU or memory is actually not available for other workloads, and thus should be considered used.

Node: Each pod running in the cluster will get a % of the capacity cost based on CPU (same as cluster) or memory