During the last optimization of the UI code, it has been observed that it loads the full tree every time it needs to refresh its cache and that the tree contains data that is not used by the UI. Three improvement directions were identified.

Lean dataset
When the UI requests objects, the inventory returns a big payload with all the object's attributes, as well as the related objects and their attributes. Basically, when calling the tree endpoint, the inventory returns almost the full database. The leads to longer request time and increased CPU and memory consumption.

We should explore the option of expanding the Inventory API semantic to allow more precise requests. Ideally, the UI should specify the attributes it needs, as well as the relationships to follow. The following example could be the payload sent by the UI to retrieve a VM:

{
  "attributes": [
    "name",
    "cpu",
    "memory",
    "host.name",
    "host.cluster.name",
    "host.cluster.datacenter.name"
  ]
}

The backend would understand that it needs to follow the host, then cluster, then datacenter relationships and then only return the names.

Note: This is just an example, and paths or syntax unlikely match the data structure.

Compression
In order to reduce the size of the payload over the wire, it could be interesting to compress the data. However, compression comes with a processing cost that may outweigh the benefit of saving network frames. We could enable it for specific requests depending on the calculation we make.

The first iteration would be to benchmark the compression against a few requests, to start understand whether compression is a valid option.

Event-driven UI refresh
The Inventory already updates its database only when catching events from the providers. In turn, it sends some events when its database is updated, so that the other controllers can watch these events to reconcile the custom resources. There is no reason why the UI couldn't watch the events and refresh its in-memory dataset when the objects have changed. This would reduce the amount of data transferred.

It is worth considering a request pooling mechanism in the UI to avoid sending too many requests to the Inventory. This may require a change in the Inventory API, so that it accepts to return more than one object.

The UI could also implement eager and lazy loading depending whether the objects are currently used in the current view. If a Host object changes and the current view is the network mapping wizard, refreshing the Host is not urgent.

Search and Pagination
In order to reduce the amount of objects returned by the API, it would be useful to implement a search mechanism in the Inventory API. The UI could then offload the filtering done in some wizards, e.g. when the user has selected folders in the Plan creation wizard, the UI should only load the VMs that belongs to the selected folders. The ability to search on other attributes in the VM selection step could then use the in-memory dataset.

And with pagination, the UI could also reduce the amount of data transferred for a single request. When one page is displayed, the UI could eagerly load the previous and next pages to make the user experience fluid.

Note: Setting the "Search and Pagination" option aside, as it may open a whole new world of issues, instead of helping some our problem.