Health insights have a lifecycle that is not suitable for the async producer/consumer architecture USC has right now, where update informers send individual insights to the controller that maintains the API instance. Health insights are expected to disappear and appear following the trigger condition, and this needs to be respected through controller restart, API unavailability etc. Basically this means that the informer should ideally work as a standard, full-reconciliation controller over a set of its insights.

We should also have an easy method to test health insight lifecycle: easy on/off switch for an artificial health insight to be reported or not, to avoid relying on true problematic conditions for testing the controller operation. Something like an existence of a label or an annotation on a resource to trigger a health insight directly.

Definition of Done

• When a resource watched by USC (so ClusterVersion ATM) has a certain annotation, USC should produce an artificial health insight
• The properties of the health insight do not matter but must clearly indicate the health insight is artificial and intended for testing
• All these scenarios must work:
  • When insight is not in API, USC is running and the annotation is added to CV -> insight is added to API
  • When insight is not in API, USC is running, annotation is not on CV, insight is manually added to API -> insight is removed from API
  • When insight is in the API, USC is running, annotation is removed from CV -> insight is removed from API
  • When insight is in the API, USC is running, insight is manually removed from API -> insight is added from API
  • When insight is not in API, USC is stopped, annotation is added to CV, USC is started -> insight is added to API
  • When insight is in the API, USC is stopped, annotation is removed from CV, USC is started -> insight is removed from API
• In all cases where there is an existing insight in the API and the annotation was never observed removed from the CV, any "refresh" by USC must respect the original properties of the insight (start time, uid, etc)
• Identical sync mechanism should be used for Status Insights