Owner: Architect:

Francesco Ilario

Story (Required)

As a Primaza Administrator, I would like to have optional healthchecks for Registered Services so that I can automatically validate generated Service Endpoint Definition

Background (Required)

To catch errors on Registered Services' Service Endpoint Definition preemptively, an healthcheck definition may be provided.
The health-check will consist of a container specification.
Primaza have to execute safely this container in a timely fashion, and use its exit code and termination message to update the status of the Registered Service.

See epic for arch document link.

Glossary

See glossary in architecture document

Out of scope

• Update Service Claim if claimed Registered Service becomes unreachable

In Scope

• Run healthchecks
• Remove healthchecks
• Update Registered Service status

Approach(Required)

When a Registered Service is created, if its Healthcheck field is defined, then an healthcheck must be registered to monitor the Service status.

The healthcheck consists of:

• A Cronjob with 1 container:
  • The container provided in the Registered Service healthcheck field: it must be executed as non-root user and with a restrictive security context
  • has the following labels:
    • app: primaza
    • component: registered-services-healthchecks
    • registered-service: <REGISTERED_SERVICE_NAME>
  • has the following fields:
    • jobTemplate.spec.parallelism: 1
    • successfulJobsHistoryLimit: 1
    • failedJobsHistoryLimit: 1
    • schedule: */5 * * * *
    • concurrencyPolicy: Forbid
• A primaza operator's controller that watches CronJobs
  • watches CronJobs with the label above
  • update Registered Service wrt CronJob status and CronJob's Pod status and terminationMessage
    • If CronJob is suspended, the Registered Service should have a condition that reports that
    • If CronJob's latest Pod terminated successfully, status should be available or, if claimed, claimed
    • If CronJob's latest Pod failed, Registered Service status must be unreachable
    • If errors happen fetching CronJob's latest Pod status, Registered Service status must be undefined

Demo requirements(Required)

NA

Dependencies

NA

Edge Case

NA

BDD Tests

You can find BDD Test specification for this story in the "Testing Instruction" Field Tab or in the GitHub Issue linked to this story.
Click here for all BDD Tests Issues.

Acceptance Criteria

• Development
  RegisteredService controller creates/updates/deletes the CronJob
  RegisteredService controller updates RegisteredService status with respect to CronJob

• QE
  There are test cases for healthcheck creation/update/deletion
  There are test cases for Registered Service status transition
  • undetermined -> available
  • undetermined -> unreachable
  • unreachable -> available
  • claimed -> unreachable
  • unreachable -> claimed

• Docs
  There is a section in our Registered Service docs dedicated to explaining how Healthchecks works
  Update architecture document with any changes while implementing

INVEST Checklist

Dependencies identified
Blockers noted and expected delivery timelines set
Design is implementable
Acceptance criteria agreed upon
Story estimated

Legend

Unknown
Verified
Unsatisfied