Uploaded image for project: 'Service Binding'
  1. Service Binding
  2. APPSVC-1292

Define RegisteredServices Healthchecks

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • Primaza 0.1
    • None
    • Service Binding
    • None
    • 0
    • False
    • Hide

      None

      Show
      None
    • False
    • Hide

      Feature: RegisteredServices Healthchecks

      Scenario: Healthcheck completing successfully

      Given Primaza Cluster "primaza-main" is running
      And On Primaza Cluster "primaza-main", Resource is created
      """
      apiVersion: primaza.io/v1alpha1
      kind: ServiceClass
      metadata:
      name: test-service-class
      namespace: primaza-system
      spec:
      serviceClassIdentity:

      • type: db_type
        value: test
        healthcheck:
        container:
        image: bash:latest
        cmd:
      • echo
      • "healthcheck ok"
        resource:
        kind: testdb
        apiVersion: testservices.redhat.io/v1alpha1
        serviceEndpointDefinitionMapping:
      • name: admin
        jsonPath: .spec.admin
        """
        When On Primaza Cluster "primaza-main", Resource is created
        """
        apiVersion: primaza.io/v1alpha1
        kind: RegisteredService
        metadata:
        name: test-registered-service
        namespace: primaza-system
        spec:
        serviceClassIdentity:
      • type: db_type
        value: test
        serviceEndpointDefinition:
      • admin: sa
        """
        Then On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" is eventually created
        And On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" completes correctly
        And On Primaza Cluster "primaza-main", RegisteredService "test-registered-service"'s status is eventually updated to "reachable"

      Scenario: Healthcheck failing

      Given Primaza Cluster "primaza-main" is running
      And On Primaza Cluster "primaza-main", Resource is created
      """
      apiVersion: primaza.io/v1alpha1
      kind: ServiceClass
      metadata:
      name: test-service-class
      namespace: primaza-system
      spec:
      serviceClassIdentity:

      • type: db_type
        value: test
        healthcheck:
        container:
        image: bash:latest
        cmd:
      • echo
      • "healthcheck failing"
      • &&
      • exit
      • 1
        resource:
        kind: testdb
        apiVersion: testservices.redhat.io/v1alpha1
        serviceEndpointDefinitionMapping:
      • name: admin
        jsonPath: .spec.admin
        """
        When On Primaza Cluster "primaza-main", Resource is created
        """
        apiVersion: primaza.io/v1alpha1
        kind: RegisteredService
        metadata:
        name: test-registered-service
        namespace: primaza-system
        spec:
        serviceClassIdentity:
      • type: db_type
        value: test
        serviceEndpointDefinition:
      • admin: sa
        """
        Then On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" is eventually created
        And On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" fails
        And On Primaza Cluster "primaza-main", RegisteredService "test-registered-service"'s status is eventually updated to "unreachable"

      Scenario: Healthcheck not runnable

      Given Primaza Cluster "primaza-main" is running
      And On Primaza Cluster "primaza-main", Resource is created
      """
      apiVersion: primaza.io/v1alpha1
      kind: ServiceClass
      metadata:
      name: test-service-class
      namespace: primaza-system
      spec:
      serviceClassIdentity:

      • type: db_type
        value: test
        healthcheck:
        container:
        image: quay.io/non-existing-repo/non-existing-image:non-existing-tag
        cmd:
      • exit
      • 1
        resource:
        kind: testdb
        apiVersion: testservices.redhat.io/v1alpha1
        serviceEndpointDefinitionMapping:
      • name: admin
        jsonPath: .spec.admin
        """
        When On Primaza Cluster "primaza-main", Resource is created
        """
        apiVersion: primaza.io/v1alpha1
        kind: RegisteredService
        metadata:
        name: test-registered-service
        namespace: primaza-system
        spec:
        serviceClassIdentity:
      • type: db_type
        value: test
        serviceEndpointDefinition:
      • admin: sa
        """
        Then On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" is not created with error "ErrImagePull"
        And On Primaza Cluster "primaza-main", RegisteredService "test-registered-service"'s status is eventually updated to "undetermined"
      Show
      Feature: RegisteredServices Healthchecks Scenario: Healthcheck completing successfully Given Primaza Cluster "primaza-main" is running And On Primaza Cluster "primaza-main", Resource is created """ apiVersion: primaza.io/v1alpha1 kind: ServiceClass metadata: name: test-service-class namespace: primaza-system spec: serviceClassIdentity: type: db_type value: test healthcheck: container: image: bash:latest cmd: echo "healthcheck ok" resource: kind: testdb apiVersion: testservices.redhat.io/v1alpha1 serviceEndpointDefinitionMapping: name: admin jsonPath: .spec.admin """ When On Primaza Cluster "primaza-main", Resource is created """ apiVersion: primaza.io/v1alpha1 kind: RegisteredService metadata: name: test-registered-service namespace: primaza-system spec: serviceClassIdentity: type: db_type value: test serviceEndpointDefinition: admin: sa """ Then On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" is eventually created And On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" completes correctly And On Primaza Cluster "primaza-main", RegisteredService "test-registered-service"'s status is eventually updated to "reachable" Scenario: Healthcheck failing Given Primaza Cluster "primaza-main" is running And On Primaza Cluster "primaza-main", Resource is created """ apiVersion: primaza.io/v1alpha1 kind: ServiceClass metadata: name: test-service-class namespace: primaza-system spec: serviceClassIdentity: type: db_type value: test healthcheck: container: image: bash:latest cmd: echo "healthcheck failing" && exit 1 resource: kind: testdb apiVersion: testservices.redhat.io/v1alpha1 serviceEndpointDefinitionMapping: name: admin jsonPath: .spec.admin """ When On Primaza Cluster "primaza-main", Resource is created """ apiVersion: primaza.io/v1alpha1 kind: RegisteredService metadata: name: test-registered-service namespace: primaza-system spec: serviceClassIdentity: type: db_type value: test serviceEndpointDefinition: admin: sa """ Then On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" is eventually created And On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" fails And On Primaza Cluster "primaza-main", RegisteredService "test-registered-service"'s status is eventually updated to "unreachable" Scenario: Healthcheck not runnable Given Primaza Cluster "primaza-main" is running And On Primaza Cluster "primaza-main", Resource is created """ apiVersion: primaza.io/v1alpha1 kind: ServiceClass metadata: name: test-service-class namespace: primaza-system spec: serviceClassIdentity: type: db_type value: test healthcheck: container: image: quay.io/non-existing-repo/non-existing-image:non-existing-tag cmd: exit 1 resource: kind: testdb apiVersion: testservices.redhat.io/v1alpha1 serviceEndpointDefinitionMapping: name: admin jsonPath: .spec.admin """ When On Primaza Cluster "primaza-main", Resource is created """ apiVersion: primaza.io/v1alpha1 kind: RegisteredService metadata: name: test-registered-service namespace: primaza-system spec: serviceClassIdentity: type: db_type value: test serviceEndpointDefinition: admin: sa """ Then On Primaza Cluster "primaza-main", CronJob "test-registered-service-healthcheck" is not created with error "ErrImagePull" And On Primaza Cluster "primaza-main", RegisteredService "test-registered-service"'s status is eventually updated to "undetermined"

      Owner: Architect:

      Francesco Ilario

      Story (Required)

      As a Service Administrator, I would like to have RegisteredService healthchecks so that I know if the service is available and reachable

      Background (Required)

      As defined in the Primaza architecture document, we need to define optional healthchecks for RegisteredServices.

      The Healthcheck is similar to how Pods do healthchecks. We want to take it beyond trusting the info provided in Service Endpoint Definition (SED) is always accurate. This could change overtime, for example a password rotation. So instead of the error going all the way down to the app not able to connect, we want to be more proactive and catch it earlier. It will be a good debugging tool.
      So, healthchecks will provide SED validation and service status checks.

      See epic for arch document link.

      Glossary

      See glossary in architecture document

      Out of scope

      • Related Claims status update
      • Other healthchecks mechanisms than container based one

      In Scope

      • RegisteredService status update
      • Container based healthchecks

      Approach(Required)

      Healthchecks will be optional and executed on Primaza cluster.
      We can rely on CronJob for executing them in a timely fashion.

      Healthchecks are defined in ServiceClass.
      Use the ServiceClassIdentity field to retrieve the ServiceClass specification and, hence, the Healthcheck one.
      Healthcheck containers will have the Service Endpoint Definition secret data projected in their filesystem.

      If Primaza has problems executing the healthchecks, like ErrImagePull, RegisteredService state and conditions should be updated accordingly.

      Healthchecks are successful if their pod terminates without any error. If the pod fails, than the healthcheck is considered failed.

      Demo requirements(Required)

      A great demo will show RegisteredService's healthchecks start failing after an incorrect update of ServiceEndpointDefinition data.

      Dependencies

      NA

      Edge Case

      NA

      BDD Tests

      You can find BDD Test specification for this story in the Story's Field 'Testing Instructions' or in the Github Issue linked to this story.
      Click here for all BDD Tests Issues.

      Acceptance Criteria

      • Development
        Primaza should run healthchecks for all RegisteredServices
        Primaza should stop and recreate running healthcheck pods on related data update, like ServiceClass.Healthcheck or RegisteredService.ServiceEndpointDefinition.
        RegisteredService status and conditions should be updated with respect to healthchecks result
      • QE
        There are test cases for passing healthchecks
        There are test cases for failing healthchecks
        There are test cases for in-error healthchecks, like pod not executable (e.g., ErrImagePull)
      • Docs
        There is a section in RegisteredService's doc dedicated to explaining what an healthcheck is and why it's needed
        Update the architecture document with any changes while implementing

      INVEST Checklist

      Dependencies identified
      Blockers noted and expected delivery timelines set
      Design is implementable
      Acceptance criteria agreed upon
      Story estimated

      Legend

      Unknown
      Verified
      Unsatisfied

              Unassigned Unassigned
              rh-ee-filario Francesco Ilario
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: