Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-1477

Standup GCP Liveness Monitoring Endpoint

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Major Major
    • None
    • None
    • None
    • False
    • None
    • False

      As part of our investigation into GCP disruption we want an endpoint separate from the cluster under test but inside GCP to monitor for connectivity.

      One approach is to use a GCP Cloud Function with a HTTP Trigger

      Another alternative it to standup our own server and collect logging

       

      We need to consider cost of implementation, cost of maintaining and how well the implementation lines up with our overall test scenario (we are wanting to use this as a control to compare with reaching a pod within a cluster under test)

       

      We may want to also consider standing up similar endpoints in AWS and Azure in the future.

       

      A separate story will cover monitoring the endpoint from within Origin

      • We want to capture the audit id and log when we receive an incoming request
      • audit id could include the build id or another field could be used to correlate back to the job instance

            [TRT-1477] Standup GCP Liveness Monitoring Endpoint

            Dennis Periquet added a comment -

            the resources are still there and working

            Dennis Periquet added a comment - the resources are still there and working

            Dennis Periquet added a comment -

            https://issues.redhat.com/browse/DPP-14288 has been closed.

            But we also added the VM/LB in the openshift-ci-data-analysis account and it's been there for a day so we are probably clear to add back the gcp liveness test.

            Dennis Periquet added a comment - https://issues.redhat.com/browse/DPP-14288 has been closed. But we also added the VM/LB in the openshift-ci-data-analysis account and it's been there for a day so we are probably clear to add back the gcp liveness test.

            Dennis Periquet added a comment -

            This LB was deleted Feb 10 or 11 again and had to create and update the LB again in https://github.com/openshift/origin/pull/28583.

            Ultimately, we are currently reverting the test at https://github.com/openshift/origin/pull/28586 because we are not sure when we can keep the LB intact due to no action in https://issues.redhat.com/browse/DPP-14288

            Dennis Periquet added a comment - This LB was deleted Feb 10 or 11 again and had to create and update the LB again in https://github.com/openshift/origin/pull/28583 . Ultimately, we are currently reverting the test at https://github.com/openshift/origin/pull/28586 because we are not sure when we can keep the LB intact due to no action in https://issues.redhat.com/browse/DPP-14288

            Dennis Periquet added a comment -

            The LB got deleted including some firewall rules.

            I re-created the LB and filed https://issues.redhat.com/browse/DPP-14288 to get the resources on the list to be preserved.

            Dennis Periquet added a comment - The LB got deleted including some firewall rules. I re-created the LB and filed https://issues.redhat.com/browse/DPP-14288 to get the resources on the list to be preserved.

            Dennis Periquet added a comment - - edited

            These resources are up in the "openshift-gce-devel" project.

            trt-openshift-tests-endpoint-lb (load balancer)

            openshift-tests-endpointinstance-1 (VM)

            trt-openshift-tests-endpoint (VPC)

            and documented in the TRT Team Tasks and Notes

            Dennis Periquet added a comment - - edited These resources are up in the "openshift-gce-devel" project. trt-openshift-tests-endpoint-lb (load balancer) openshift-tests-endpointinstance-1 (VM) trt-openshift-tests-endpoint (VPC) and documented in the TRT Team Tasks and Notes

            Dennis Periquet added a comment -

            see cloud function

            Next step is to use this repo and make a VM where we can deploy a container.
            The Dockerfile is done and pushed to docker.io/dperique/openshift_tests_endpoint:0.1

            Dennis Periquet added a comment - see cloud function Next step is to use this repo and make a VM where we can deploy a container. The Dockerfile is done and pushed to docker.io/dperique/openshift_tests_endpoint:0.1

              dperique@redhat.com Dennis Periquet
              rh-ee-fbabcock Forrest Babcock
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: