Uploaded image for project: 'Red Hat 3scale API Management'
  1. Red Hat 3scale API Management
  2. THREESCALE-9453

Replace template-based CVP functional tests

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Unresolved
    • Icon: Major Major
    • None
    • None
    • Productization
    • None
    • False
    • None
    • False
    • Not Started
    • Not Started
    • Not Started
    • Not Started
    • Not Started
    • Not Started

      The functional tests [1] that we run as part of the Container Verification Pipeline (CVP) (which is triggered after every regular Brew build), are not really useful any more: 

      1. They deploy from an openshift template, which is not a supported installation method any more.
      2. They were never designed to work on Z / P, and are often failing. 
      3. They have too broad a scope, as they deploy all of 3scale. Instead, they should have a more limited, per-container-image, scope: rather than deploy all of 3scale, the tests should just check the individual container image, because the QE Group Tests take care of 3scale-wide deployments. For some components, there is already a `make smoke` target (e.g. [2]) that we used to run as part of the old, pre-cpaas 3scale productization pipeline, so perhaps we could re-use those. 

       

      After discussing this issue with mjaros@redhat.com , we concluded that we can sunset these particular tests. We can then replace them with some new tests (perhaps based on the `make smoke` targets, where available) that focus on just running the individual container image on the respective platform the test is running on and check whatever else is possible/feasible to test on the container-image level. Input from the app maintainers will be required on that front. Perhaps some checks from community CI can be used as well. 

       

       [1] - https://docs.engineering.redhat.com/pages/viewpage.action?spaceKey=CVP&title=Container+Verification+Pipeline+E2E+Documentation#container-functional-tests 

      [2] - https://pkgs.devel.redhat.com/cgit/containers/3scale-amp-system/tree/Makefile?h=3scale-2.13-rhel-7#n55 

            Unassigned Unassigned
            rh-ee-gsaslisl Yorgos Saslis
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated: