Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-14937

(TP) Enabling test automation of OCM.io policies

XMLWordPrintable

    • False
    • None
    • False
    • Not Selected
    • 100% To Do, 0% In Progress, 0% Done

      Feature Overview

      "dryrun" command in policy CLI is in a dev preview / experimental state, this issue is to evaluate and and progress the functionality to Tech Preview based on user/customer feedback.

      Previous work:  https://issues.redhat.com/browse/ACM-14161 

      Goals

      This Section: Provide high-level goal statement, providing user context
      and expected user outcome(s) for this feature

      Requirements

      This Section: A list of specific needs or objectives that a Feature must
      deliver to satisfy the Feature.. Some requirements will be flagged as MVP.
      If an MVP gets shifted, the feature shifts. If a non MVP requirement slips,
      it does not shift the feature.

      Requirement Notes isMvp?
      CI - MUST be running successfully with test automation This is a
      requirement for ALL features.
      YES
      Release Technical Enablement Provide necessary release enablement details
      and documents.
      YES

      (Optional) Use Cases

      This Section:

      • Today the dryrun experience is designed more around affirming non-violation status of a ConfigurationPolicy (CP) against a set of "input resources".  In the instances where the CP would be violated, a non-zero exit code status is returned.  We need to allow for affirming that a violation status will be returned, as expected, when the "input resources" do not match the desired state of the CP.
        • The non-zero exit code could be checked for, but this creates some more handling needed by the end user to check for non-zero, instead of expecting the negative case to happen and, in fact, expect a zero status return)
      • Allow for returning & annotating the line number of the "input resources" that are offending the CP.
        • Customer is leveraging pipelines-as-code approach that is integrated into their Git(Hub) workflows.  In other tools that they use, annotations are applied to show more quickly indicate the "problematic" lines, and this allows for the problems to be more easily surfaced in the PR.
      • For more granular testing, beyond checking for violation status, the users may need to more specifically assert on other aspects of the dryrun results.  For example, asserting on the compliance message or asserting on the policy diff.
      • Allow for the template-resolver to return the resources that will be affected by a policy from a live cluster and outputted into a file; so that those resources can be used as part of dryrun command evaluation.

      Questions to answer

      • From 24 September, GRC sync call
        • Current limitations:
          • I’m very open to feedback on how to print the “results” users might want
        • Support mapping file from an environment variable
        • Consider showing the Compliance message first, it may scroll off the screen so maybe not?
        • Color the diff, feature not available yet in go library used

      Out of Scope

      Background, and strategic fit

      This Section: What does the person writing code, testing, documenting
      need to know? What context can be provided to frame this feature?

      Assumptions

      • ...

      Customer Considerations

      • ...

      Documentation Considerations

      Questions to be addressed:

      • What educational or reference material (docs) is required to support this
        product feature? For users/admins? Other functions (security officers, etc)?
      • Does this feature have a doc impact?
      • New Content, Updates to existing content, Release Note, or No Doc Impact
      • If unsure and no Technical Writer is available, please contact Content
        Strategy.
      • What concepts do customers need to understand to be successful in
        [action]?
      • How do we expect customers will use the feature? For what purpose(s)?
      • What reference material might a customer want/need to complete [action]?
      • Is there source material that can be used as reference for the Technical
        Writer in writing the content? If yes, please link if available.
      • What is the doc impact (New Content, Updates to existing content, or
        Release Note)?

              jkulikau@redhat.com Justin Kulikauskas
              showeimer Sho Weimer
              Luke Bainbridge Luke Bainbridge
              Matthew Prahl Matthew Prahl
              Derek Ho Derek Ho
              Gus Parvin Gus Parvin
              Sho Weimer Sho Weimer
              Votes:
              2 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: