Summary

Prepare the Network Policy resources that restrict the network communication for OSUS Operator. Because the OLM support for Network Policy manifest presence in the bundle is not ready yet and bundles cannot contain NPs (KONFLUX-9372), this card targets only creation of the policies, their inclusion in the o/cincinnati-operator repository and testing that they have the desired effect and (optionally, see Necessary Decisions) no interference with unrelated workloads when manually applied to the namespace where OSUS Operator is installed. Actually shipping these NPs in the bundle will be a separate effort later when necessary OLM/pipeline support is available.

Necessary Decisions

Until now there was no policy on whether the namespace where OSUS Operator (and by extension, its OSUS operands) is installed should allow co-existence with user workloads. Default deny policies are easier and safer if the operator is able to carry a blanket "deny all networking in a namespace" policy, but if we expect users to co-locate custom workloads in the namespace where OSUS Operator is installed, such broad default-deny policy would interfere with them. In the absence of existing policy, I think we need to assume we have at least one customer who installed custom workloads alongside OSUS Operator, and we would break this customer's operation if we ship a broad default-deny policy and take no measures to prevent breaking them.

We have the following options and need to make a decision:

1. Explicitly do not support co-located workloads with OSUS. Prepare (and eventually ship) a namespace-wide default deny. Add a release note about this and expect customers to move their co-located workloads elsewhere. If they do not, updating OSUS Operator breaks them.
2. Support co-located  workloads provided customer deploy NPs to allow their networking. Prepare (and eventually ship) a namespace-wide default deny. Add a release note about this and expect customers to create permissive NPs for their wokloads. If they do not, updating OSUS Operator breaks them.
3. Support co-located workloads without further action. Prepare (and eventually ship) a deny policy that only covers Pods associated with the OSUS Operator (see Deny for Layered Operators doc)

OSUS Operator Network Policy Requirements

1. Default deny for Layered Products or Default deny for Entire Namespace , depending on decision whether OSUS Operator should allow co-located user workloads (see above)
2. Allow egress to apiserver: pragmatic port filter suggested as sufficient for OCPSTRAT-819
3. Allow egress to DNS: pragmatic port filter suggested as sufficient for OCPSTRAT-819
4. Allow ingress from Prometheus Pods (monitoring stack) to operator metrics endpoint (the Developing Network Policies doc suggests to just allow all ingress to the metrics endpoint: “We allow anything to reach our metrics endpoint.")

Definition of Done

1. Network Policy manifests are committed to the o/cincinnati-operator repository, but are not included in the bundle and therefore they are not installed with the operator by OLM
2. One Network Policy enforces default deny everything for operator pods
3. One Network Policy allows networking specified above in OSUS Operator Network Policy Requirements
4. One Network Policy enforces default deny everything for any workloads created for operands (operator is supposed to create permissive policies for each operand, tracked in OTA-1609)
5. Operator manifests are labeled in a way that allows targeting them with policy from item 2 and 3
6. Operand manifests are labeled in a way that allows targeting them with policy from item 4

Testing

• When the operator is installed in a namespace, apply the policies manually to the same namespace (delivery with the operator is out of scope for this card, see summary)
• Operator pod should not be able to make any egress to either cluster internal or cluster external locations, except apiserver and dns (can probably be tested with exec & curl)
• Assuming OCPBUGS-60905 and OCPBUGS-51196 are fixed, Prometheus should be able to scrape operator metrics endpoint
• Operator must be able to work, that means create and manage OSUS instances as normal
• Other unrelated workloads in the namespace should or should not be affected per the decision on allowing co-located workloads, see above
• Unless OTA-1609 is already implemented, the policies from item 4 of DoD will restrict all networking for created OSUS instances