Iptables-alerter DaemonSet Lacks Tolerations for Custom Taints

The iptables-alerter DaemonSet fails to schedule on nodes that have custom taints (e.g., for infra nodes). This is because its pod template only tolerates the standard node-role.kubernetes.io/master taint, preventing it from running on any other tainted nodes.

$ oc get ds/iptables-alerter -n openshift-network-operator -o yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
...
  name: iptables-alerter
  namespace: openshift-network-operator
  ownerReferences:
  - apiVersion: operator.openshift.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: Network
    name: cluster
...
spec:
...
  template:
  ...
    spec:
     ...
      tolerations:
      - effect: NoSchedule
        key: node-role.kubernetes.io/master
        operator: Exists  
...

1. Function: The iptables-alerter DaemonSet serves as a critical compliance tool. Its purpose is to run on every node and generate an event if any workload creates iptables rules, ensuring that the cluster is migrating away from this deprecated technology as required for future RHEL compatibility.

2. The Flaw: We have identified a scheduling flaw that compromises its function. The DaemonSet's pod template only includes a toleration for the node-role.kubernetes.io/master taint. It does not tolerate any custom taints, such as those designating 'infra' nodes.

3. Impact: This scheduling limitation means the iptables-alerter pod cannot run on these tainted nodes. Consequently, these nodes are completely unmonitored for iptables rule creation. This defeats the purpose of the alerter and leaves a critical gap in our ability to enforce cluster-wide compliance and prepare for the iptables removal.

4. Required Action: To ensure full monitoring coverage and enforce the intended policy, the iptables-alerter DaemonSet must be updated to include tolerations for all relevant custom taints, allowing it to be deployed on every node in the cluster.

Proposed Solution:

Proposal 1: (Preferred) Introduce a Custom Tolerations API in the Network CR

The most robust and flexible solution is to enhance the Network CR specification with a new field, for example, iptablesAlerter.tolerations.

• Function: The network operator would then reconcile these custom tolerations into the iptables-alerter DaemonSet's pod template, in addition to any defaults.
• Benefit: This provides a generic, future-proof mechanism for administrators to allow the iptables-alerter to run on any node with any custom taint strategy.

Proposal 2: (Fallback) Add a Default Toleration for Infra Nodes

If implementing a new API field is not feasible in the short term, a more direct solution would be to modify the network operator to automatically add a default toleration for the common infra node role.

• Function: The operator would hardcode a toleration for taints like node-role.kubernetes.io/infra.
• Benefit: This would immediately solve the issue for the most common use case without requiring API changes.