Opening this BZ to track this issue. Following rule was created and xdp program was attached to the interface

ens192: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 xdpgeneric qdisc mq master ovs-system state UP mode DEFAULT group default qlen 1000
    link/ether 00:50:56:ac:ca:fb brd ff:ff:ff:ff:ff:ff
    prog/xdp id 2 tag ef65c8e7d746da72 jited 

Rule
--------

ingress:
    - rules:
      - action: Deny
        order: 1
        protocolConfig:
          protocol: TCP
          tcp:
            ports: 30321-33000
      sourceCIDRs:
      - 10.0.5.26/12
    interfaces:
    - ens192
    nodeSelector:
      matchLabels:
        node-role.kubernetes.io/worker: ""
  status:
    syncStatus: Synchronized
kind: List

4.12

rarely

1. A rule was created to Deny traffic at a particular TCP port
2. daemons ds and controller manager deployment was deleted. Hence they re-spawned sucessfully
3. the same rule at step 1 was applied but with Allow rule
4. step 2 repeated
5. Deny rule was reapplied but its allowing the traffic at that port 

rule becomes non functional post daemons ds and controller manager deletion and they re spawned successfully

Post manager and daemons redeployment, rule should become functional again