From both the compute16 and compute17 ovndb, I see same behavior where logical flow(table=13(lr_in_ip_routing   ), priority=69) on ovn_cluster_router is missing 
compute17:


$  ovn-sbctl lflow-list ovn_cluster_router|grep -i 100.69.0.0/23


  table=1 (lr_in_lookup_neighbor), priority=100  , match=(inport == "rtos-compute-17.ncw-az1-001.caas.bbtnet.com" && arp.spa == 100.69.0.0/23 && arp.op == 1 && is_chassis_resident("cr-rtos-compute-17.ncw-az1-001.caas.bbtnet.com")), action=(reg9[2] = lookup_arp(inport, arp.spa, arp.sha); next;)


  table=3 (lr_in_ip_input     ), priority=90   , match=(inport == "rtos-compute-17.ncw-az1-001.caas.bbtnet.com" && arp.op == 1 && arp.tpa == 100.69.0.1 && arp.spa == 100.69.0.0/23), action=(eth.dst = eth.src; eth.src = xreg0[0..47]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[0..47]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)


  table=13(lr_in_ip_routing   ), priority=71   , match=(ip4.dst == 100.69.0.0/23), action=(ip.ttl--; reg8[0..15] = 0; reg0 = ip4.dst; reg1 = 100.69.0.1; eth.src = 0a:58:64:45:00:01; outport = "rtos-compute-17.ncw-az1-001.caas.bbtnet.com"; flags.loopback = 1; next;)
[root@acc7d829efe2 ~]# 


compute16:


root@83690ee03607 ~]# ovn-sbctl lflow-list ovn_cluster_router|grep -i 100.69.6.0/23


  table=1 (lr_in_lookup_neighbor), priority=100  , match=(inport == "rtos-compute-16.ncw-az1-001.caas.bbtnet.com" && arp.spa == 100.69.6.0/23 && arp.op == 1 && is_chassis_resident("cr-rtos-compute-16.ncw-az1-001.caas.bbtnet.com")), action=(reg9[2] = lookup_arp(inport, arp.spa, arp.sha); next;)


  table=3 (lr_in_ip_input     ), priority=90   , match=(inport == "rtos-compute-16.ncw-az1-001.caas.bbtnet.com" && arp.op == 1 && arp.tpa == 100.69.6.1 && arp.spa == 100.69.6.0/23), action=(eth.dst = eth.src; eth.src = xreg0[0..47]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[0..47]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)


  table=13(lr_in_ip_routing   ), priority=71   , match=(ip4.dst == 100.69.6.0/23), action=(ip.ttl--; reg8[0..15] = 0; reg0 = ip4.dst; reg1 = 100.69.6.1; eth.src = 0a:58:64:45:06:01; outport = "rtos-compute-16.ncw-az1-001.caas.bbtnet.com"; flags.loopback = 1; next;)

From my test lab:



sh-5.1# ovn-sbctl lflow-list ovn_cluster_router|grep -i 10.128.2.0/23


  table=1 (lr_in_lookup_neighbor), priority=100  , match=(inport == "rtos-worker-0.testclustermroyovn2.lab.example.redhat.com" && arp.spa == 10.128.2.0/23 && arp.op == 1 && is_chassis_resident("cr-rtos-worker-0.testclustermroyovn2.lab.example.redhat.com")), action=(reg9[2] = lookup_arp(inport, arp.spa, arp.sha); next;)


  table=3 (lr_in_ip_input     ), priority=90   , match=(inport == "rtos-worker-0.testclustermroyovn2.lab.example.redhat.com" && arp.op == 1 && arp.tpa == 10.128.2.1 && arp.spa == 10.128.2.0/23), action=(eth.dst = eth.src; eth.src = xreg0[0..47]; arp.op = 2; /* ARP reply */ arp.tha = arp.sha; arp.sha = xreg0[0..47]; arp.tpa <-> arp.spa; outport = inport; flags.loopback = 1; output;)


  table=13(lr_in_ip_routing   ), priority=71   , match=(ip4.dst == 10.128.2.0/23), action=(ip.ttl--; reg8[0..15] = 0; reg0 = ip4.dst; reg1 = 10.128.2.1; eth.src = 0a:58:0a:80:02:01; outport = "rtos-worker-0.testclustermroyovn2.lab.example.redhat.com"; flags.loopback = 1; next;)


  table=13(lr_in_ip_routing   ), priority=69   , match=(ip4.src == 10.128.2.0/23), action=(ip.ttl--; reg8[0..15] = 0; reg0 = 100.64.0.6; reg1 = 100.64.0.1; eth.src = 0a:58:64:40:00:01; outport = "rtoj-ovn_cluster_router"; flags.loopback = 1; next;)
Since the issue is resolved for now, Could  you please login to ovnkube-node pod running on the compute 16 and 17 and run the below command and provide output of same in the case.
for compute16:
$ ovn-sbctl lflow-list ovn_cluster_router|grep -i 100.69.6.0/23
$ ovn-nbctl lr-route-list ovn_cluster_router |grep -i 100.69.6.0/23
for compute17:
$ ovn-sbctl lflow-list ovn_cluster_router|grep -i 100.69.0.0/23
$  ovn-nbctl lr-route-list ovn_cluster_router |grep -i 100.69.0.0/23
--
Regards,
Manish Roy
Red Hat 

PCAPS indicate that we ARE failing to send traffic from eth0 on the client pod to ENS192 on the node consistently for this NAT call to the upstream service. Every time a SYN leaves from the host nic, we do reply with a syn/ack. So the issue is in routing from eth0 (pod) to ens192 (node/NAT). --> OVNKUBE DB issue is likely.
Looking at the nat dump I do not see any duplicate/stale flows that pertain to this source pod IP: 100.69.0.44 - we've only got the one NAT entry and no other hosts have this IP logged.
It may be prudent to open a bug with engineering to get their opinion - largely I am concerned about the fact that a DB rebuild WAS performed on all nodes (including compute-17) several days ago, and again the behavior has returned (now presenting on this host).
Impact is somewhat limited (for now) but it's also not super clear that we're not going to see this behavior shift again to a new host. Engineering review could help with the NBDB and SBDB analysis from the network gather as well.
A direct NAT to the host net should basically always succeed so I'm wondering if there's something else unexpected happening with the network config on the node (bug or otherwise).
Engineering engagement might be warranted.
~WR