Some edge deployments may have multiple network interfaces and different DNS servers to each different network. For example, a network with public access and a second one for local/private network. This use case is typically known as "Connect On-Demand" deployments, and ends up with 2 upstream nameserver, where one resolves ONLY local names and other resolves all public domain names. 

In such scenario, when multiple DNS servers are configured, openshift-dns (CoreDNS) pod fails to resolve local domain names: 

$ oc rsh -n appnamespace podname-85846613g-8gr10

$ nslookup redhat.com                      
Server:         10.43.0.10
Address:        10.43.0.10:53

Non-authoritative answer:                                              
Name:   redhat.com                                                         
Address: 34.235.198.240
Name:   redhat.com                                                          
Address: 52.200.142.250                                                                        

$ nslookup acmesample.local                                           
Server:         10.43.0.10                                                 
Address:        10.43.0.10:53                                            

** server can't find acmesample.local: NXDOMAIN                                                
** server can't find acmesample.local: NXDOMAIN                                                                                                                          
$ cat /etc/resolv.conf                                                                    
search appnamespace.svc.cluster.local svc.cluster.local cluster.local in.acme.com acmesample.local                                                                            
nameserver 10.43.0.10                                                                          
options ndots:5                                                                                

Also evaluated dnsmasq within NetworkManager, based on RHEL9 documentation https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/configuring_and_managing_networking/using-different-dns-servers-for-different-domains_configuring-and-managing-networking#using-dnsmasq-in-networkmanager-to-send-dns-requests-for-a-specific-domain-to-a-selected-dns-server_using-different-dns-servers-for-different-domains. 

After DNSmasq configuration, we can see that BaseOS processes and Podman containers are respecting the configuration, but MicroShift PODs and openshift-dns (CoreDNS) are still failing to resolve local domains.      

4.14+    

Always    

    1.Install RHEL 9.2+ and MicroShift 4.14+ 

    2.Create a file "/etc/NetworkManager/dnsmasq.d/dns.conf" with the following contents:
server=/acme.local/192.168.1002
server=/in.acme.com/172.10.10.100
server=172.10.10.100     

    3.restart the NetworkManager and verify that the contents of "/etc/resolv.conf"
     

`openshift-dns` pod fails to resolve local domain names. 

$ nslookup acmesample.local
Server:         10.43.0.10
Address:        10.43.0.10:53
** server can't find acmesample.local: NXDOMAIN
** server can't find acmesample.local: NXDOMAIN        

Name resolution for both networks, public and local.