-
Bug
-
Resolution: Done
-
Normal
-
None
-
4.9.z
-
None
This is a clone of issue OCPBUGS-224. The following is the description of the original issue:
—
Description of problem:
OCP v4.9.31 cluster didn't have the $search domain in /etc/resolv.conf, which was there in the v4.8.29 OCP cluster. This was observed in all the nodes of the v4.9.31 cluster.
~~~
OpenShift 4.9.31
sh-4.4# cat /etc/resolv.conf
- Generated by KNI resolv prepender NM dispatcher script
nameserver 172.xx.xx.xx
nameserver 10.xx.xx.xx
nameserver 10.xx.xx.xx - nameserver 10.xx.xx.xx
OpenShift 4.8.29
- Generated by KNI resolv prepender NM dispatcher script
search sepia.lab.iad2.dc.paas.redhat.com
nameserver 172.xx.xx.xx
nameserver 10.xx.xx.xx
nameserver 10.xx.xx.xx - nameserver 10.xx.xx.xx
~~~
ENV: OpenStack IAD2, IPI installation. Connected cluster.
Version-Release number of selected component (if applicable):
OCP v4.9.31
How reproducible:
Always
Steps to Reproduce:
1. Install IPI cluster on OpenStack IAD2 platform having cluster version 4.9.31
2. Debug to any of the node(master/worker)
3. Check and confirm the missing search domain on all nodes of the cluster.
Actual results:
The search domain was missing when checked in `/etc/resolv.conf` file on all nodes of the cluster causing serious issues in the cluster.
Expected results:
The installer should embed the search domain in /etc/resolv.conf file on all nodes of the cluster.
Additional info:
- Cu was trying to deploy secure Kerberos on the CoreOS nodes and it failed when the IPA-client install command failed. This is when the customer noticed this unusual behavior. They did not manually update the resolv.conf file to include the $search domain. They instead added the script below to /etc/NetworkManager/dispatcher.d/ and restarted NetworkManager on the node to fix this issue and installation was successful.
~~~
#!/bin/bash
set -eo pipefail
DISPATCHER_FILE="/etc/NetworkManager/dispatcher.d/30-resolv-prepender"
DOMAINS="$(grep -E '\s*DOMAINS=.*iad2.dc.paas.redhat.com' $DISPATCHER_FILE \
grep -oE '[a-z0-9]*.dev.iad2.dc.paas.redhat.com' \ |
tr '\n' ' ')" |
>&2 echo "IT-PaaS: overwriting search domains in /etc/resolv.conf with: $DOMAINS"
sed -e "/^search/d" \
-e "/Generated by/c# Generated by KNI resolv prepender NM dispatcher script \nsearch $DOMAINS" \
/etc/resolv.conf > /etc/resolv.tmp
mv /etc/resolv.tmp /etc/resolv.conf
~~~
- Cu confirms that the $search domain was missing since the cluster was freshly installed/ They even confirmed this with a fresh new cluster as well that it was missing.
- The fresh cluster was initially installed at v4.9.31 but was updated afterward to v4.9.43 (the latest z-stream) to see if the updates fixed anything but it didn't make any difference. The cluster is currently running v4.9.43 and shows the $search domain missing in the /etc/resolv.conf file on all nodes.
- blocks
-
OCPBUGS-2070 Missing $SEARCH domain in /etc/resolve.conf for OCP v4.9.31 cluster
- Closed
- clones
-
OCPBUGS-224 Missing $SEARCH domain in /etc/resolve.conf for OCP v4.9.31 cluster
- Closed
- is blocked by
-
OCPBUGS-224 Missing $SEARCH domain in /etc/resolve.conf for OCP v4.9.31 cluster
- Closed
-
OCPBUGS-1997 Missing $SEARCH domain in /etc/resolve.conf for OCP v4.9.31 cluster
- Closed
- is cloned by
-
OCPBUGS-2070 Missing $SEARCH domain in /etc/resolve.conf for OCP v4.9.31 cluster
- Closed
- links to