Adjusting OVN log levels using existing documentation does not change log level

4.13.z, 4.12.z, and 4.11.z

100%

$ oc -n openshift-ovn-kubernetes get po -l 'app=ovnkube-master' -ojsonpath='{..metadata.name}{"\n"}' | tr ' ' '\n' | while read pod ; do echo "---- $pod ----" ; oc -n openshift-ovn-kubernetes get po $pod -o json | jq '.spec.containers[] | select(.name == "ovnkube-master") | .name,.env' -c ; done
---- ovnkube-master-8fqpd ----
"ovnkube-master"
[{"name":"OVN_KUBE_LOG_LEVEL","value":"4"},{"name":"K8S_NODE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}]
---- ovnkube-master-nv5q4 ----
"ovnkube-master"
[{"name":"OVN_KUBE_LOG_LEVEL","value":"4"},{"name":"K8S_NODE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}]
---- ovnkube-master-qrmrh ----
"ovnkube-master"
[{"name":"OVN_KUBE_LOG_LEVEL","value":"4"},{"name":"K8S_NODE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}]

$ cat cmv1.yaml 
kind: ConfigMap
apiVersion: v1
metadata:
  name: env-overrides
  namespace: openshift-ovn-kubernetes
data:
  _master: |
    # This sets the log level for the ovn-kubernetes master process as well as the ovn-dbchecker:
    OVN_KUBE_LOG_LEVEL=2
    # You might also/instead want to enable debug logging for northd, nbdb and sbdb on all masters:
    OVN_LOG_LEVEL=dbg

$ oc create -f cmv1.yaml$ oc -n openshift-ovn-kubernetes get po -l 'app=ovnkube-master' -ojsonpath={..metadata.name} | tr ' ' '\0' | xargs -0 -I POD oc -n openshift-ovn-kubernetes delete pod POD 
pod "ovnkube-master-8fqpd" deleted
pod "ovnkube-master-nv5q4" deleted
pod "ovnkube-master-qrmrh" deleted

$ oc -n openshift-ovn-kubernetes get po -l 'app=ovnkube-master'
NAME                   READY   STATUS    RESTARTS        AGE 
ovnkube-master-4gqcn   6/6     Running   0               4m46s
ovnkube-master-ltvlv   6/6     Running   2 (3m32s ago)   4m48s
ovnkube-master-vrblh   6/6     Running   2 (4m4s ago)    4m50s
$ oc -n openshift-ovn-kubernetes get po -l 'app=ovnkube-master' -ojsonpath='{..metadata.name}{"\n"}' | tr ' ' '\n' | while read pod ; do echo "---- $pod ----" ; oc -n openshift-ovn-kubernetes get po $pod -o json | jq '.spec.containers[] | select(.name == "ovnkube-master") | .name,.env' -c ; done
---- ovnkube-master-4gqcn ----
"ovnkube-master"
[{"name":"OVN_KUBE_LOG_LEVEL","value":"4"},{"name":"K8S_NODE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}]
---- ovnkube-master-ltvlv ----
"ovnkube-master"
[{"name":"OVN_KUBE_LOG_LEVEL","value":"4"},{"name":"K8S_NODE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}]
---- ovnkube-master-vrblh ----
"ovnkube-master"
[{"name":"OVN_KUBE_LOG_LEVEL","value":"4"},{"name":"K8S_NODE","valueFrom":{"fieldRef":{"apiVersion":"v1","fieldPath":"spec.nodeName"}}}]
 

OVN_KUBE_LOG_LEVEL=4

OVN_KUBE_LOG_LEVEL=2

We can see the containers have /env/_master content and is sourced by the container, but is being overwritten or just not being applied:

$ oc rsh -c ovnkube-master ovnkube-master-ddcq9 cat /env/_master
# This sets the log level for the ovn-kubernetes master process as well as the ovn-dbchecker:
OVN_KUBE_LOG_LEVEL=2
# You might also/instead want to enable debug logging for northd, nbdb and sbdb on all masters:
OVN_LOG_LEVEL=dbg$ oc logs -c ovnkube-master ovnkube-master-ddcq9 | grep OVN_KUBE_LOG_LEVEL
++ OVN_KUBE_LOG_LEVEL=2$ oc rsh -c ovnkube-master ovnkube-master-ddcq9 env | grep OVN_KUBE_LOG_LEVEL
OVN_KUBE_LOG_LEVEL=4