Upgrade is stuck in loop, As machine config rollout is not happening, 

The cluster is upgrade from 4.14 to 4.15 and post that mcp rollout happen 

- kube-apiserver operator was degraded due to the below file missing in the new revision pods.

E0212 10:25:36.580644      17 run.go:74] "command failed" err="failed to patch: unable to read External OAuth Metadata file: open /etc/kubernetes/static-pod-resources/configmaps/oauth-metadata/oauthMetadata: no such file or directory" 

- We have created the file manually from the older revision backup on all the master nodes.  

- After the kube-apiserver pods new revision got initiated and they started. 

- After that we have seen MCP rollout was stuck and openshift-authentication and openshift-oauth-authentication projects are in terminating state. 

- We have suggested to move ahead with the MCP rollout. So we have initiated the MCP rollout after seeing node stuck in draining failed status. 

  So manually drain the node which initiated the MCP rollout. 


$ oc adm drain compute-102 --delete-local-data --ignore-daemonsets --force --grace-period=0 --disable-eviction

MCP rollout got completed. But console and authentication operators are keep in degraded state. 


We have checked the authentication operator logs below is the error:
I0212 09:58:45.016926       1 event.go:298] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-authentication-operator", Name:"authentication-operator", UID:"7fd1cb03-83f1-4929-a385-d84c3b1bf0cb", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'OperatorStatusChanged' Status for clusteroperator/authentication changed: Available message changed from "OAuthServerRouteEndpointAccessibleControllerAvailable: failed to retrieve route from cache: route.route.openshift.io \"oauth-openshift\" not found\nOAuthServerServiceEndpointAccessibleControllerAvailable: service \"oauth-openshift\" not found\nOAuthServerServiceEndpointsEndpointAccessibleControllerAvailable: endpoints \"oauth-openshift\" not found\nWellKnownAvailable: The well-known endpoint is not yet available: failed to get oauth metadata from openshift-config-managed/oauth-openshift ConfigMap: configmap \"oauth-openshift\" not found (check authentication operator, it is supposed to create this)" to "OAuthServerRouteEndpointAccessibleControllerAvailable: failed to retrieve route from cache: route.route.openshift.io \"oauth-openshift\" not found\nOAuthServerServiceEndpointAccessibleControllerAvailable: service \"oauth-openshift\" not found\nOAuthServerServiceEndpointsEndpointAccessibleControllerAvailable: endpoints \"oauth-openshift\" not found\nWellKnownAvailable: route.route.openshift.io \"oauth-openshift\" not found"


We have tried to create the configmap in the openshift-config-managed namespace. But it rollout the kube-apiserver revision and the issue is running in the loop 

those 2 projects are keeping in terminating state

openshift-oauth-apiserver and openshift-authentication

    4.15

    NA

    1.
    2.
    3.
    

 Upgrade stuck

 It should be upgraded to the expected version,