Normal Description of problem:
In various OpenShift Container Platform 4.12 clusters (also earlier version) we see regularly the below messages reported even though the OpenShift Container Platform 4 - Control-Plane Node(s) look healthy and have enough resources. Further, also openshift-apiserver looks healthy and no sign of problem. Yet there appears to be problems when talking to openshift-apiserver. 2023-04-25T09:26:05.456861999Z E0425 09:26:05.456748 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/foo/routes?allowWatchBookmarks=true&resourceVersion=3049544273&watch=true" audit-ID="17df2f86-5504-4894-88fe-8f83e7830c65" 2023-04-25T09:26:05.754898161Z E0425 09:26:05.754789 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/bar/templates?allowWatchBookmarks=true&resourceVersion=3049544290&watch=true" audit-ID="f9142e12-8dcd-47ba-8330-d9d372674546" 2023-04-25T09:26:05.796985286Z E0425 09:26:05.796868 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/abc/routes?allowWatchBookmarks=true&resourceVersion=3049544292&watch=true" audit-ID="fc7f9c6b-af53-41fb-af6f-d723880d7c0f" 2023-04-25T09:26:05.952677485Z E0425 09:26:05.952580 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/xyz/templates?allowWatchBookmarks=true&resourceVersion=3049544298&watch=true" audit-ID="4baaa4cf-3542-4ba0-b4ac-6c9cd930f960" 2023-04-25T09:26:05.976663210Z E0425 09:26:05.976553 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/def/routes?allowWatchBookmarks=true&resourceVersion=3049544302&watch=true" audit-ID="31948a9b-61fb-4f68-bac0-549b552a6f38" 2023-04-25T09:26:06.013610251Z E0425 09:26:06.013488 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/foo/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544309&watch=true" audit-ID="87d0f4c3-0174-4104-ab64-fbde7ca4171b" 2023-04-25T09:26:06.329287187Z E0425 09:26:06.327531 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/apps.openshift.io/v1/namespaces/bar/deploymentconfigs?allowWatchBookmarks=true&resourceVersion=3049544335&watch=true" audit-ID="66f8a57f-8866-4607-b038-fc29ff0868fb" 2023-04-25T09:26:06.380052191Z E0425 09:26:06.377449 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/def/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544338&watch=true" audit-ID="eed22620-e44a-4745-84b3-ddd5d224852f" 2023-04-25T09:26:06.394277976Z E0425 09:26:06.394151 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/abc/templates?allowWatchBookmarks=true&resourceVersion=3049544340&watch=true" audit-ID="966f3946-35a0-48a1-a83e-cde576cabf84" 2023-04-25T09:26:06.453989174Z E0425 09:26:06.453875 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/xyz/routes?allowWatchBookmarks=true&resourceVersion=3049544342&watch=true" audit-ID="61a85cca-bb6c-4c56-89cf-e23b2ac24431" 2023-04-25T09:26:06.705008639Z E0425 09:26:06.704889 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/apps.openshift.io/v1/namespaces/abc/deploymentconfigs?allowWatchBookmarks=true&resourceVersion=3049544351&watch=true" audit-ID="6bd31193-c2fd-4b25-9d98-807a83974019" 2023-04-25T09:26:06.798081380Z E0425 09:26:06.797951 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/lmn/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544359&watch=true" audit-ID="12d13a73-bf51-4de5-82b2-8da07d97e389" 2023-04-25T09:26:06.858225902Z E0425 09:26:06.857918 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/build.openshift.io/v1/namespaces/bar/buildconfigs?allowWatchBookmarks=true&resourceVersion=3049544365&watch=true" audit-ID="943f1337-3ad5-4cf7-ae8a-543883657338" 2023-04-25T09:26:07.118518079Z E0425 09:26:07.118416 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/apps.openshift.io/v1/namespaces/xyz/deploymentconfigs?allowWatchBookmarks=true&resourceVersion=3049544384&watch=true" audit-ID="a9b95c61-9773-41e7-87f9-813e40d02459" 2023-04-25T09:26:07.596106858Z E0425 09:26:07.596004 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/hij/templates?allowWatchBookmarks=true&resourceVersion=3049544423&watch=true" audit-ID="f88a123f-d946-4eb2-b525-db0a35da9702" 2023-04-25T09:26:07.615173411Z E0425 09:26:07.615036 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/bar/routes?allowWatchBookmarks=true&resourceVersion=3049544432&watch=true" audit-ID="1f8146c7-d441-43c4-a220-5da39f858f30" 2023-04-25T09:26:07.804898790Z E0425 09:26:07.804785 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/xyz/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544451&watch=true" audit-ID="a7974474-48ca-47ea-80ad-7207e5d2864c" Currently there does not appear to be an impact on the overall OpenShift Container Platform 4 - Cluster but it would be good to understand the root cause for them so we can document it and also highlight whether it's actually a problem or not. Specifically in Red Hat Technical Support such errors are a problem as it's never clear whether those are expected and can be ignored or if they expose a problematic state and therefore need to be resolved.
Version-Release number of selected component (if applicable):
OpenShift Container Platform 4.10, 4.11 and 4.12
How reproducible:
Always
Steps to Reproduce:
1. Steps are not clear as they just show up and it's currently not clear what is triggering them
Actual results:
The below logs are reported by kube-apiserver when it tries to reach the aggregated openshift-apiserver 2023-04-25T09:26:05.456861999Z E0425 09:26:05.456748 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/foo/routes?allowWatchBookmarks=true&resourceVersion=3049544273&watch=true" audit-ID="17df2f86-5504-4894-88fe-8f83e7830c65" 2023-04-25T09:26:05.754898161Z E0425 09:26:05.754789 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/bar/templates?allowWatchBookmarks=true&resourceVersion=3049544290&watch=true" audit-ID="f9142e12-8dcd-47ba-8330-d9d372674546" 2023-04-25T09:26:05.796985286Z E0425 09:26:05.796868 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/abc/routes?allowWatchBookmarks=true&resourceVersion=3049544292&watch=true" audit-ID="fc7f9c6b-af53-41fb-af6f-d723880d7c0f" 2023-04-25T09:26:05.952677485Z E0425 09:26:05.952580 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/xyz/templates?allowWatchBookmarks=true&resourceVersion=3049544298&watch=true" audit-ID="4baaa4cf-3542-4ba0-b4ac-6c9cd930f960" 2023-04-25T09:26:05.976663210Z E0425 09:26:05.976553 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/def/routes?allowWatchBookmarks=true&resourceVersion=3049544302&watch=true" audit-ID="31948a9b-61fb-4f68-bac0-549b552a6f38" 2023-04-25T09:26:06.013610251Z E0425 09:26:06.013488 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/foo/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544309&watch=true" audit-ID="87d0f4c3-0174-4104-ab64-fbde7ca4171b" 2023-04-25T09:26:06.329287187Z E0425 09:26:06.327531 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/apps.openshift.io/v1/namespaces/bar/deploymentconfigs?allowWatchBookmarks=true&resourceVersion=3049544335&watch=true" audit-ID="66f8a57f-8866-4607-b038-fc29ff0868fb" 2023-04-25T09:26:06.380052191Z E0425 09:26:06.377449 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/def/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544338&watch=true" audit-ID="eed22620-e44a-4745-84b3-ddd5d224852f" 2023-04-25T09:26:06.394277976Z E0425 09:26:06.394151 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/abc/templates?allowWatchBookmarks=true&resourceVersion=3049544340&watch=true" audit-ID="966f3946-35a0-48a1-a83e-cde576cabf84" 2023-04-25T09:26:06.453989174Z E0425 09:26:06.453875 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/xyz/routes?allowWatchBookmarks=true&resourceVersion=3049544342&watch=true" audit-ID="61a85cca-bb6c-4c56-89cf-e23b2ac24431" 2023-04-25T09:26:06.705008639Z E0425 09:26:06.704889 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/apps.openshift.io/v1/namespaces/abc/deploymentconfigs?allowWatchBookmarks=true&resourceVersion=3049544351&watch=true" audit-ID="6bd31193-c2fd-4b25-9d98-807a83974019" 2023-04-25T09:26:06.798081380Z E0425 09:26:06.797951 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/lmn/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544359&watch=true" audit-ID="12d13a73-bf51-4de5-82b2-8da07d97e389" 2023-04-25T09:26:06.858225902Z E0425 09:26:06.857918 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/build.openshift.io/v1/namespaces/bar/buildconfigs?allowWatchBookmarks=true&resourceVersion=3049544365&watch=true" audit-ID="943f1337-3ad5-4cf7-ae8a-543883657338" 2023-04-25T09:26:07.118518079Z E0425 09:26:07.118416 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/apps.openshift.io/v1/namespaces/xyz/deploymentconfigs?allowWatchBookmarks=true&resourceVersion=3049544384&watch=true" audit-ID="a9b95c61-9773-41e7-87f9-813e40d02459" 2023-04-25T09:26:07.596106858Z E0425 09:26:07.596004 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/template.openshift.io/v1/namespaces/hij/templates?allowWatchBookmarks=true&resourceVersion=3049544423&watch=true" audit-ID="f88a123f-d946-4eb2-b525-db0a35da9702" 2023-04-25T09:26:07.615173411Z E0425 09:26:07.615036 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/route.openshift.io/v1/namespaces/bar/routes?allowWatchBookmarks=true&resourceVersion=3049544432&watch=true" audit-ID="1f8146c7-d441-43c4-a220-5da39f858f30" 2023-04-25T09:26:07.804898790Z E0425 09:26:07.804785 18 wrap.go:54] timeout or abort while handling: method=GET URI="/apis/image.openshift.io/v1/namespaces/xyz/imagestreams?allowWatchBookmarks=true&resourceVersion=3049544451&watch=true" audit-ID="a7974474-48ca-47ea-80ad-7207e5d2864c"
Expected results:
No timeout or aboard message reported by kube-apiserver when talking to the aggregated openshift-apiserver or at least have understanding whether the messages reported are indicating a problem or not.
Additional info:
